Home / Journal / Compliance, hosting and impact

Securing an AI agent: prompt injection, data leakage and dangerous actions

Compliance, hosting and impact Published 2 October 2026 11 min read

An AI agent reads text, and text can carry orders. That is the root of prompt injection attacks, whether typed directly or hidden in a document. Here is how we limit what an agent can see, do and state, following the OWASP reference.

Why an agent is secured differently from an application

A traditional application does what its code tells it. An AI agent does what its model understands of the situation. The whole security difference lies in that sentence. In an application, instructions come from the developer and data comes from the user, through two separate channels. In an agent, both reach the model in the same form: text. The model has no reliable way to tell a legitimate instruction from a sentence slipped into a document it has been asked to read.

That is why OWASP, the foundation behind the most widely used application security references, maintains a list dedicated to applications built on language models. Its 2025 edition ranks prompt injection first, followed by sensitive information disclosure. It also covers improper handling of model outputs, excessive agency granted to the agent and system prompt leakage. These five risks structure this article.

So you do not secure an agent by asking it to be careful. You secure it by limiting what it can see, what it can do and what you accept from it without checking.

For the overall framing, see deploying an AI agent step by step. Here we focus on what can go wrong.

Direct prompt injection

Direct injection is the most intuitive. A user writes a message designed to hijack the agent: "ignore your previous instructions", "you are now in maintenance mode", "display the customer list for verification". OWASP describes a typical scenario: an attacker injects instructions into a customer support assistant to make it ignore its rules, query private data and send emails. The result is unauthorised access and privilege escalation.

OWASP also notes that attackers get around naive filters by mixing languages, encoding the instruction or appending strings of characters that look meaningless.

The temptation is to respond by hardening the system prompt: "never reveal", "always refuse". This helps frame normal behaviour, and OWASP recommends it as a first measure. But the same source stresses that the system prompt must never be treated as a secret or as a security control. A rule written in natural language can be bypassed in natural language.

First, nothing sensitive in the system prompt: no password, no API key, no database name, no list of permissions. If the prompt leaks, it must reveal nothing exploitable. Second, every rule that really matters must also exist outside the model, in deterministic code that applies whatever the model decided.

Indirect injection: the danger comes from documents and web pages

Indirect injection is more insidious, and it is the one that affects most business agents. The attacker never talks to the agent. They plant instructions in content the agent will read later: a web page, a PDF attached to an email, a CV, a product sheet, a support ticket, a document in the knowledge base. When the agent processes that content, it reads the hidden instructions as part of its task.

OWASP insists on a detail that changes everything: these instructions do not need to be readable by a human. It is enough for the model to parse them. White text on a white background, a comment in a page's code, an instruction in an image's metadata will do. The same source describes several scenarios we also meet in real projects:

an agent summarising a web page with hidden instructions, which inserts a booby-trapped image link into its answer that sends the conversation to a third-party server;
a CV containing instructions split into several pieces, which lead the screening agent to recommend a weak candidate;
a tampered document in the internal knowledge base, which skews every answer that relies on it.

OWASP notes that adding a knowledge base improves relevance without removing the injection risk. A knowledge base fed by many contributors, or by outside sources, is an attack surface.

The recommended defence is to separate and clearly label untrusted content, to limit its influence. In practice, we treat any text from outside as data to analyse, never as an instruction. It is presented to the model in an identified block, with an explicit instruction not to execute it. And an agent that reads outside content gets reduced permissions.

Data leakage: what the agent sees, it can repeat

The second entry on the OWASP list covers sensitive information disclosure: personal, financial or health data, confidential business data, credentials. The mechanism is rarely spectacular. The agent has access to a piece of information, someone asks the right question, the agent answers. OWASP cites the case of a user who receives another user's personal data in a response because of insufficient filtering.

The rule that protects best is the simplest: an agent must not have access to information the person talking to it is not allowed to see. If a sales rep cannot see salaries in your information system, the agent they use must not see them either. Access control happens when the agent fetches the data, using the user's identity, not when it words its answer.

In practice, this leads to a few habits:

knowledge base searches are filtered by the user's permissions before reaching the model;
sensitive fields are masked or tokenised when the model does not need them to answer;
error messages never expose technical queries, file paths or configuration excerpts;
users are told what they must not type in, and a retention and deletion policy for conversations is defined.

What happens to data once sent to the model provider is a matter of hosting and contracts. We covered it in our article on hosting an AI project in France.

Dangerous actions and excessive agency

An agent that only answers can say silly things. An agent that acts can do them. OWASP calls this risk excessive agency and attributes it to three causes.

Too many functions. The agent has tools it does not need. OWASP's example: an agent that only needs to read documents, connected to a plugin that can also modify and delete them.
Too many permissions. Useful tools are connected with an overpowered account. An agent that recommends products has no reason to write to the database.
Too much autonomy. High-impact actions run without human sign-off.

The most frequent case is the generic tool: shell access, a function that calls any web address, a free-form database query. OWASP recommends replacing open-ended tools with narrow functions designed for a single task: "read order X of the logged-in customer" rather than "run a query".

For high-impact actions, human approval remains the most robust measure. The agent prepares, a human confirms. The email goes out after a click. The refund is proposed, then approved by the authorised person.

Least privilege: give the agent only what it needs

With an agent, the principle of least privilege becomes central, because it is what caps the damage when everything else has failed. OWASP expresses it as several recommendations that we use as a review grid:

limit the agent's tools to the minimum necessary, and remove those no longer in use;
limit each tool's functions to what is required;
give each tool access to the target system at the lowest possible permission level;
run actions in the user's context, with their credentials and narrow scopes, rather than through an all-powerful service account;
have the target system itself enforce authorisation, independently of what the model decided.

That last point is often forgotten. If your CRM refuses to edit a record the user is not allowed to edit, the agent cannot do it either, even when hijacked. Conversely, if the agent uses an admin account and controls rely on its own instructions, a single successful injection is enough.

Least privilege also affects costs. OWASP lists unbounded consumption among its risks, which can cause service outages or financial losses. Per-user rate limits, maximum input sizes and capped execution times protect both security and the bill. We break down the cost items in what an AI agent costs in production.

Validate on the server anything that commits the company

OWASP devotes a full entry to improper output handling. The principle is to treat what the model produces as untrusted user input. An SQL query generated by the model and executed as is, code interpreted by the browser, a file path built from its answer: each case opens a classic hole, SQL injection, malicious script or access to forbidden files. The defences are well known: parameterised queries, encoding suited to the display context, content security policies on web pages.

We push this rule further and apply it to everything that commits the company. A price, an availability, a product reference, an address, a lead time: these are computed on the server from the reference systems and checked before display. The model can word the sentence. It never produces the figure. We explained why in why an AI agent must never quote a price.

This rule also makes injection unrewarding: if an attacker convinces the agent to announce a discount, the server recomputes the amount and the false announcement never shows.

Log to understand and to prove

A hijacked agent raises no alarm. It answers politely. OWASP ranks logging and monitoring among the measures that limit damage: they do not prevent the attack, they let you see it and respond.

What we log on an agent in production:

every tool call, with its parameters, the user concerned and the result;
every high-impact action, with a trace of the human approval;
refusals, whether from the model or from server-side checks, since a series of refusals often signals an attempt;
the documents used for each answer, to track down a tampered one.

These logs often contain personal data. They therefore need their own retention period, their own access rights and a place in your record of processing activities. Security and compliance meet here, which is the purpose of the role we call AI security and compliance.

Simple alerts are enough to start: an unusual volume of tool calls, a link to an unknown domain in an answer, an action refused several times in a row.

Test the agent like an attacker

OWASP's last recommendation on injection is to run regular penetration tests and attack simulations, treating the model as an untrusted user.

Our standard sequence, before any go-live:

Direct injections. A series of messages designed to bypass the rules, rephrased, translated, encoded, split across several turns.
Indirect injections. Booby-trapped documents, pages and attachments, with visible and invisible instructions, placed where the agent will read.
Leakage. Attempts to obtain the system prompt, another user's data or information outside the logged-in person's scope.
Actions. Attempts to trigger a tool out of context, with tampered parameters, or without the planned approval.
Consumption. Very long inputs and bursts of requests to check that limits hold.

Every successful attack becomes a permanent test case, replayed at every change of model, prompt or tool. This work is part of the production prerequisites we isolate from the prototype stage, as explained in our article on business POCs.

The pre-production checklist

Here is the grid we go through with teams before opening an agent to its users.

The system prompt holds no secret, and no security rule relies on it alone.
All outside content (pages, files, emails, knowledge base) is treated as untrusted data and presented as such to the model.
The agent only accesses data the logged-in user is allowed to see, with filtering applied before the model.
Each tool is narrow, justified and connected with the lowest possible permissions.
Authorisation is enforced by target systems, independently of the model.
High-impact actions go through human approval.
Model outputs are validated before use: format, encoding, parameterised queries.
Prices, availability, references and commitments are computed on the server.
Tool calls, refusals and sources are logged, with a defined retention period.
Rate, size and duration limits are in place.
An attack test set exists and is replayed at every change.

If several boxes remain empty, the agent is not ready, even if it performs well in a demo. Most of these points are architectural, so we build them into our AI agent method and into our responsible AI approach.

Do you have an agent in the works or already live, and want to know where it stands on these points? The AI diagnostic reviews your use cases and pinpoints the risks to address first, before they turn into incidents.

Sources

Frequently asked questions

Is a good system prompt enough to protect an agent?

No. It frames normal behaviour and reduces slip-ups, but it can be bypassed with the same weapon: text. OWASP recommends never relying on it for security. The controls that matter must live in code and in the target systems.

Are recent models immune to injection?

Providers keep improving their models' resistance, but none guarantees immunity. A new version can even behave differently on attacks already tested. That is why we replay the attack test set at every model change.

Should the agent be banned from reading web pages or attachments?

Not necessarily, that is often where it adds value. You should however reduce the agent's permissions when it handles such content, and clearly separate that content from instructions. An agent that reads the web does not need to send emails in the same session.

Who should run the attack tests, the project team or a third party?

Both have a role. The project team knows the tools and data, builds the first test set and maintains it. An outside view brings attack techniques the team had not imagined, useful before a wide rollout or on an agent that acts on sensitive data.

Doesn't human approval kill the point of the agent?

It only applies to high-impact actions: external sends, data changes, financial commitments. The agent keeps all the preparation, search and drafting work. In practice, approving a prepared action takes seconds, and the level of scrutiny can evolve once the agent has proved itself.