There's a demo everyone does when they want to show they've "tested the AI." They type a clever prompt, the chatbot says something it shouldn't, and everyone nods. It makes for a good screenshot. It is also almost entirely beside the point.
We've spent enough time inside real agent deployments now to say this plainly: getting a model to produce a rude sentence is not the risk. The risk is what happens the moment an AI system can act — call a tool, move money, update a record, email a customer, open a ticket in a downstream system. The instant it can do things, it stops being a chatbot and becomes an application with agency. And agency without tight boundaries is how small mistakes become expensive ones.
An agent is a system, not a text box
When people say "we're securing our AI," they usually mean they're securing the model. But a production agent is a stack, and every layer of it is an attack surface.
There's the model itself, vulnerable to prompt injection and system-prompt leakage. There's the retrieval layer — the documents and knowledge bases it reads — which can carry indirect prompt injection, where the malicious instruction isn't typed by a user at all but planted in a file the agent ingests. There's memory, which can persist a hostile instruction across sessions. There are the tools and connectors it's wired to, each with its own permissions. And underneath all of it is an identity — a service account with a set of things it's allowed to touch.
Test only the model and you've inspected the front door while leaving the loading dock open. In our assessments, the interesting findings almost never live in the chat interface. They live in the plumbing behind it.
Indirect prompt injection is the one to understand
If you take one technical idea away, make it this. Direct prompt injection — a user typing something malicious — is real but visible. Indirect prompt injection is subtler and, in our view, more dangerous.
Imagine an agent that summarises incoming support emails and can take actions based on them. An attacker doesn't need to compromise anything. They just send an email containing text crafted to be read by the agent, not the human: instructions telling it to forward account details, or trigger a refund, or ignore its previous constraints. The agent reads the email as data and executes it as a command, because to a language model the line between the two is far blurrier than we'd like.
We've reproduced variants of this against systems whose owners were confident they were safe. The fix is never a cleverer prompt. It's architecture — treating everything the agent reads as untrusted, constraining what it can do with what it reads, and putting hard limits between "the agent decided something" and "the agent did something irreversible."
Excessive agency is the headline, every time
Strip away the jargon and nearly every serious agent risk we've found reduces to one sentence: the agent could do more than it needed to.
An agent scoped to read a customer's order history is a contained problem even if someone manipulates it. An agent that can read order history, issue refunds, modify accounts, and send emails — all under a single over-privileged service identity, with no human approval gate on the consequential actions — is a catastrophe waiting for the right prompt. Same model, same vulnerabilities, wildly different blast radius.
This is why we push so hard on least privilege for agents specifically. Give the agent the narrowest identity that lets it do its job. Put approval gates in front of anything that moves money or changes state irreversibly. Bound its tool access and its transaction limits. Log what it does in a way you can actually reconstruct later. A well-constrained agent that gets manipulated fails safe. An over-permissioned one fails all the way.
What real agentic testing covers
When we red-team an agent properly, the prompt tricks are maybe a fifth of the work. The rest is the boring, essential probing of the whole chain.
We test whether the agent can be induced to take actions beyond its intended scope. We test the tool calls, the transaction limits, and whether the human-approval controls can be bypassed or fatigued into uselessness. We test indirect injection through every channel the agent reads from. We look at the supply chain — the models, the data, the dependencies. And we look hard at the identity: what could an attacker reach if they got the agent to act on their behalf?
That last question is the one that tends to change the room. It's abstract until you show a team that their helpful little assistant, under the right nudge, could reach a system nobody expected it to touch.
Do this before production, not after
The uncomfortable pattern we keep seeing is organisations shipping agents into production on the strength of a demo, then treating security as something to bolt on after the incident. AI moves fast and the pressure to launch is real, especially with the UAE's push toward AI across so many sectors. But an agent with real permissions is a piece of production infrastructure, and it deserves the same adversarial scrutiny you'd give any other system that can act on your behalf — arguably more, because its behaviour is probabilistic and its failure modes are still being discovered.
The good news is that the discipline here isn't exotic. It's the same instinct that's always separated secure systems from fragile ones: assume the input is hostile, give the component only what it needs, and put a human in the loop before anything irreversible happens. Agents just make the cost of forgetting those lessons a lot higher.
A quick self-test for your own agents
You don't need a full red team to get a feel for your exposure. Sit down with whoever built the agent and ask a few uncomfortable questions. What identity does it run as, and what is that identity actually allowed to reach — not in theory, but if you listed every permission? What's the worst single action it can take without a human approving it, and are you comfortable with that action happening because of a cleverly worded email? Does it read from any source an outsider can influence — inbound messages, uploaded documents, third-party data — and if so, what stops instructions hidden in that data from being obeyed? Can it be made to loop, spend, or act far more than intended, and is there a ceiling that stops it? If the answers make the room go quiet, that's your finding. Most teams have never mapped their agent's real authority end to end, and the exercise of doing so surfaces more risk than any single test. It won't replace adversarial testing before you rely on the thing, but it will tell you honestly whether you're closer to "well-bounded assistant" or "over-trusted intern with the keys."
If you're putting an agent into production — or you already have — it's worth having someone try to break it on purpose first.