Runtime controls are where most current investment in AI agent security is going. That is not a criticism. It is the right response to a real problem. The teams standing up runtime controls are addressing the largest and most immediate risk category, and they are doing it well.

We build runtime controls too. Agent inventory, policy enforcement, input and output filters, real-time detection of misbehaviour. Our platform ships all of it. What follows is not a critique from outside. It is a description of what these systems do well, and the class of problem they cannot catch, written by people who build them.

What runtime controls actually do

At the highest level, runtime security for an AI agent is a set of checks placed between the agent and the world.

An input filter examines what enters the model. User prompts, tool results, retrieved documents, API responses. It looks for known patterns of prompt injection, sensitive data being exposed to the model, and content that violates policy. If it finds one, it blocks, sanitises, or flags.

An output filter examines what the model produces. Text going to a user, function calls headed for a tool, structured data being written to a system. It checks for policy violations, sensitive information being leaked, and outputs that break constraints the operator has set.

Policy enforcement sits between the agent and the tools it can call. When the agent decides to take an action or access a data source, policy enforcement checks whether that action is permitted for that agent, in that context, for that user. Actions that are not permitted are blocked. Actions that are permitted with conditions are gated.

Real-time detection watches the agent’s behaviour over time. It looks for patterns that indicate the agent is being manipulated, malfunctioning, or being used for something it was not deployed for. When a pattern appears, the system alerts, intervenes, or halts the agent.

Together these techniques form the perimeter around a deployed agent. They are the security controls the agent lives inside.

Where these controls do their best work

Most attacks against AI agents are variants of techniques that have been seen before. Prompt injection has a family of known forms. Jailbreaks follow patterns that adversarial ML research has documented. Data exfiltration attempts leave recognisable signatures. Runtime controls catch these variants, and the good ones catch most of them.

Policy violations that can be written down are enforceable. If an agent should not send email outside the company domain, that is a rule. If it should not call the payments tool without human confirmation, that is a rule. Runtime enforces the rules the operator can articulate.

Anomalies with statistical signals are detectable. If the agent’s tool call rate suddenly triples, or its data reads shift to sensitive tables it does not normally touch, those changes leave a trace. Runtime picks up the trace and acts on it.

None of this is trivial to build. All of it belongs in a serious deployment. The absence of these controls is a security posture failure. Their presence is the necessary floor.

The structural limit of a perimeter defense

Everything a runtime control does depends on recognising a pattern. The pattern can be a rule someone wrote, a signature encoded into a classifier, a distribution of behaviour learned from data, or a threshold on a metric. The form varies. The requirement does not. Something has to have identified the shape of the problem in advance, so that runtime can act on it when the problem occurs.

This works well for problems that fit that description. Two kinds of failure do not.

The first is the novel attack. An adversary using a technique that has not been catalogued yet, in a form the classifier was not trained on, producing signals that were not represented in the anomaly model. The whole discipline of AI security research is set up to close this gap. Each new class of attack is analysed, and defenses are added. But at any point in time, the current defenses are current, which is to say incomplete. The distance between what is known and what is being tried has never been zero, and against a determined attacker it will not be.

The second failure is deeper. An agent taking an action that is not on any denylist. That does not trip any anomaly detector, because the behaviour is inside the statistical envelope of normal use. That does not violate any policy the operator wrote, because the policy did not anticipate this situation. And the action is harmful anyway, because the agent’s training led it to that behaviour in a scenario nobody predicted.

There is no pattern to catch here. There is no rule to enforce. The runtime layer, which reasons about the traffic passing through the perimeter, has no way to intervene, because the harm is not in the traffic. It is in what the model has learned to do.

Runtime cannot see this. That is not a criticism of runtime. It is a description of what a layer that reasons about content and behaviour can and cannot address.

What the next layer has to address

Everything above still holds. Runtime belongs in every deployment. What we build should be built. What comparable teams are building should be built. The failure modes runtime does address are large, real, and urgent.

The failure mode runtime cannot address is a different kind of problem. It cannot be solved by better rules, because the rules have to be written in advance. It cannot be solved by better classifiers, because there is nothing yet for the classifier to be trained on. It has to be solved somewhere else in the stack.

The next post looks at evaluation and red-teaming, and where those systems reach their own limit.