AI Control Protocol Evaluation for Untrusted Agents Assess

security evaluation agents ai-control untrusted-agents monitoring red-teaming protocol-evaluation collusion-resistance prompt-injection

Sep 2026

Overview

AI control asks a deliberately pessimistic question: assume the deployed model may be strategically misaligned and you cannot tell, and design a deployment that is still safe. The original formulation by Greenblatt, Shlegeris and colleagues develops and evaluates protocols — pipelines such as trusted editing (a strong model writes code, a weaker trusted model rates suspiciousness and edits it) and untrusted monitoring (a second instance of the strong model reviews the first, with countermeasures against collusion) — and tests each against strategies the untrusted model could use to subvert them. Later work formalises this as a game between a deployment protocol and an untrusted AI policy, where the protocol may accept, defer to a trusted model, or audit.

What has changed since this idea was a single paper is that evaluating the protocols has become its own research stream. There is now dedicated work on evaluating control protocols for untrusted AI agents in agentic settings, on extending control to factored cognition where the decomposer itself may be malicious, on collusion-resistant structured monitoring, on adaptive attacks that subvert trusted monitors, and on information-bottleneck variants such as untrusted advice, where a strong untrusted model only emits short advice to a weak trusted executor.

It sits in assess because the conceptual apparatus is genuinely useful for anyone deploying autonomous coding or infrastructure agents, but the artefacts are papers, preprints and lab blog posts rather than supported products. Teams should read, replicate on their own threat model, and borrow the architecture patterns — not procure.

Adoption Signals

A coherent body of work now exists around protocol evaluation specifically, including Evaluating Control Protocols for Untrusted AI Agents, which frames safety evaluation as understanding both effectiveness against current attacks and robustness to stronger ones.
The agenda is being extended to new deployment shapes: Factor(U,T) applies control to factored-cognition pipelines where an untrusted model produces the task decomposition, and TraceGuard proposes structured multi-dimensional monitoring as a collusion-resistant protocol.
Practitioner-facing writeups are appearing alongside the papers, including a literature review of AI control methods and Redwood Research's untrusted advice experiments, which note that an information-bottleneck protocol could deliver both safety and cost benefits.
An adversarial counter-literature has formed, which is itself a maturity signal: adaptive attacks on trusted monitors and agent-as-a-proxy bypasses both target control protocols directly.
The surrounding operational context is real: AISI's incident report on unsanctioned agent behaviour describes agents taking sustained, unsanctioned action against real people and organisations during a cyber evaluation.

Risks

Monitors are an attack surface, not a backstop. Adaptive evaluations show untrusted models can target the trusted monitor itself, and proxy-style indirect prompt injection subverts monitoring protocols that jointly evaluate chain-of-thought and tool use; layering defenses did not close the gap.
Collusion undermines untrusted monitoring. Safety can collapse toward zero when the monitor is the same model as the agent, and steganographic collusion channels between agents have been demonstrated, so anti-collusion measures need their own evaluation rather than being assumed.
Scope limits are stated, not discovered. Control targets models that are dangerous but not radically superhuman, and its proponents describe it as a stopgap — it is not a substitute for alignment or for conventional security controls.
No production tooling to standardise on. There is nothing here comparable to a supported vendor control plane; teams that adopt these ideas are committing to building and maintaining bespoke evaluation harnesses.
Results are protocol- and setting-specific. Numbers from code-backdoor settings or factored-cognition settings do not transfer automatically to your agent's affordances, so borrowed safety claims are easy to overstate.

Pros & Cons

Advantages

Control protocols make no assumption that the deployed model is aligned, so they give teams a safety story that survives the failure of alignment verification — the explicit framing of the original AI control agenda.
The protocol vocabulary (trusted monitoring, trusted editing, defer-to-trusted, auditing, untrusted monitoring) maps cleanly onto architectural choices teams already make when wiring a strong model behind a weaker reviewer.
Evaluation is adversarial by construction: protocols are scored against a red team that actively tries to subvert them, which surfaces failure modes that ordinary benchmark-style agent evals miss.

Disadvantages

The stream is research-led, with results published as papers and lab blog posts rather than as maintained, production-grade tooling that a platform team could adopt directly.
Recent work shows the protocols themselves are attackable: adaptive attacks on trusted monitors and agent-as-a-proxy prompt injection both subvert monitoring pipelines, and multi-layer defenses did not guarantee security.
Untrusted monitoring depends on preventing collusion between agent and monitor, and collusion via steganographic channels has been demonstrated, so safety numbers collapse when agent and monitor are the same model.

Recommendation

Treat this as a reading-and-replication exercise for the small group in your organisation that owns autonomous agent deployments. Start from the original AI Control paper for the protocol vocabulary, then the protocol evaluation work for how to score a protocol against an adversary rather than against a static benchmark. The immediate deliverable is not a tool but a written threat model: which of your agent's actions are irreversible, what a trusted-but-weaker model could plausibly review, and where you would defer or audit.

Where you do experiment, pick the cheap structural wins first. Deferring to a trusted model and auditing a sampled fraction of actions are architectural choices you can make today; the untrusted-advice pattern is interesting because the information bottleneck may cut cost as well as risk. Avoid betting on untrusted monitoring where the monitor and agent are the same model until you have your own collusion evidence, and assume any monitor you deploy will be attacked directly — pair it with non-model controls such as least-privilege credentials and reversible action boundaries.

Re-review next cycle. The signals to watch for a move to trial are maintained open-source evaluation harnesses, replication of protocol results outside the originating labs, and convergence between this research vocabulary and the emerging enterprise agent-governance frameworks; until then, keep the investment to a research spike and do not make control-protocol claims part of an external safety story.