Patterns from “Designing the Trust Management for Agentic AI,” a workshop held at the AI for Good Summit in Geneva, feeding into the Summit’s plenary and grounded in ITU Study Group 17’s work on AI security standardization.
The workshop drew practitioners from across the stack: security architecture, financial services risk, AI lab policy, systems integration, agentic tooling, and academic research. Different institutions, different incentives, different vocabularies for the same underlying problem. What stood out was not where the room disagreed. It was how many separate lines of argument arrived, unprompted, at the same shape of answer.
Human approval stops working at agentic speed
The clearest point of convergence concerned a safeguard most governance frameworks still treat as a default. Put a human in a queue approving agent actions thousands of times an hour, one line of argument went, and the human will eventually approve everything, because that is what happens when a review process is outpaced by the volume flowing through it. A second, independent line of argument reached the same conclusion from a different direction, naming human approval a fallacy specifically for agentic security, since the control cannot run at machine speed.
A more grounded account gave the pattern texture. Security operators, one perspective noted, are shifting from making decisions to supervising model decisions, a change in job function that current oversight and conduct processes were not built to support. Across every version of this argument in the room, from security engineering, from enterprise risk, from AI policy, the direction was the same. No speaker defended human approval as the primary control mechanism for autonomous agents operating at scale. Instead, every perspective shifted toward automated controls, continuous monitoring, or policy enforcement at machine speed.
Delegation is the object, not autonomy
One recurring argument held that the interesting shift in agentic AI is not automation, and it is not autonomy treated as a capability. It is delegation, specifically what happens once a delegated task passes from one agent to another, sometimes across organizational and jurisdictional lines, with no reliable way to confirm that scope and context traveled with it.
A sharper theoretical version of the same point placed agentic systems on a spectrum from automation, where tasks are tightly scoped and rule-based, to autonomy, where a task is deliberately left underspecified, closer to asking a trusted colleague to keep things running while you’re away than handing them a checklist. The value of agentic systems, and the unsolved governance problem, sits on the autonomy end, where an agent has to infer intent well enough to act on a vague instruction. That is described as unsolved even for a single agent acting alone, before any multi-agent handoff is added.
A product-oriented framing arrived at the same emphasis from a different angle, placing identity and delegation as the first layer of any control architecture, ahead of authority, audit, and containment, on the reasoning that trust cannot be established downstream of a delegation nobody can trace.
Capability is not trust
One of the clearest distinctions to emerge across the morning was that capability and trustworthiness are different properties, not two ends of the same scale. Capability describes what an agent can do. Trustworthiness describes whether people, organizations, and systems should rely on it, in a given context, at a given moment. That distinction did not surface only in the Q&A. It quietly underpinned most of the presentations that came before it, every mention of graduated autonomy, every argument for revocable trust, every insistence that a boundary has to be enforced rather than declared.
Three analogies made the point directly. Vehicle autonomy levels classify technical capability, but what actually made them meaningful to the public was the moment they marked a shift in legal responsibility, from driver to manufacturer. Public trust in elevators arrived not from a reliability improvement but from a visible demonstration that removed the safety net. And financial institutions never bin customers into discrete trust levels at all. They price trust continuously, adjusting credit exposure as behavior changes. Trustworthiness, on this reading, is not a fixed rating assigned once. It is a dynamic operational judgment, closer to a credit line than a certification.
Verification does not terminate on its own
Multiple lines of argument converged on continuous evidence generation as a prerequisite for trust, recording what an agent did, why, and under what authorization, as the raw material for any later judgment about whether it behaved. But one exchange exposed the deeper problem sitting underneath that idea. Recording trajectories and verifying agent behavior only moves the trust question one level higher if the verifier is itself another agent. Something then has to verify that verifier. At some point, verification has to terminate in a trusted root that is not itself agentic, or governance becomes an infinite regress, evidence checking evidence with no floor underneath it.
That is a harder problem than it first appears, because the room’s own instinct, build another layer of monitoring, is the instinct that produces the regress in the first place. The gap is not a missing layer. It is the absence of anything the chain of layers can terminate on.
The architecture emerging between the talks
No two arguments in the room used the same vocabulary. Identity, control planes, dimensions of trust, workforce risk, trajectories, credit exposure. Strip the vocabulary away and a shared architecture is visible underneath it, arrived at independently from security engineering, financial risk, AI policy, and academic research alike.
| Layer | The question each speaker was really answering |
|---|---|
| Identity | Who is acting? |
| Delegation | On whose behalf? |
| Authority | What is permitted? |
| Runtime enforcement | Are boundaries mechanically enforced, not just declared? |
| Evidence | Can actions be reconstructed continuously? |
| Accountability | Which humans remain responsible? |
Every one of those layers appeared repeatedly, from different disciplines, without coordination. What remains unresolved is not whether governance should operate continuously, most of the room had already accepted that. It is where execution-time decisions should ultimately be evaluated, and what constitutes a trusted root once delegated authority, shifting context, and machine-speed autonomy all intersect at once. That increasingly looks like the frontier for both standards bodies and enterprise implementations. The discussion repeatedly converged on identity, delegation, enforcement, and evidence. The remaining challenge is determining what should decide, at execution time, whether a delegated action is still admissible.