Do your guardrails work when the agent reaches a consequential action?

What we test

The scope starts with the use case and the systems and models beneath it. We identify consequential actions, the authority required to take them, and the guardrails expected to constrain them. Business and control owners agree the scenarios and acceptance thresholds before testing begins.

Tests exercise the controls in the configured environment. We record permitted actions, denied actions, escalation paths and the evidence retained. Findings state the access available for testing, the conditions covered and any limitations that remain.

Aivance’s Governance Firewall describes a control pattern for consequential actions. At a defined enforcement threshold, it can trigger a Suspended Handoff State pending human ratification. Tests check the trigger, the information delivered to the reviewer and the outcome when the reviewer is unavailable. The pattern is selected according to the use case.

Failure scenarios to consider

Testing covers failure modes relevant to the use case and its materiality. Scenarios can include:

Hidden instructions in data. Test whether instructions in documents, emails or retrieved content can redirect an agent beyond its delegated task. Record which actions the guardrails block and which remain possible.

Access beyond the task. Attempt actions outside the agent’s approved tools, data permissions and transaction limits. Include related actions that can exceed a limit in aggregate.

Human intervention. Exercise approval, rejection, escalation, timeouts and unavailable reviewers. Check that the system reaches the agreed halt, ratification or fail-safe outcome.

Multi-agent coordination. Test delegation, shared resources and handoffs between agents. Check whether a sequence of individually permitted actions can breach the use case’s constraints.

Data leakage and unreliable output. Test sensitive data flows, unsupported claims and responses outside agreed boundaries. Record the effectiveness of the relevant filters, access controls and review mechanisms.

Third-party services are tested in the institution’s use-case context where access allows. Missing provider information and untestable controls remain explicit limitations in the findings.

References that inform test scenarios

IMDA’s Model AI Governance Framework for Agentic AI v1.5 informs scenarios for risk boundaries, human accountability, technical controls and user responsibility. It helps frame the questions we test in the configured environment.

MAS’s Safeguards for Agentic Finance at Runtime (SAFR) publication also informs runtime safeguard scenarios. MindForge’s Toolkit and Handbook provide industry practice references where relevant. Firms are not supervised against SAFR or MindForge. These references do not establish a certification or conformance target.

Aivance selects scenarios with your business and control owners. The test plan links each scenario to the risk, configured control, expected outcome and retained evidence.

What the assessment produces

The three-week engagement produces:

• A scope record linking each use case to its systems, models, providers and guardrails
• A test plan covering agreed failure modes and acceptance thresholds
• Observed results, configuration versions and supporting decision records
• Control findings, residual risks and prioritised remediation actions
• An evidence pack that second line, internal audit or a supervisor can review

We agree access and the testing environment before work begins. The findings identify any behaviour we could not test and the further evidence required.

After testing

Owners review failed controls and agree the work needed to reduce residual risk. Aivance can define remediation requirements and retest the affected scenarios after changes are made. Implementation responsibilities are agreed separately.

For the full supervisory context and phased dates, see the MAS AIRG guide.