claim candidate goes to: this-edition T town planner
Who can pull the plug: an agent that cannot be stopped is riskier than one that can
Most AI projects do not fail technically; they stall when the business finally asks the good questions: can you stop it, what if it issues ten thousand of something, who is liable. Every board asks the same question about an agent in one sentence, who can pull the plug, how fast, what breaks and what cannot be undone, and a deployed agent's risk cannot be denied, only accepted by a named owner for an interval with an expiry. An agent that cannot be stopped is far riskier than one that can; containment (terminate, revoke, block, quarantine, ticket) is designed in advance and executed by the company's own stack.
Decided by. Nobody yet: a candidate for Dinis to read, comment on and decide (keep, rewrite, merge, cut, article first, later, not doing it).
Note. RiskMandate's position as the article states it: not inline, not an insurer, not a one-button solution; it 'makes agents insurable' and does not insure them; no risk transfer exists today.
Where it comes from
- Ten hard questions for RiskMandate, answered 4. How are we insuring or assuring agent behaviour?, 'An audit trail is not a guarantee'; 8. Remediation when a policy is bypassed; 10. Measurable business and technical outcomes, Getting to production (newsroom)
- What can your agent reach? RiskMandate and the Agent Behaviour Policy, explained for the person who has to decide How does it get into the room?, the board's risk appetite (newsroom)
What it answers in the 2018 book
- Don't touch production answered outline, Security
- Chaos Engineering answered outline, Concepts
Where it goes
Chapter Controlling your agent: RiskMandate, part Build it without a server (drafting); topic Controlling your agent: mandate, reach and hope versus enforcement.
Other ideas proposed for the same chapter: Mandate, reach and the gap: hope versus enforcement, Code is a liability; the future is the smallest code that does the job, Learning to hack is still how you learn how things work: draw the line where the attacker is, The secret goes from the drop into the tool, never into the chat, Security experience is what lets you ship with known gaps: list them, match them to the reader, say what would change the call, The agent behaviour policy is the due diligence document: a buyer reads the blast radius off it, and its absence says the most, A vault per conversation and per job: a leaked key exposes one small thing, not an inbox, The desktop is the barrier, not the model: prompt injection is not solved, so the box has to hold, Reach is a computed value, and new capability lands as excess, never as new authority, A third input: a behaviour policy beside the system prompt and the prompt, which the vendor acknowledges, enforces where it can and answers for, Models do not separate code from data, so a rule in the prompt can be argued with by an email, Aviation and payments got safe when the rule, the liability and the business model pointed the same way, Zoom into an agent's behaviour policy and you find the business logic, which only the business can write, Software was the law: the rule the screen enforced by omission is no longer enforced once an agent calls the API, Every rule lands in one of three states: backed by a control, governed by an accepted risk, or resting on hope, Write down what your agent can reach in twenty minutes: list its tools, say what you wanted it for, write the gap.
Node idea:who-can-pull-the-plug in graph.json; see it in the graph. Source: admin/content/graph/ideas.json.