evidence candidate goes to: this-edition T town planner
Aviation and payments got safe when the rule, the liability and the business model pointed the same way
Aviation did not get safe out of altruism but because every crash became news: at 1995's rate of 2.72 fatal airliner accidents per million flights, 2025's 38.7 million flights would have meant about 105 fatal accidents; there were 12. Payments got some of the best security teams anywhere because the liability was moved onto the companies: a US cardholder liable for at most $50 since 1970, £35 in the UK, the 2015 liability shift onto whoever had not adopted chip cards, the PCI council founded by the card brands in 2006, and UK banks reimbursing scam victims up to £85,000 since October 2024, after which losses fell by around 21%. Regulation alone does not do it (Meta's €1.2 billion GDPR fine in 2023, €200 million in 2025); the improvement comes where a rule, a liability that lands on the company and a systemic risk to the business model line up, and agents can be that case.
Decided by. Nobody yet: a candidate for Dinis to read, comment on and decide (keep, rewrite, merge, cut, article first, later, not doing it).
Note. The article's own caveats: the aviation numbers mix sources (Aviation Safety Network via Our World in Data, the Flight Safety Foundation, IATA); the 21% fall is one evaluation's estimate; 'it does not take many users' to lose trust in agents is Dinis's judgement with no number behind it.
Where it comes from
- A policy beside the prompt Aviation did not get safe out of altruism; Payments got safe because liability moved; Regulation alone does not do it (newsroom)
What it answers in the 2018 book
- Bug Bounties answered outline, Security
- GDPR answered outline, Concepts
Where it goes
Chapter Controlling your agent: RiskMandate, part Build it without a server (drafting); topic Controlling your agent: mandate, reach and hope versus enforcement.
Other ideas proposed for the same chapter: Mandate, reach and the gap: hope versus enforcement, Code is a liability; the future is the smallest code that does the job, Learning to hack is still how you learn how things work: draw the line where the attacker is, The secret goes from the drop into the tool, never into the chat, Security experience is what lets you ship with known gaps: list them, match them to the reader, say what would change the call, The agent behaviour policy is the due diligence document: a buyer reads the blast radius off it, and its absence says the most, A vault per conversation and per job: a leaked key exposes one small thing, not an inbox, The desktop is the barrier, not the model: prompt injection is not solved, so the box has to hold, Reach is a computed value, and new capability lands as excess, never as new authority, Who can pull the plug: an agent that cannot be stopped is riskier than one that can, A third input: a behaviour policy beside the system prompt and the prompt, which the vendor acknowledges, enforces where it can and answers for, Models do not separate code from data, so a rule in the prompt can be argued with by an email, Zoom into an agent's behaviour policy and you find the business logic, which only the business can write, Software was the law: the rule the screen enforced by omission is no longer enforced once an agent calls the API, Every rule lands in one of three states: backed by a control, governed by an accepted risk, or resting on hope, Write down what your agent can reach in twenty minutes: list its tools, say what you wanted it for, write the gap.
Node idea:aviation-and-payments-got-safe-when-liability-landed in graph.json; see it in the graph. Source: admin/content/graph/ideas.json.