Governing AI agents that touch money: the agent proposes, it never executes
15 Jul 2026
One rule holds the whole design up: the agent can propose, it can never act on anything material. Every write — a payment, a journal entry, an account change — leaves the model's reach and passes through an authorization service the model cannot author or bypass, and then a human with the right role. Governance built as architecture, not as a carefully worded prompt.
Why "just prompt it carefully" fails
If the model holds a tool that can move money, then the safety of your money depends on the model's judgment on every single call — including the calls where a retrieved email, a poisoned document, or an ambiguous instruction pushed it somewhere you didn't intend. A prompt is a request for good behavior. It is not an enforcement boundary. The moment a material action is reachable from inside the model's tool surface, you've made the model your last line of defense — and the model is precisely the part you can't fully predict.
So the fix isn't a better prompt. It's to make the dangerous action unreachable from where the model lives.
The one rule, and how each guarantee is enforced
The agent reads from least-privilege, read-only sources and drafts a proposed action. That proposal — never a direct write — is all it can emit. Everything that matters is enforced in a layer the agent has no tool to reach:
- No unauthorized payment or journal. Write tools are not in the agent's tool surface at all. Only the authorization service can reach them, and only after a role-checked human approval.
- Segregation of duties. The identity that requests an action can never be the identity that approves it — enforced in the authorization service, not left to convention.
- Every figure is traceable. Each number carries a lineage pointer — source document, page, retrieval citation. No lineage means it's shown as "unsourced", never presented as fact.
- Prompt injection can't act. Retrieved emails and documents are untrusted data, never instructions; tool use is allowlisted; outputs are schema-validated. A malicious document can lie to the model, but it cannot reach a write.
- Bounded blast radius. Each integration runs on its own least-privilege, read-only credential — one compromise is one read-only surface, with no lateral movement to the rest.
A worked example: where money actually escapes an escrow flow
The same instinct — assume the dangerous path will be taken, and design so it's contained — is what separates an escrow flow that holds from one that quietly leaks. Two parties negotiate, they agree, a hold goes on the card. Drawn end to end, here are four places the money actually escapes, and the design answer for each:
- "Zero-UI" vs strong authentication. An off-session charge on a 3DS card comes back "authentication required" — and there is no screen to authenticate on, so the hold silently never happens. The answer: establish the mandate before negotiation (a setup step that saves an off-session payment method), plus a defined fallback for cards that still challenge.
- Hold expiry. An uncaptured authorization lapses in roughly a week. If your capture fires after the appointment and the booking was far enough out, the hold is already dead. The answer: bound how far ahead a slot may be booked, or re-authorize near the appointment — and treat "released because visited" and "expired because we were late" as two different events.
- Webhook retries. Payment webhooks are delivered at-least-once — the same event will arrive twice, and without dedup a retry captures the deposit twice. Worse, an unverified endpoint is a public "capture this hold" button. The answer: verify the signature against the raw body, persist each event id and drop repeats at the database, and send idempotency keys on capture and release.
- A hashed phone number is not anonymous. Phone numbers live in a tiny space; a plain hash of one is brute-forced offline in seconds, so a "hashed key" leaks identity to anyone who reads the table. The answer: a keyed hash with a server-held secret — not reversible without the secret.
The through-line
Whether it's an accounting agent or a payment flow, the discipline is the same: keep the irreversible action out of reach of the part you can't fully trust, make every consequential number carry its source, give each integration only the access it needs, and assume the ugly path will be taken so you design for it up front. It's the same rule I apply on the industrial side — the safety-critical limit is re-checked on the controller, never trusted from the layer above. Governance you can bypass isn't governance.
This is how I design these systems — the pattern comes from years of backend work and from building production LLM pipelines and MCP tool-calling systems where a wrong write has a real cost.
This argument is runnable. I put it in a public repo: an authorization service the agent cannot call, content-bound single-use approvals, segregation of duties, lineage-required amounts, an append-only hash-chained audit log, and a mock provider that reproduces all four escrow failure modes above. Then twelve attacks try to move money anyway — prompt injection, self-approval, an amount swapped after approval, a replayed webhook, and an "anonymised" phone hash brute-forced in a few thousand tries. All twelve fail, in about a second, with no dependencies — and the harness runs in CI on every push. Code →
If a machine you build needs an interface, a device connection, or data that has to land somewhere else, tell me what it's costing you now. You'll get an honest read on whether it's solvable, and usually something running to look at. Start here →