The Keys Test
Ten questions to ask before you hand an AI agent real infrastructure
You're about to hand an AI agent real keys — to your hypervisor, your backups, your mail gateway, something you'd feel losing. Every tool on the shelf says "safe." The word is free. Receipts aren't.
This is the test. Ten questions. A tool either answers them or it doesn't, and most answers are checkable in an afternoon with the tool's own docs open. If the docs go quiet on one — that's an answer too.
I build one of these tools, so the rules apply to me first: mine is scored at the bottom, including the questions where my honest answer is "partial." Score anything else the same way. The test is yours.
The questions
1. Before it acts, can you see what it's about to do?
Not a log after. A plan before — what changes, what's in the blast radius, what it touches that you didn't think of. If the first time you learn what the agent did is after it did it, you're not operating a tool; you're reading a confession.
2. Is there a way to act without that plan?
One preview path and three side doors is zero preview paths. Ask where every mutation enters the tool. If the answer isn't "one funnel, no exceptions," the plan is decoration.
3. When it acts, does a record land that the agent can't rewrite?
An append-only log the agent itself can edit is a diary, not evidence. You want tamper-evidence — a chained record where a rewrite breaks something checkable. Then ask the second half: who holds the key that seals it?
4. Can you verify that record without trusting the tool that wrote it?
"Trust me" from the thing being audited is a circle. There should be a check you run yourself, and the docs should say exactly what that check proves — and what it doesn't.
5. Can it be undone — and does the tool say plainly where it can't?
Nothing undoes everything. A firewall flush, a deleted token, a wiped disk — some of that has no rollback primitive on any platform. The trustworthy answer isn't "fully reversible." It's a map: undo covers this surface, and past that line you're exposed. A tool that won't draw the line is drawing it over you.
6. Does it tell you what a credential can do before the AI touches it?
You should be able to hand the tool a token and get back the honest shape of it — what it reaches, what it can destroy — while the AI is still unplugged. Least privilege you can't inspect is a hope, not a policy.
7. What happens when the agent reads hostile bytes?
The agent will read things strangers wrote — guest output, file contents, ticket text. That's the injection door: content becomes instruction, instruction becomes action. Ask what the tool does about it. Marks the read? Restricts what a tainted agent may do next? Anything? Most tools haven't met the question.
8. Who holds the off switch — and does it live outside the agent's reach?
A kill switch the agent can flip back isn't a kill switch. Same for every control on this list: config the agent can rewrite, ledgers it can reach, consent files it can forge — those are manners, not walls. The honest question is where the control state lives. If it's inside the agent's write reach, you have discipline. Only outside do you have a boundary.
9. Does it phone home?
What does the vendor learn when you run it? The right answer is a number: nothing. No telemetry, no install ping, no usage data. Anything more than nothing, the tool has two masters, and you're the one who didn't get asked.
10. Does it lead with its limits?
Read the front page. Count the sentences that tell you what the tool can't do, what's unproven, where the edges are. Zero is a verdict. A tool with nothing to hide leads with the sharp edges — because the maker knows the limits are load-bearing, and hiding them just moves the discovery to your outage.
How to score
Pass / partial / fail, straight down the list. No weights, no total — the point isn't a number, it's that you asked. A vendor who answers all ten in writing is telling you something. A vendor who answers eight and names the two — more. A vendor who won't take the test took it anyway.
Proximo, scored (by its builder)
Rules apply to me first. Proximo is a Proxmox MCP server; this is its honest sheet, checkable against SECURITY.md:
- See it before it acts — pass. Every mutation is planned first: dry-run preview, blast radius shown.
- No side door — pass. One mutation funnel. In-guest exec exists — opt-in, loud, allowlisted.
- A record it can't rewrite — pass. Hash-chained ledger, keyed (HMAC) by default.
- Verify without trusting — pass, with reading.
audit_verify checks the chain yourself. The strong tail-attack guarantee is the opt-in pinned head — the docs say exactly which check proves what.
- Undo, with the line drawn — pass on honesty, partial on coverage. Snapshots before it acts, where the platform can. Firewall/SDN/ACL/token planes have no rollback primitive — said plainly, not papered over.
- Inspect the credential first — pass.
proximo doctor prints what a token can and cannot do before any AI is wired in.
- Hostile bytes — partial. A taint control is built — adversarial reads set a sticky marker; tainted agents lose a pre-declared action set. But it's opt-in, inert until configured. Built ≠ on.
- Off switch outside reach — partial. Kill-switch, consent, lease, and scope controls all exist — and they're only a real boundary when their state lives outside the agent's write reach. On a single box they're discipline, not a wall. Proximo's docs call this the two-deployment trust model and tell you which one you're running.
- Phone home — pass. Nothing. No telemetry, no install data, no tracking.
- Leads with limits — pass. "Risk ratings are an advisory heuristic, not a sandbox — LOW means no state change, not safe" is on the front page. So is this table.
Seven pass, one pass-with-reading, two partial — and the partials are published here, by me, on purpose. That's the whole method: the test only works if the person holding it can't exempt himself.
Take the test. Apply it to anything — including mine.