How would you design a permission system for an agent?

Answer

Design around the fact that the model is not the security boundary. It can be manipulated, so controls must live in the code that executes tool calls.

Classify actions by reversibility, not by type. Reading a file, running a test, and listing a directory change nothing and can be auto-approved. Deleting, force pushing, deploying, and sending money are unrecoverable and should require explicit approval or be blocked outright.

Default deny for anything consequential. Allowlist the safe operations rather than blocklisting the dangerous ones, because you cannot enumerate everything dangerous.

Scope credentials to the task. The real limit on damage is what the tokens can reach. An agent with an admin token is an admin regardless of what your prompt says.

Make approval prompts meaningful. Show the exact command or the actual diff. A prompt saying "the agent wants to run a command" trains people to approve blindly, which is worse than no prompt because it manufactures the appearance of a control.

Avoid approval fatigue deliberately. If users approve twenty times an hour they stop reading. Auto-approve the genuinely safe majority so the remaining prompts carry signal.

Isolate for real boundaries. Containers, VMs, restricted filesystem mounts, and network egress rules are actual controls. Pattern matching on command strings is not: the same effect is reachable through a different command, a script, or a language runtime.

Log everything, because you will need to reconstruct what happened.

Common wrong answer

Treating a blocklist of dangerous command strings as a security boundary. Blocking rm -rf does not prevent deletion through a Python one liner, a shell script, or a different flag ordering. It is a useful guard against accidents and provides no protection against an adversary, and conflating the two produces false confidence.