A secret placed in a prompt is a secret you have handed to something whose whole purpose is to reproduce text.
ConceptWhat it is
Secrets management is the practice of keeping credentials out of source, out of logs and out of the hands of anything that does not need them. It is ordinary engineering hygiene with one new failure mode: a model's context is a place secrets get put, and a model's output is a place they come back out.
The mechanism is not exotic. Someone puts an API key in a system prompt so a tool can use it, or a connection string ends up in retrieved context, and later the model is asked to repeat its instructions. There is no reliable prompt-level defence, because the secret is genuinely there.
How it worksThe mechanics
Credentials live in a secret store and are read by the process that calls the tool, never by the process that assembles the prompt. The model requests an action by name; the executing code attaches the credential. The model never sees it and therefore cannot leak it, whatever it is asked.
The rest is scope and rotation. Each credential is narrow enough that its compromise is survivable, short-lived where the platform allows, and rotated on a schedule that does not depend on anyone remembering. Logs and traces are scrubbed at write time rather than reviewed later, because a secret in a log has already spread to everyone with read access.
At a glanceSee it
The credential is attached by the executor, after the model has asked for an action by name. What the model never receives, it cannot repeat.
When to use itWhere it fits
- Always, for any agent with a tool that authenticates to anything.
- Especially where a system prompt is long-lived and shared across many users and sessions.
- When traces or prompt logs are retained, since those are read by more people than the runtime.
- When a third party supplies a tool or plugin whose implementation you do not control.
When NOT to use itLimits & anti-patterns
- As a defence against a compromised executor — if that process is owned, it holds the credentials legitimately.
- As a reason to skip scoping; a well-stored key with excessive permissions is still an excessive-permission key.
- Placing secrets in environment variables and considering the job done, when the process also writes verbose traces.
- Rotation theatre — frequent rotation of a credential nobody scoped buys very little.
Trade-offsAdvantages & costs
Advantages
- Removes an entire class of leak by construction rather than by instructing the model not to tell.
- Standard tooling exists and is mature; almost none of this needs building.
- Short-lived credentials shrink the window a leak is useful in.
- Scrubbing at write time protects the log consumers you have not thought about yet.
Trade-offs & costs
- An executor indirection is one more moving part between the model and the tool.
- Short-lived credentials need refresh inside long-running agent loops, a common mid-task failure.
- Scrubbers work on patterns and will miss an unusual credential format.
- It solves the credential case and not the general one — sensitive retrieved content is still in context and still repeatable.
ExampleIn the real world
An internal assistant is given a system prompt containing a read-only database URL so its query tool can connect. Months later a user asks it to print its instructions verbatim for debugging, and it does, because that is a reasonable request and the string is right there. The fix was never a better instruction; it was that the executor should have held the connection string and the model should only ever have named the query.
ToolsHow to implement it
- HashiCorp Vault, AWS Secrets Manager or GCP Secret Managerstorage with rotation and access policy, rather than a file.
- Workload identityshort-lived credentials issued to the process, so there is no long-lived secret to steal.
- gitleaks or trufflehog in CIcatching the commit before it becomes history.
- Scrubbing middleware on the trace writerredaction at write time, since a secret in a log has already spread.
Cost & effortWhat it takes
Negligible runtime cost — one lookup, usually cached. Effort is low and almost entirely configuration if a secret store already exists. The cost worth naming is the executor pattern itself, which is a small architectural commitment made early and an unpleasant refactor once forty tools each read their own key.
What changedWhat changed here
Updated this page Anthropic now offers enterprise-managed authentication for MCP connectors, letting organizations centrally govern credentials before agents call external tools.
Three kinds of claim, strongest first. Signal runs every morning.