"What should we buy to make the agent safe?" is usually the wrong first question. Most of an agent security stack is infrastructure you already operate. What's new is smaller than the vendor noise suggests. This is the stack we deploy, layer by layer: what each part does, what it protects against, and when you actually need it.
If you want the threat-model side first (what goes wrong and how), start with Securing AI Agents in Production and the hardening checklist. This post is about the tooling.
The five layers
A production agent needs five kinds of protection. Every tool worth deploying fits one of them:
- Identity & credentials: who the agent is, what it may touch
- Secrets: keeping keys out of the model's reach
- Gateway & authorization: controlling every tool call
- Guardrails: screening what goes into and out of the model
- Observability: knowing what happened, always
Layer 1: Identity (your cloud IAM, used properly)
The most important agent security tool is the one you already have. AWS IAM, Azure Entra, GCP IAM: issue each agent its own service identity with least-privilege grants and short-lived credentials via workload identity. No agent-security product compensates for an agent running on a human's over-privileged key.
When you need it: always. This is the floor.
Layer 2: Secrets (a broker between the agent and its keys)
The specific product (Vault, AWS Secrets Manager, GCP Secret Manager, Doppler) matters far less than the pattern: the agent's runtime fetches capabilities at execution time, and no secret ever appears in a prompt, a system message, or a conversation log. If a credential can be printed by the model, it will eventually be printed by the model.
When you need it: the moment an agent touches any authenticated tool.
Layer 3: Gateway and per-tool authorization (the enforcement point)
Everything the agent does should pass through a chokepoint you control:
- An API gateway or LLM gateway gives you rate limiting, spend caps, model allowlists, and a single audit stream. Gateways like Kong or cloud-native equivalents work; LLM-specific proxies (e.g., LiteLLM as a self-hosted proxy) add per-model routing and budget controls.
- Per-tool authorization is the piece you usually build: a thin policy check between "the model asked to call this tool" and "the tool executes," validating the agent's identity, the tool's declared scope, and the arguments' schema. A policy engine such as Open Policy Agent fits here well; so does a hundred-line middleware if your tool surface is small.
- Human/policy gates for irreversible actions: approvals for payments, deletions, outbound communication.
When you need it: as soon as the agent has more than one tool or any write access.
Layer 4: Guardrails (LLM-aware screening)
This is the genuinely new layer. Its job is screening text the model consumes and produces:
- Input screening flags or strips instruction-shaped content in retrieved documents and user messages before it reaches the context window. This is the prompt-injection defense.
- Output filtering catches secrets, personal data, and policy violations on the way out.
Open-source options include NVIDIA NeMo Guardrails and Guardrails AI for programmable policies; the model providers' moderation endpoints cover baseline content classes. Two honest caveats: every screening layer has false negatives, which is why it sits on top of layers 1–3 rather than instead of them; and rules need maintenance as your tools and content change.
When you need it: whenever the agent reads content you didn't write, which is nearly every useful agent.
Layer 5: Observability (OpenTelemetry, not a bespoke product)
Agents are distributed systems with unusual control flow; the tooling that already works for distributed systems works here. Trace every tool call with the triggering conversation ID, ship it to whatever you already run (Grafana/Tempo, Datadog, Honeycomb) and alert on the behavioral tells: novel tool sequences, authorization-denial spikes, output-filter hits, anomalous retrieval volume.
When you need it: before the first production request. Retrofitting tracing after an incident is archaeology.
Sandboxing: the sixth layer for code-running agents
If the agent executes code or arbitrary commands, add isolation: container sandboxes (gVisor, Firecracker-based services) or WebAssembly runtimes, with no default network egress and an allowlist for what its tools legitimately reach.
What we'd deploy tomorrow, in order
- Scoped agent identity + workload identity credentials (half a day, highest value)
- Secrets broker between agent and tools (half a day)
- Gateway with rate limits and spend caps (a day)
- Per-tool authorization middleware with schema validation (one to three days, the real work)
- Tracing wired to your existing observability (a day)
- Guardrail layer for input/output screening (iterative: start simple, tune with red-teaming)
- Sandbox, if code execution is on the menu
Five of the seven steps are boring infrastructure discipline, not AI products. That is the actual state of agent security in 2026. The teams that get breached are rarely missing a guardrail library; they're missing least privilege.
Stuck taking an AI system to production?
This is our whole job. InfoDive Labs is an engineering studio: senior engineers take systems the rest of the way: fix what's stuck, harden what's exposed, and stay on the pager afterwards. Every engagement starts with a call with the engineer who would do the work, not a salesperson.