Agentic AI Security
Test the agent’s permissions, memory, and side effects—not just its prompts.
30 minutes. Bring the evidence, trace, or deadline.
This page is for
- Platform and security engineers shipping tool-using or multi-agent systems
- Product teams adding actions such as refunds, provisioning, deletion, or configuration changes
- Founders who need an evidence-backed answer about safe autonomy and human approval
Start here
The short answer
Direct answer
An AI agent security assessment tests the full control loop: user intent, planning, memory, tool selection, authorization, downstream effects, and recovery. The central question is whether an agent can be persuaded into exceeding the authority of the human or service identity behind it.
Search intent this page serves: Your agent can read, write, transact, browse, or coordinate across systems and needs bounded autonomy before a customer-facing launch or enterprise review.
System boundary
What the assessment follows
The assessment starts with the real trust boundaries. Every included surface has a corresponding scenario, evidence expectation, or retest condition.
Representative flow
- User request → policy classification → agent plan and bounded context
- Plan → tool selection → trusted authorization → validated tool arguments
- Tool result → state update or delegated task → user-visible response
- Approval or failure → side-effect commit, compensating action, audit trail
Risk and controls
The control matrix is specific to this system
A finding is useful only when the team can locate the control, observe the failure, name the evidence, and decide what happens next.
| Surface | Failure or risk | Control to inspect | Evidence | Priority |
|---|---|---|---|---|
| Delegated authority | The agent retains broader privileges than the requesting user or the task requires. | Issue short-lived, task-scoped capabilities with server-side subject, tenant, and resource binding. | Token claims, policy decisions, denied escalation tests, and privilege review. | Critical |
| Tool registry | A tool description or synonym hides a side effect, destination, or permission requirement. | Classify side effects, declare schemas, allowlist tools per role, and require explicit confirmation for high-impact actions. | Tool manifest, scope map, approval policy, and side-effect simulation. | Critical |
| Memory and state | An attacker poisons durable memory or causes a later task to inherit untrusted instructions. | Separate user facts from instructions, authenticate writes, expire state, and validate memory before use. | Memory schema, write policy, poisoning scenario, retention configuration, and audit events. | High |
| Multi-agent handoff | A downstream agent trusts messages that have lost the original identity, scope, or uncertainty. | Carry signed provenance, task scope, confidence, and allowed actions through every handoff. | Message envelope, delegation test, agent-to-agent policy, and trace correlation. | High |
| Outbound access | Browsing, files, or network tools turn prompt injection into data exfiltration or command execution. | Use destination allowlists, content isolation, egress controls, and result validation outside the model. | Network policy, malicious document case, secret-canary result, and blocked request logs. | High |
| Recovery and idempotency | Retries or partial failure repeat an irreversible action or leave state inconsistent. | Use idempotency keys, commit boundaries, compensating actions, and human-visible recovery status. | Retry drill, state transition log, idempotency implementation, and rollback runbook. | Medium |
Reproducible testing
Scenarios that exercise the boundary
These cases are intentionally concrete. They can be run in a production-adjacent environment with synthetic data and become regression fixtures after remediation.
Privilege escalation through a plausible task
- Precondition
- A support or operations agent can read account data and initiate a restricted action.
- Exercise
- Ask the agent to complete a legitimate task while changing the target account or asserting an emergency role in the conversation.
- Expected control
- The authenticated user and policy engine, not the agent narrative, determine the permitted subject, resource, and action.
- Evidence captured
- Decision trace, denied tool call, token claims, and the user-visible explanation.
Memory poisoning before a later action
- Precondition
- The agent writes durable notes or preferences reused in future tasks.
- Exercise
- Insert an instruction-looking memory through a low-trust path, then start a high-impact task in a fresh session.
- Expected control
- Only authorized facts are persisted; untrusted text cannot become an instruction or alter a future tool scope.
- Evidence captured
- Memory write event, provenance, expiration policy, replay outcome, and deletion test.
Cross-agent confused deputy
- Precondition
- A planner delegates work to a specialist agent with different integrations.
- Exercise
- Provide a task whose language asks the specialist to use the planner’s authority for a resource the user cannot access.
- Expected control
- Delegation carries the original subject and least-privilege capability; the specialist denies the out-of-scope operation.
- Evidence captured
- Delegation envelope, authorization decision, agent traces, and policy test fixture.
Retrying a partially committed action
- Precondition
- The agent calls an external API that succeeds before the local request times out.
- Exercise
- Force a timeout and retry the same turn with duplicate or changed arguments.
- Expected control
- The idempotency key prevents duplication, exposes state, and triggers a compensating path where needed.
- Evidence captured
- Provider request IDs, transaction log, retry trace, and reconciliation result.
Scope and handoff
What the team receives, and what access is needed
The fixed scope is designed to leave behind implementation artifacts, not a report nobody can rerun.
Deliverables
- Agent capability map covering identities, tools, state, and downstream effects
- Least-privilege matrix for user, agent, service, and delegated identities
- Multi-turn, memory-poisoning, confused-deputy, and retry scenarios
- Tool manifest review with side-effect classification and approval rules
- Priority fixes and negative tests for authorization and recovery boundaries
- Agent security evidence pack with trace IDs, owners, and refresh triggers
Access requirements
- Tool registry, agent prompts/policies, and state or memory schemas
- Test identities representing normal users, admins, and delegated agents
- A safe environment where side effects can be simulated or rolled back
- Read-only access to authorization decisions and correlated traces
- An owner for every high-impact tool and recovery path
Day 1
Capability inventory
Enumerate tools, roles, state, delegation, side effects, secrets, egress, and approval boundaries.
Days 2–3
Authorization model
Review subject/resource binding, capability lifetime, tool scopes, handoffs, memory writes, and recovery semantics.
Days 4–5
Adversarial execution
Run multi-turn escalation, memory poisoning, confused-deputy, exfiltration, and retry scenarios.
Day 6
Control handoff
Deliver fixes, negative tests, evidence links, and the retest matrix for changed capabilities.
Retest and limitations
How the result stays current
Retest whenever a new tool, agent, memory store, delegation path, external destination, or high-impact action is introduced. Rerun the full Critical suite after changes to identity or tool authorization; rerun the affected High scenarios for prompt, memory, or planner changes.
Scope limit
This assessment does not certify an agent as safe or replace application authorization testing.
Scope limit
Model refusal is treated as a helpful behavior, never the primary permission boundary.
Scope limit
Multi-agent coverage is limited to the delegation paths and tools in the agreed scope.
Sources and limits
References used for the operating model
These sources provide the framework or vocabulary. The page adds product-boundary tests, evidence requirements, and implementation decisions so the result can be inspected in a live system.
Provides a threat-model-based reference for autonomous systems, tool use, planning, and multi-agent behavior.
Use it as a risk vocabulary, then test the actual product boundary rather than claiming a checklist is a control.
FAQ
Questions buyers and engineers ask
Does this assessment include prompt injection testing?
Yes, but prompt injection is tested as one path through the agent’s trust boundaries. The assessment also validates identity binding, tool authorization, memory writes, delegation, egress, and recovery.
Can we test a multi-agent system without exposing production data?
Yes. Use synthetic tenants, realistic tool contracts, representative policy, and a production-adjacent environment. The most important requirement is preserving the same authority and side-effect boundaries.
What does safe autonomy mean here?
Safe autonomy means the agent can complete bounded work while the application enforces identity, scope, approval, validation, and recovery outside the model. It does not mean the agent is trusted to grant itself authority.
Next step
Turn this page into an owned engineering decision.
Bring the questionnaire, trace, failed workflow, or provider deadline. We will help decide whether a focused implementation is the right scope.
30 minutes. No deck. Leave with a clear next step.