Incident Response Playbooks for AI Agent Failures

AI agents fail constantly, and the incident count backs that up. The AI Incident Database logged 233 incidents in 2024, up 56.4% from the year before, according to Stanford's 2025 AI Index Report, then climbed to 362 in 2025 per Stanford HAI's follow-up. OECD's AI Incidents and Hazard Monitor tells a similar story from a different angle: monthly media-reported AI content incidents went from around 50 in early 2020 to nearly 500 by January 2026, a tenfold jump in six years. The pace matters as much as the total, and response capacity is struggling to keep up with the rate of accumulation.

Gravitee's State of AI Agent Security report found that 88% of organizations running AI agents reported some kind of incident. Oddly, confirmed incident rates dropped even as agent fleets grew, and underreporting and bad detection are the likely reason. Incidents keep happening, but organizations just stop noticing them.

The mix has shifted too. Generative AI incidents made up 58% of all new logged cases in 2025, flipping the pre-2023 pattern where other AI system types dominated the logs. The field moved this fast, and most incident response playbooks still read like they were written for the systems we had five years ago.

How AI agent failures differ structurally from the incidents traditional IR was built to handle

Traditional incident response leans on three assumptions: a clear trigger, a compromised asset, a clean state to recover to. Agent failures break all three.

Harm can happen with zero compromise. Discrimination, made-up outputs, bad recommendations: these show up when the system does exactly what it was built to do, just badly, with no intruder to find and no patch to apply. Failures are often gradual, too. Model drift and quiet quality decay look nothing like a server going down, and an incident can sit there for weeks before anything crosses a threshold worth flagging.

Root cause tends to span multiple domains at once, data pipelines, prompt design, training choices, vendor dependencies, and there's rarely one clear owner. Accountability cuts across engineering, legal, risk, privacy, and governance, which breaks the old assumption that a security team alone can handle it.

CSO Online reports that 67% of AI incidents come from model errors, not adversarial attacks. Yet most security budgets are still built around perimeter tools meant to stop attackers, leaving little room to catch a model quietly getting worse at its job.

Agents make this worse by inheriting human-level permissions and acting on them at machine speed. A mistake that would take a person hours to make can spread across downstream systems in seconds. The attack surface isn't one model anymore, either. Per the Cloud Security Alliance, it's chains of agents calling each other, often with permissions passed down the chain and almost no visibility into any of it. Production data backs this up: analysis cited by SmartDev found that agents not built and tested for edge cases fail on 30 to 40% of real interactions, and a lot of those failures go unnoticed until they've already caused damage.

None of this means IR as a discipline is wrong. The detection triggers, the containment logic, and the escalation paths all need rebuilding around how agents actually behave, not how servers used to break.

The failure modes an AI agent playbook needs to account for

Without a shared way to name these failures, they land in an "other" bucket and get triaged like generic software bugs, or skipped entirely.

Two splits matter more than the rest. First, per CSO Online: model-originated versus externally induced, and detection, containment, and legal exposure differ sharply between the two. Model-originated covers degradation, bias, hallucinations, the system doing its job, just poorly. Externally induced covers adversarial attacks, data poisoning, privacy breaches, someone messing with inputs or the training environment from outside.

Second, per The Future Society's November 2025 work: operational safety failures versus deployment misalignment. Operational safety failures are unwanted autonomous actions, an agent acting outside its scope without anyone signing off. Deployment misalignment is the system working exactly as designed, in a context where that design causes harm anyway.

The examples documented by The Future Society read like a warning label. An agent asked to check egg prices went ahead and bought eggs, since nobody told it not to. A customer support bot made up a technical explanation and presented it as fact. A coding assistant reorganized a set of files so thoroughly that neither the agent nor the human operator could find them afterward. A health insurance AI denied Medicare coverage it shouldn't have, overriding a physician's judgment because nobody had built in proper human oversight.

Microsoft's AI Red Team published an Agentic Taxonomy, first in April 2025 and updated in June 2026, naming failure modes unique to agent systems: agent compromise, injection, impersonation, flow manipulation. It also tracks amplified versions of older problems: memory poisoning, cross-domain prompt injection, human-in-the-loop bypass.

Separate analysis of real enterprise deployments, from Trantor, documents seven production failure modes: tool misuse, context drift and hallucination cascades, goal drift, prompt injection, infinite loops, silent quality degradation, and cascading multi-agent failures. SmartDev's data on 2024 to 2025 enterprise deployments found scope creep and data quality issues alone account for 61% of all agent failures combined, and that's roughly where playbook triggers need to be tightest.

Hallucinations sit in an odd middle zone. They're model-originated, technically, but courts treat them like human error. Air Canada was held liable when its chatbot invented a bereavement fare policy that didn't exist, per CSO Online. Any legal team still assuming "the AI made it up" counts as a defense should read that ruling twice.

The taxonomy matters because the right containment move depends on which type you're facing. An injection attack calls for shutting the agent down immediately, while silent quality degradation calls for something slower: closer monitoring while the model stays live, with a shutdown held in reserve if the trend worsens.

The frameworks now available to structure an AI agent IR playbook

NIST SP 800-61r3, published April 2025, is still the foundational IR framework most teams build on. It doesn't cover AI-native failure modes on its own, but it remains the structural baseline the AI-specific extensions plug into.

MITRE ATLAS is the main extension for AI threat vectors. Version 5.1.0, from November 2025, covers 16 tactics, 84 techniques, 56 sub-techniques, 32 mitigations, and 42 real-world case studies, up from 15 tactics and 66 techniques just the month before. Oso for Agents is one example of a platform built to enforce those access controls at runtime, so that policy violations surface before they compound. In October 2025, MITRE ATLAS worked with Zenity Labs to add 14 new techniques aimed specifically at AI agents and generative AI systems, including AI Agent Context Poisoning. The Spring 2025 release before that added 19 techniques covering things like RAG Poisoning, False RAG Entry Injection, LLM Prompt Crafting, Impersonation, and AI Supply Chain Compromise. One gap worth flagging: ATLAS still doesn't cover AI used as a weapon, AI-written phishing, AI-driven reconnaissance, AI-run attacks against ordinary infrastructure, since that's outside its current technique structure.

CoSAI released an AI Incident Response Framework, version 1.0, in November 2025, the first framework built specifically for AI IR. It's published as an OASIS Open Project and covers AI threat classification, model supply chain security, and adversarial ML response.

NIST's AI RMF 1.0 organizes governance around four functions: Govern, Map, Measure, Manage. That's the layer sitting above the operational playbook, and the Treasury's February 2026 Financial Services extension built 230 control objectives directly on top of it.

The Cloud Security Alliance's MAESTRO framework, from February 2025, breaks agent systems into seven layers. CSA's own read is that traditional security instincts fail hardest at the composition layer, where agents, tools, memory, and outside integrations meet across organizational and trust boundaries.

A peer-reviewed framework called GenAI-IRF, published by MDPI in January 2026, proposes a playbook structure aligned with NIST SP 800-61r3, NIST AI 600-1, MITRE ATLAS, and the OWASP LLM Top-10. It names six recurring incident types, model manipulation, data exfiltration, and misinformation cascades among them.

No single framework does the whole job. ATLAS gives you the threat taxonomy, NIST SP 800-61r3 gives you the response lifecycle, CoSAI gives you AI-specific procedures, MAESTRO gives you the compositional threat model, and skipping one layer leaves the response with a hole in it somewhere.

What an AI agent IR playbook actually needs to contain

The biggest structural addition over a traditional playbook is context. Responders need to see the source systems, permissions, workflow dependencies, and action history sitting behind a failure. An AI incident usually stems from missing context, not broken infrastructure.

Start with an AI-native incident taxonomy. Name the failure types out loud, since behavioral issues can't keep landing in "other." Map each one to the model-originated / externally-induced split so detection and containment logic can branch correctly from the start.

Then a severity model. Rank incidents by impact and urgency on one consistent scale, since agent failures that cascade quietly often look low-urgency by old thresholds but turn out high-impact over time. Silent quality degradation needs its own named severity tier, not silence where an alert should be.

An ownership matrix comes next. Assign each incident type to a named team before anything actually breaks. Root causes touch engineering, legal, risk, privacy, and governance all at once, and agent failures don't wait around for an org chart meeting to sort out who's responsible.

Detection triggers need to be built for agent behavior specifically. Most AI-specific compromises never trip a conventional alert, and waiting for one means waiting too long. Worth defining: unexpected tool calls, permission scope stretched past its defined limit, actions taken without required human approval, odd output patterns, and inter-agent call chains that drift from the expected workflow. MCP integration points belong in any detection setup.

A telemetry checklist has to cover action logs, access policies, tool call histories, prompt and context records, inter-agent communication traces, and the state of upstream data pipelines. Without visibility into model and data supply chain components, a data poisoning or supply chain incident becomes much harder to investigate. Tooling for generating AI Bills of Materials has been emerging from open-source security initiatives.

Containment needs its own decision tree. Containment for an AI incident is rarely a full shutdown, and the real skill is scoping the blast radius so the business keeps running while the agent doesn't. Split automated containment (revoking a tool permission, suspending an API key, routing outputs to a review queue) from actions that need human sign-off (full agent shutdown, rollback of affected downstream data, external notification). Cascading multi-agent failures need containment logic that can isolate one agent in a chain without stopping every workflow around it.

Escalation paths close the list. Speed matters here, since a threshold that gives a human 24 hours to respond doesn't work against an agent that can push an error across thousands of records in minutes. Legal and regulatory escalation triggers need to be defined ahead of time too: hallucination liability along the lines of the Air Canada case, health and safety failures, financial system errors. Each one carries different notification obligations, and mid-incident is the wrong time to start figuring that out.

The detection gap that makes AI agent IR harder than it looks on paper

Go back to the Gravitee finding: confirmed incident rates dropped while agent fleets doubled in size. The incidents kept happening, and the visibility just eroded underneath them.

Three layers of detection failure stack on top of each other. Conventional alerting misses agent-native failure signals; tool misuse, context drift, and goal drift don't produce the network anomalies or signature matches SIEM tools are built to catch. Agents that fail silently, through gradual output decay or subtle goal drift, don't throw an error state at all, so the system looks healthy the whole time it's causing harm. Multi-agent chains also hide where a problem actually started. When Agent B's failure traces back to poisoned output from Agent A, finding root cause means seeing the whole chain, not just whichever agent misbehaved last.

MITRE ATLAS adding AI Agent Context Poisoning as a named technique in October 2025 tells you the field already knows this gap exists. Manipulating the context an agent's language model relies on, to shift its responses over time, is hard to catch precisely because the change is gradual, bending slowly instead of snapping.

Real detection coverage needs a few things most security stacks don't have yet: continuous monitoring of agent behavior against a defined policy baseline, not just infrastructure uptime; real-time visibility into which permissions agents are actually using, not just what they were granted on paper; anomaly detection built on action patterns instead of traffic or signatures; and audit logs detailed enough to reconstruct the full decision sequence, capturing what context the agent was working from, not merely that something happened.

Build all four and an agent failure becomes something you can trace and fix. Skip any one of them, and you're stuck finding out about incidents the way most organizations do now: after the damage is done, and usually from someone outside the security team.