Multi-Agent System Design Patterns

The vocabulary problem that makes pattern selection harder than it should be

There is no authoritative taxonomy for multi-agent design patterns, and this is causing real problems before anyone writes a line of code. Andrew Ng published four foundational agentic patterns. Anthropic published a workflow taxonomy. Community catalogs exist alongside both, some listing over twenty distinct patterns. The research community, the framework teams, and the practitioners are all describing identical structures with entirely different words, and the coordination failures this produces happen at the design table, not in deployment.

The naming situation is almost funny. LangGraph calls its default structure the "supervisor" graph. CrewAI calls the same thing a "hierarchical process." AutoGen calls it "group chat with a manager." The OpenAI cookbook calls it "orchestrator-worker." Four names. One pattern. It is the software equivalent of four people showing up to a party in the same costume and arguing about who did it first.

Teams working across frameworks end up designing the same system with mismatched mental models, and by the time anyone notices, the architectural decisions are already load-bearing. This article adopts consistent structural names, maps the major framework terms to each, and focuses on the tradeoffs that actually govern pattern choice.

Andrew Ng's four agentic patterns as the conceptual foundation

Ng's four patterns function as behavioral primitives: the atomic units composing every structural pattern discussed below. Understanding them is not optional background reading; it is the prerequisite for making any structural decision that holds up under pressure.

Reflection is the pattern where an agent critiques and iterates on its own output. Ng's practical finding was that splitting this into a generator agent and a critic agent produces better results than either produces alone. The insight is structural, not prompt-related: self-critique works better when the critic is architecturally separated from the generator.

Tool use converts a language model from sophisticated autocomplete into an operational system. API calls, database queries, code execution, external service interaction: this is the pattern that makes agents capable of affecting the world rather than merely describing it.

Planning means decomposing a goal into executable steps and adapting when one fails. The adaptive component is what distinguishes genuine planning from a static checklist, and it is the prerequisite for any workflow with more than two steps that go sideways.

Multi-agent collaboration introduces multiple LLM instances playing distinct expert roles. Specialization and diversity of perspective improve solution quality, particularly on problems where a single generalist agent hits context limits or the edges of its training.

These four patterns are not mutually exclusive. A production system typically stacks all of them: planning decomposes the task, tool use gathers data, multi-agent collaboration generates and critiques solutions, reflection refines the output. Every structural pattern in the sections below is, at some level of abstraction, a specific arrangement of these four primitives.

Orchestrator-worker: the most deployed pattern and why it dominates production

The structure is simple. One central orchestrator receives the request, decomposes it into subtasks, delegates to specialized workers, and decides when the task is complete. The orchestrator owns intent classification, domain routing, and context continuity across the entire workflow.

This pattern dominates production deployments for a reason that has nothing to do with elegance: it is controllable. One agent owns the outcome, so when something fails, the failure is traceable. Customer support is a common deployment context; Salesforce Agentforce, Intercom Fin, and Zendesk's AI agents use variations of this structure because accountability matters when the system is talking to customers and someone will eventually need to explain what happened.

The core weakness is that the orchestrator becomes a bottleneck. This is usually fine. Most enterprise workflows are sequential by nature, and the control guarantees are worth the throughput cost. The pattern breaks down on exploratory or creative tasks, where the single point of decision-making becomes a constraint rather than a feature. Knowing when you have hit that ceiling is harder than it sounds.

Name mapping: "supervisor graph" in LangGraph, "hierarchical process" in CrewAI, "group chat with a manager" in AutoGen, and "orchestrator-worker" in the OpenAI cookbook are the same structural pattern. Use whichever term your team has standardized on, but recognize them as equivalent when you encounter them elsewhere.

Hierarchical patterns: when orchestrator-worker needs its own layer of management

Hierarchical patterns extend orchestrator-worker by adding a tier. A top-level planner delegates not to individual workers but to mid-level supervisors, each running its own orchestrator-worker team. The structure is appropriate when subtasks are themselves multi-step workflows requiring their own planning and delegation, not merely parallel execution of discrete units.

Some research suggests two-level hierarchies can outperform flat architectures on certain task types, and that a third level may introduce substantially more latency and state complexity relative to any performance gain. The practical scaling limit in controlled settings is often cited at roughly ten parallel agents before coordination overhead begins to affect throughput. These numbers come from specific benchmarks; production variance is real, so treat them as directional guidance rather than capacity planning figures.

The case for adding hierarchical complexity is narrow: workflows where the subtask is itself a workflow. Software engineering pipelines with distinct planning, implementation, and review phases are the recurring example. The case against is simpler: if the problem fits flat orchestrator-worker, the extra supervisory layer adds latency and state complexity with no measurable benefit. Every startup I've seen add hierarchy in anticipation of scale they never reach pays for that decision in maintenance overhead for years. That is an expensive habit, and it is common enough that it deserves to be called out plainly.

Parallel fan-out and sequential pipelines: opposite tradeoffs on the same axis

These two patterns sit at opposite ends of a single dimension: the tradeoff between auditability and throughput.

Sequential pipelines run tasks one after another. Straightforward to debug, predictable, easy to instrument. They also cannot parallelize, which makes them the wrong choice when throughput is the binding constraint.

Parallel fan-out sends independent subtasks to multiple agents simultaneously and aggregates results when all complete. Send one research question to five specialized agents searching different sources, then synthesize: the wall-clock time is the longest single-agent time, not the sum of all five. The payoff is real when tasks are genuinely decomposable.

Fan-out's characteristic failure mode is hidden dependencies between subtasks. The pattern breaks silently when independence assumptions do not hold, and the aggregation step frequently requires more judgment than a simple merge. Teams discover this in production. That is an unpleasant place to discover it.

The selection logic: choose sequential when auditability and step-by-step control matter most; choose fan-out when tasks are demonstrably independent and latency is the binding constraint. The key word is demonstrably. Independence must be proven, not assumed, because the failure mode of assuming it is subtle and the debugging is miserable.

Event-driven and blackboard patterns: architectures for problems that can't be decomposed upfront

Both patterns are designed for workflows where the solution path cannot be fully planned before execution begins. They handle uncertainty by design and pay for it in debuggability, which is a real price worth acknowledging plainly rather than burying in a footnote.

Event-driven architectures have agents respond to events via publish/subscribe messaging rather than direct invocation. Confluent has documented pub/sub multi-agent patterns that include several canonical sub-patterns within this family: Orchestrator-Worker (the orchestrator emits task events, workers consume and emit results), Hierarchical Agent (parent agents spawn ephemeral child agents per event), Blackboard (all agents share a single event log as shared memory), and Market-Based (agents bid on opportunity events, a coordinator assigns work). Event-driven designs can offer latency advantages over polling-based alternatives in appropriate contexts. The weakness is that execution flow is implicit rather than explicit, which makes debugging a categorically different exercise than orchestrator-worker requires.

The blackboard pattern is architecturally distinct enough to warrant separate treatment. Agents post and retrieve from a shared knowledge base; an agent activates when it sees something on the board it can contribute to, rather than being directly invoked. This suits problems where the solution emerges incrementally: medical diagnosis support, scientific hypothesis generation, complex document analysis. The primary risk is write conflicts when multiple agents update the same entry simultaneously, corrupting shared state without careful concurrency controls. Powerful for the right problems. Painful to debug when something goes wrong.

Neither pattern is appropriate when step-level traceability is a hard requirement. Both require additional instrumentation to achieve the observability that orchestrator-worker delivers by default.

Decentralized and swarm architectures: the resilience-observability tradeoff at its extreme

Decentralized architectures have no central coordinator. Agents communicate peer-to-peer, discover tasks, and self-organize. The system is horizontally scalable and has no single point of failure.

The tradeoff does not soften with engineering effort: maximum resilience and horizontal scale at the cost of debuggability, observability, and predictability. Handoff loops, where Agent A passes to Agent B which passes back to Agent A, are the characteristic failure mode. These require explicit guard conditions on every handoff and remain difficult to detect in production because there is no central coordinator generating a unified execution trace. You find out about them when something downstream breaks in a way that requires reading message logs backward.

Microsoft's AI agent design patterns guidance recommends starting centralized and decentralizing only when concrete scalability bottlenecks are identified. Most production teams never reach the threshold where full decentralization is warranted. The cases where it applies are narrow: adversarial resilience requirements, massive horizontal scale where central coordination is a proven bottleneck, research or simulation contexts where emergent behavior is the intended output rather than an unwelcome side effect.

Swarm architectures belong here because they share the same structural logic. In most enterprise contexts, they are a solution to a problem the enterprise does not actually have.

When a single agent outperforms multi-agent and what that means for pattern selection

Single agents outperform multi-agent systems on a meaningful share of benchmarked tasks. This gets buried in the enthusiasm for multi-agent architecture, and burying it causes expensive mistakes.

The conditions that actually justify multi-agent architecture are specific. Tasks involve agents with distinct roles requiring distinct tools and permissions. Parallelism materially improves throughput for tasks that are genuinely decomposable. Separate permission levels per agent are a functional security requirement. When those conditions are met, the specialization benefits are real and measurable.

When those conditions are absent, adding agents to compensate for weak prompting, unclear task definitions, or poor tool design makes systems worse. Coordination layers amplify underlying specification problems rather than absorbing them. Every post-mortem I've read from teams that moved too fast into multi-agent architecture identifies this same pattern: coordination overhead compounds the original specification problem rather than hiding it. It is worth calling out plainly rather than treating it as a nuanced edge case.

Token cost is a concrete forcing function that makes this argument financial rather than philosophical. Different patterns vary enormously in token consumption depending on reasoning iterations and coordination layers. A multi-agent demo workflow that costs a few dollars can generate substantially higher monthly API costs at production scale. Teams that optimize for demo impressiveness rather than production economics learn this the hard way, usually around the time the billing alert fires.

How production failure modes map back to pattern choice

The MAST failure taxonomy, developed by researchers and validated across more than 1,600 execution traces, categorizes multi-agent system failures into three root categories: Specification Problems, Coordination Failures, and Verification Gaps. Production failure rates in multi-agent systems can be substantial, and even well-implemented systems fail at rates that warrant careful attention. Practitioners should consult current published research for specific figures, as benchmarks in this area vary significantly by task type and system design.

Specification Problems map most directly to orchestrator-worker. Role ambiguity and unclear task definitions are amplified when a central orchestrator must interpret under-specified delegation boundaries. The orchestrator does not fail gracefully under ambiguity; it makes confident decisions based on bad inputs and propagates those decisions downstream, which is roughly the worst possible failure mode for a system designed to be traceable.

Coordination Failures are prominent in hierarchical and decentralized patterns. State synchronization failures, where agents act on stale or divergent shared data, account for a large share of this category. Event-driven and blackboard patterns introduce their own variant: write conflicts and implicit execution flow make state bugs harder to detect before they propagate.

Compounding errors behave differently from isolated failures and deserve explicit attention. Agents are stateful; context accumulates across tool calls without pruning; minor failures cascade. This is a structural issue, not a tuning problem. It is what happens when context compounds in multi-agent systems and nobody has built a pruning strategy.

Security deserves a line. Message-passing mechanisms are the primary attack surface. Prompt injection in one agent can propagate adversarially through the coordination layer to downstream agents, and this is most severe in decentralized and event-driven architectures where there is no central coordinator validating messages before they are acted upon. It is a pattern-level concern, not just an individual agent concern, and it is frequently treated as an afterthought until it isn't.

Interoperability protocols and what they mean for pattern implementation today

Four protocols are now defining the implementation layer for multi-agent systems: MCP from Anthropic (now stewarded by the Linux Foundation), A2A from Google (donated to the Linux Foundation in June 2025), ACP from IBM, and ANP from the broader community. These address different layers of the stack and are complementary rather than competing, though the ecosystem is young enough that the usual growing pains apply.

MCP, the Model Context Protocol, standardizes how a single agent connects to external tools, data sources, and services. It has seen wide adoption since its release, making it among the most broadly used protocols in this family. It handles tool use at the implementation level.

A2A, the Agent-to-Agent Protocol released in April 2025, standardizes how agents discover, communicate, and collaborate across frameworks. It is the relevant protocol for orchestrator-worker coordination and peer-to-peer interaction. A production system implementing orchestrator-worker with tool-using workers likely needs both: MCP for individual agent-to-tool connections and A2A for agent-to-agent coordination. Running one without the other is a common gap that surfaces when cross-framework deployments scale.

Protocol convergence is making cross-framework multi-agent systems more practical, which means pattern choice is increasingly decoupled from framework lock-in. Teams can choose a pattern based on structural fit with the problem rather than based on which framework happens to support it. That is a genuine improvement over where things stood eighteen months ago, even if the protocols themselves are still maturing.

A decision framework for matching patterns to problem structure

Start with task structure. Can the workflow be decomposed upfront? If yes, orchestrator-worker or hierarchical. If no, blackboard or event-driven. That single question eliminates half the decision space before anything else needs to be evaluated.

Then ask whether subtasks are genuinely independent. If yes, fan-out parallelism is available. If hidden dependencies exist or independence cannot be proven, sequential pipeline or orchestrator-worker with explicit dependency tracking is the safer choice. The burden of proof runs toward independence, not toward sequential execution.

Use permission and role boundaries as a selection signal. If agents genuinely need separate permissions, different data access, different tool rights, different trust levels, multi-agent architecture is justified on that basis alone regardless of parallelism benefits. Security architecture and system architecture are the same decision at this layer. Treating them separately is how teams end up revisiting both after deployment.

Apply auditability as a constraint, not a preference. Orchestrator-worker and sequential pipeline produce explicit, traceable execution paths by default. Event-driven and decentralized patterns require additional instrumentation to achieve equivalent traceability. If your compliance environment requires step-level audit logs, part of the pattern choice is already made before you evaluate anything else.

Stop at two hierarchy levels unless a concrete workflow requirement forces a third. The latency and state complexity costs of a third level rarely justify the marginal performance gain, and most teams that add one do so because the architecture diagram looks more complete, not because the problem demands it.

The default, supported by Microsoft's design guidance and practitioner experience, is to start centralized with orchestrator-worker, instrument thoroughly, and add complexity only when a measured bottleneck demands it. Add it when a concrete, measured production problem requires it — not in anticipation of scale. The gap between broad enterprise adoption and thin production deployment is not primarily a technology problem. The patterns are known, the technology exists, and what remains is the discipline to match pattern to problem, which turns out to be harder than it sounds when everyone in the room has a different name for the same thing and a strong opinion about which framework to use.