Multi-agent reference architecture in production

Microsoft, Google Cloud and AWS each publish guidance on multi-agent systems. The guidance is useful, and it agrees on more than you might expect. It is also vendor guidance with little measured evidence behind it. In my reading it says least about limits, rollback and ownership. This page grades the main claims and says plainly where the research found nothing.

A multi-agent system, in these sources, puts several agents on one task. Each agent is a model with its own instructions and tools. One agent usually coordinates the rest. The sources call it an orchestrator, a supervisor, a coordinator, a manager agent or a lead agent. This page says orchestrator. The agentic workflow entry in the glossary covers the wider idea.

Outside claims name their source, and a date where the page has one. I read each page on 2 October 2026, except the Gartner release. Gartner blocks scripted reads, so I checked its sentences on a reproduction. Several of the pages change often. Where I write that I found no source, I mean none among the sources I read. A miss does not prove nobody says it elsewhere.

How far can each claim be trusted?

Treat most of it as advice. Almost none of it was measured in a production workload. The table grades the points that matter most, and the sections after it give the detail.

Do the big vendors agree on a shape?

My reading is mostly yes, on four ideas. A coordinating agent routes work to others. Tool access goes through one controlled boundary. State lives outside the model. Every call is logged against a named identity. The guides do not all say all four, but each idea recurs.

Microsoft’s reference architecture, last updated on 18 August 2025, has an orchestrator. It consults a classifier to route a request and a registry to find agents. It connects tools through a Model Context Protocol server and keeps state in persistent storage. The multi-agent guide from Google Cloud, last reviewed on 16 September 2025, also centres on a coordinator agent.

Microsoft, Google Cloud and Anthropic each advise starting small. Microsoft (12 February 2026) says to use the lowest level of complexity that reliably meets your requirements. Google Cloud (reviewed 28 May 2026) recommends starting with a single agent if you are early in agent development. Anthropic (19 December 2024, updated 10 August 2026) advises finding the simplest solution possible and adding complexity only when needed.

AWS (10 June 2026) warns of coordination overhead, handoff complexity and distributed failure modes. An undated AWS Marketplace learning module, written in the first person, also recommends the simplest pattern that fits your workflow.

These are companies that sell the platforms. The Microsoft and Google Cloud pages quoted above report no measured data. Their agreement tells you what the vendors advise.

Should the default be one agent or several?

Start with one. Microsoft, Google Cloud and Anthropic each advise starting small, and the sources still disagree about what comes next. Two of the sources landed a day apart, and my reading is that they disagree on where the line sits. Cognition, which makes the Devin coding agent, published “Don’t Build Multi-Agents” on 12 June 2025. Anthropic described its multi-agent research system on 13 June 2025.

Cognition wrote that “in 2025, running multiple agents in collaboration only results in fragile systems.” One of its principles was to share context. Anthropic says its research system “uses a multi-agent architecture with an orchestrator-worker pattern”. A lead agent coordinates specialised subagents that run in parallel. Anthropic’s own evaluations show that such systems “excel especially for breadth-first queries”. I read that as a read-heavy task, graded by the vendor itself.

Cognition came back to the question on 22 April 2026. Its original observations, it wrote, “still hold today for parallel-writer swarms”. It also described a narrower pattern that works, in which several agents add intelligence to a task while “writes stay single-threaded”.

The same post gives two rules that pull apart. One says to “share as much context as possible between the agents”. For review agents, it says the technique works best when the coding and review agents “do not share any context beforehand”.

My default puts the line between reading and writing. Many independent reads suit several agents. Writes that must agree suit one. I found no source that tested this in a production workload.

The closest measurement found is a Google Research study of agent designs on benchmark tasks. Its paper, posted on 9 December 2025 and revised on 8 April 2026, compared several designs with a single agent. Results ran from a gain of 80.8% to a loss of 70.0%, depending on the task.

The Google Research blog post about the paper, of 28 January 2026, reports that independent agents amplified errors 17.2 times, and centralised ones 4.4 times. The blog counts fewer test configurations than the paper’s latest abstract, so the paper changed after the blog. I cannot say the multipliers hold for the paper.

Anthropic’s own post says multi-agent systems “work mainly because they help spend enough tokens to solve the problem”. One reading is that more tokens help, however they are spent. I found no source that separates the gain from structure from the gain from extra tokens.

What do the failure studies show?

Multi-agent systems fail often in tests, and the studies below do not show how often they fail in production. One is MAST, from Cemri and colleagues, accepted to the NeurIPS 2025 Datasets and Benchmarks track. It read the traces of seven open-source multi-agent frameworks and found failure rates from 41% to 86.7%. Repeated steps and agents that miss a finished task were among the common failures.

The paper is peer reviewed. It tested open-source frameworks, not production systems with an eval gate in front of them.

A runaway loop is a different case, because it costs money. The research found no dated, checkable post-mortem of one in production. The only accounts I found were blog posts that repeat one set of figures with no primary record.

What I found is weaker. Anthropic says early versions of its research system made errors such as spawning 50 subagents for simple queries. A preprint of 2 July 2026, “When Agents Do Not Stop”, scanned public agent repositories for code that can start an infinite loop. It counts code patterns and does not show a loop that ran in production.

My own case is the one first-hand example here: a runaway loop in client work, with no client named. An automatic stop ended it: a step limit and a cost ceiling. The account is anonymised.

Hijacking is another documented failure. NIST’s CAISI tested agent hijacking on a public benchmark. The strongest new attack succeeded 81% of the time, against 11% for the strongest baseline attack.

NIST published the result on 17 January 2025 and updated it on 19 December 2025. That is a benchmark, so it shows what is possible. It gives no rate for real systems. NIST’s wording is that many agents that read untrusted text are open to prompt injection.

Where do limits and cost controls belong?

On every run, set by a person, with a defined result when the limit is reached. The defaults in popular frameworks differ, and they count different things. The table shows five frameworks as read on 2 October 2026. The OpenAI default comes from its source code on GitHub, and the other rows from documentation pages.

A turn, a step and an iteration are different units, so in my reading the three numbers cannot be compared. Step limits and cost limits are also separate controls.

I give every run a step limit. A run that reaches it stops, an alert goes out, and a person decides whether it resumes. I print no numbers.

The Claude Agent SDK documentation says the loop runs until Claude finishes if you set no limits. It calls a budget a good default for production agents. The FAQ on cost per outcome gives this site’s definition of the measure, and I would set a ceiling from your own traces.

Does an agent need durable state and traces?

Yes, the sources say both. State has to survive a crash, and every call needs a trace.

Anthropic’s managed agents post of 8 April 2026 keeps the session log outside the process that runs the agent. Nothing in that process has to survive a crash. Temporal, which sells durable execution, says its service hands the work to another process after a crash and rebuilds the state. LangGraph’s documentation says in-memory checkpoints are lost when the process restarts, and it recommends a durable checkpointer.

The principle is well supported. I found no source that says every production agent needs a separate engine, and none that measures the benefit.

Microsoft’s Cloud Adoption Framework (3 December 2025) says to attribute every action to a specific agent instance and conversation ID. The OpenTelemetry conventions for agent spans were still marked as in development in the repository file read on 2 October 2026. Whether the core traces signal is stable is unconfirmed, so this page makes no claim about it.

OpenAI’s Agents SDK turns tracing on by default, warns that traces may capture sensitive data, and does not offer tracing under zero data retention.

A full trace is kept for 30 days, then only metrics are kept. Personal data is masked before it enters the trace.

Anthropic (9 January 2026, updated 18 March 2026) says an eval suite makes problems visible before they reach users, and that it needs clear ownership. Running evals in the pipeline that ships changes is a narrower claim. Two primary sources say it: Microsoft’s Cloud Adoption Framework and Anthropic’s evals post, which calls automated evals especially useful pre-launch and in CI/CD.

The Microsoft Cloud Adoption Framework says to integrate evaluations into CI/CD pipelines before production. The post on evals in CI is about running them without a golden answer.

In my work, a failing eval blocks the merge. I set the threshold on the first run against historical data, and it only rises, never falls. The owner can override a block, in writing in the pull request.

Who approves what, and who owns the system?

A person should approve high-impact actions, and one named person should own the system. The sources agree on the first and say less about the second.

The Model Context Protocol specification (revision 28 July 2026) says there should always be a human in the loop who can deny tool invocations. The word is “should”, and the specification says the protocol cannot enforce its own security principles. AWS sorts actions into three tiers, autonomous, notify and approve, by impact and reversibility, so that not every action waits for a person.

Approval is also a control that can fail. Anthropic reports that its telemetry showed users approved roughly 93% of permission prompts (25 May 2026). It adds that the more approvals a user sees, the less attention each one gets.

Microsoft’s AI Red Team calls bypass of the human step the most consistently exploited failure mode (4 June 2026). Both are the vendors’ own findings. Anthropic’s post gives no method for its figure. This page cannot say how many prompts is too many.

In my work, two things always need a person: money, and anything that goes outside the company. Everything else runs within limits.

Access follows the same logic. Microsoft’s Cloud Adoption Framework (page updated 26 June 2026) says agent actions must be attributable and enforceable to a unique identity. The MCP authorization specification says authorization is optional for implementations. I read that as follows: a claim of least privilege through MCP describes how someone configured it, and the protocol does not guarantee it.

On ownership, Microsoft’s framework says to give agent governance to the leaders who already own cloud governance, security and compliance. It also says to designate a human role accountable for approving deployments and monitoring compliance.

Microsoft’s reference architecture site describes a cross-functional governance committee. In my reading these fit together if the committee advises and one person decides. This site’s approach page asks for one named person who can refuse a launch and is accountable for AI risk.

In my work, one named owner holds the budget, the access rights and the stop button. That owner cannot change the risk policy or switch off the eval gate alone. Both need a second person.

The EU AI Act names people with competence and authority too. Its human oversight duties apply to high-risk systems only. The Commission’s AI Act service desk still shows the 2024 text of those articles.

Can you roll back a multi-agent change?

Microsoft’s guidance says yes, if you version everything that shapes behaviour. I found no target time in the sources I read, so they cannot say how fast. My target is that the last deploy rolls back in minutes with one command. I publish no measured case until one happens and is written up.

Microsoft’s security page for its reference architecture (updated 12 May 2025) says to version every agent’s logic, prompt configuration and communication contract. It also says to enable immediate rollback to a previous version when anomalies are detected. Temporal’s “Why Temporal?” page promises instant rollback, with no measurement behind the word.

I pin models by exact ID. A model changes only through a commit in version control, and a rollback reverts that commit.

Pinning a model does not always pin behaviour. Anthropic’s documentation says infrastructure updates occasionally produce minor differences in observable behaviour even when the model ID and weights have not changed. I read that as follows: rolling back the model ID does not always roll back behaviour. Evals run after a rollback are how you would see the difference.

Providers also retire models on published schedules, so a pinned model has an end date. Anthropic and OpenAI each publish a deprecations page, and someone has to read it.

Which claims did the research not find a source for?

The research looked for each of these and did not find it.

• A dated, checkable post-mortem of a production agent loop that ran away and cost real money.
• Any multi-agent reference architecture from a standards body. NIST announced an AI agent standards initiative on 17 February 2026. Its page, updated on 14 August 2026, describes convenings and gap analyses, and the research found no finished NIST guidance for agents. The ISO, IEC and IEEE catalogues were not browsed, so the honest wording is “not found”.
• Measured results that compare one agent with several inside one company’s production workload.
• A target time for rollback in any source reviewed.
• Support for the “four planes” model of control, execution, state and capability beyond one undated AWS Marketplace learning module, written in the first person.

One figure rests on a weak source. A LangChain blog of 16 June 2025 says to start small with evals and that even about twenty examples is enough. LangChain sells agent tooling and the post is commentary, so this page does not adopt the figure.

Retrieval and memory design were outside this research. The retrieval-augmented generation entry covers the basics.

Does this site commit to any of it?

Yes, to five controls. The approach page names evals in CI, tracing, cost per outcome, a rollback path and a named owner. In my reading, the sources above back them unevenly. Tracing recurs across the guides, and a named owner is clearest in Microsoft’s Cloud Adoption Framework.

Rollback appears as a practice in three of the 14 pages I searched. Evals in CI appears in the two primary sources named above. The outside sources cover spend limits and spend per reasoning cycle or per agent, which is a different measure from cost per outcome. I found no target time for rollback in the sources I read. The FAQ on what counts as in production gives the bar this site uses.