AI Agent Network Design: A Plain Guide for Founders.

Good AI agent network design starts small. Begin with one coordinator and a few specialist agents. Give each one a clear job. Set rules for who can do what. Test it on real work before you trust it.

This guide shows you how. It uses plain words and kitchen metaphors. It draws on public guidance from Microsoft, Google and AWS.

What AI agent network design means

An AI agent is software that uses a language model to do a job. It can read, decide and act with tools.

An agent network is several of these working together. Think of a restaurant kitchen. One head chef reads the order. Then the grill cook, the pastry cook and the prep cook each do their part. Nobody does everything.

A single agent is one cook with a huge menu. It works until the orders pile up. Then it makes mistakes.

Our glossary defines the common terms if any word here is new. For the wider idea, see multi-agent systems, explained.

Start with one agent. Then split.

Most networks should not exist yet. Microsoft says multi-agent designs add coordination overhead, latency and new ways to fail. So add an agent only for a real reason.

Good reasons to split:

• Tools. One agent has too many tools and picks the wrong one.
• Context. The instructions are so long that the agent loses track.
• Security. One part of the job needs data the rest should never see.
• Domain. Two parts need very different skills, such as legal checks and friendly emails.

Google gives a similar rule. Use several agents when one prompt (the written instructions you give the model) becomes too large, too hard to test or too hard to secure.

If none of those apply, keep one agent. We wrote about how to check this in multi-agent vs single agent: measure before you build.

Pick the pattern from the shape of the work

Do not pick a pattern because it sounds clever. Look at how the work flows.

Microsoft documents five common patterns. Here they are in kitchen terms.

Pattern	Kitchen version	Use it when
Sequential	An assembly line	Steps happen in a fixed order
Concurrent	Two cooks, two dishes	Jobs are independent
Coordinator	A head chef calling orders	One router sends work to specialists
Handoff	Passing a table to another waiter	Ownership changes as you learn more
Group chat	Chefs tasting together	You need debate or a quality gate

Google describes a similar set: sequential, coordinator and hierarchical. Hierarchical means a manager agent who runs other managers.

A simple example

Say you run a course business. A student sends a question about week three.

A coordinator reads it. It decides this is a content question, not a billing question. It passes the job to a content agent. That agent answers using your own lessons. A review agent then checks the tone against your standard. Only then does the student see it.

That is a coordinator pattern with a sequential check at the end. Two patterns can sit in one network.

Why hand-offs need a clear form

The weakest spot in most networks is the moment one agent passes work to another. If the note is vague, the next agent guesses.

Write a fixed form for every hand-off. It should say what the job is, what has been done, and what "done" looks like. We lay out a six-field version in designing multi-agent workflows.

The two layers every network needs

A prompt alone is not a network. You also need something that runs it and something that governs it.

A recent paper on an Agent Operating System splits the problem into two planes. A control and governance plane handles intent, trust, authority, audit and human oversight. A runtime plane handles routing, memory, scheduling and traffic.

In kitchen terms, the runtime is the line moving plates. The governance is the health inspector, the recipe book and the rule that only the sous chef opens the safe.

Microsoft's reference architecture lists the parts a practical network needs:

• An orchestrator (the coordinator that routes work)
• An intent classifier (it works out what the request is)
• An agent registry (a list of who can do what)
• Knowledge stores and vector databases (places to look things up)
• Persistent state (memory that survives between steps)
• Model fallback (a backup model if the first fails)
• Tool connections through MCP

Its advice is to keep control separate from execution. Track agent versions, permissions, routing, state, tool calls, failures and final outputs. If you cannot see what happened, you cannot fix it.

Memory deserves its own plan. See what to keep and what to forget.

Choosing how agents talk and use tools

Two protocols come up often. They solve different problems.

• MCP lets an agent call tools and data. It is like giving a cook a key to the pantry.
• A2A lets independent agents pass tasks to each other across teams, platforms or companies. It is like two restaurants sharing orders.

Microsoft's multi-agent patterns guidance points the same way. Use A2A when agents are owned by different groups. For capability discovery it suggests published agent cards, task contracts and least-privilege access.

For a founder-led business, the simple rule is this. Keep your own agents inside one platform where you can. Add outside protocols only when someone else owns an agent, or you must cross platforms.

One more point. MCP does not make anything safe by itself. Its specification says tool descriptions should be treated as untrusted and user consent is required. You must add your own access controls and data protection.

Treat every boundary as a locked door

Every time one agent hands work to another, you have a trust boundary. Something could go wrong there. A message could be faked. A tool could return poisoned text.

AWS treats this as a coordination problem, not only a model problem. Its guidance includes:

• Verified identities for each agent
• Signed and encrypted messages across boundaries
• Schema checks, so odd messages get rejected
• Scoped permissions, so an agent only gets what it needs
• Circuit breakers, so one failure does not spread
• Logs for discovery, hand-offs and results

A draft from the IETF goes further. It proposes an agent gateway with explicit trust boundaries, privacy scoping, egress filtering and audit trails. It is a draft, not a final standard. Use it as a checklist, not as a legal rule.

A practical example. Your billing agent can read invoices. It should not be able to read your coaching notes. Your content agent can read lessons. It should not be able to refund anyone.

For more on this, read AI agent security needs layers, not a single lock.

Watch cost and speed, not only quality

More agents often means better answers. It also means more bills and more waiting.

One 2026 study of fenced multi-agent orchestration reported 92.0% mean task success against 53.5% for a single agent. The cost was about 2.7 times higher. Latency (the wait) was about 3.7 times higher. The same study saw prompt-injection success fall from 71% to 11%.

Those are results from one study. Do not treat them as a promise for your business.

The shape of the network matters too. The 2025 G-Designer paper tested networks that change shape with task difficulty. It reported up to 95.33% lower token use on one coding benchmark. Tokens are the small chunks of text that models charge for.

The lesson is simple. A fixed web where every agent talks to every other agent can waste money. Easy jobs should take a short path. Hard jobs can bring in more specialists.

At larger scale, noise becomes the problem. A 2026 study tested pools of up to 200 agents. A task manager cut waiting time for high-priority work by 14% to 75%. So set clear priorities and routing early. Do not let agents shout over each other.

We cover cost control in more detail in stop runaway agent costs.

Test it like you would test a new hire

A benchmark score tells you little about your Tuesday morning inbox.

The 2026 READY study shows why. Two systems scored 72.8% and 72.5% on their own. To reach the same 76% reliability target, they needed 39.2% and 29.6% human review. That came from a clinical-audit case study with 16 systems and 750 cases. The lesson still applies to you.

Before launch, decide three numbers:

1. How reliable must the output be?
2. How much human review can you afford?
3. What is the most each task may cost?

Then run real cases from your own business. Include the awkward ones. If you want a view on how teams track productivity gains once agents are live, this write-up on AI productivity results for agencies is a useful read.

For a full test plan, see AI agent testing: 10 scenarios, CI checks and replay.

A build order that works

Here is the order we suggest for a first network.

1. Write down the job. Pick one workflow that eats founder time.
2. Map the decisions. List what you decide and why. This is your recipe.
3. Build one agent. Make it work end to end on real cases. We build ours in Claude Code, so a non-technical founder can follow what is happening.
4. Split only where it hurts. Add a specialist for a tool, a context or a security reason.
5. Add a coordinator. Let it route work and enforce the hand-off form.
6. Lock the doors. Set permissions, logs and limits.
7. Test, then add a human check. Remove the check only when the numbers say you can.

This is slower than buying a tool and pressing go. It is also the reason the network keeps working after the demo.

Our view: the recipe matters more than the wiring

Most guides on this topic talk about wiring. Which agent talks to which. That matters. But we see a different problem first.

Founders in education and consulting have valuable IP (their own methods, frameworks and judgment). It lives in their heads. A network with perfect wiring still fails if it does not know how the founder decides.

So we start with the recipe, not the kitchen layout. We pull out how you judge a good answer, a good client and a good outcome. Then we write it into the agents. That is how a team of AI employees can deliver to your standard without you in every loop. We call the whole set an AI Operating System, and we build it with Claude Code.

Some businesses need one good agent, not five. Some jobs should stay human. If a network would add cost with no gain, we tell you.

You should also own what gets built. You should be able to see every agent, every rule and every hand-off.

James Killick

If you want help building it

We work with founder-led education and consulting businesses earning $1M or more a year.

There are two ways in:

• Foundation Month: $3,500 one-off.
• 90-day program: $18,500 one-off. We build the network with you, on your own workflows.

There is also Platform Access at $90 per month or $900 per year. It is a self-serve tool for building and managing AI systems from your own expertise.

See the program details and pricing. If you teach online, start with AI consulting for online educators. If you coach or consult, see AI consulting for coaches and consultants.

Sources

• AI agent orchestration patterns, Microsoft Learn
• Choose a design pattern for your agentic AI system, Google Cloud
• Multi-agent patterns, Microsoft Learn
• Multi-agent reference architecture, Microsoft
• Agent Operating System paper, arXiv
• Model Context Protocol specification
• Secure multi-agent orchestration, AWS Agentic AI Lens
• Agent Communication Gateway draft, IETF
• Capability-fenced orchestration study
• G-Designer, PMLR
• Event-driven multi-agent orchestration at scale, arXiv
• READY or Not: Reliable Enterprise Agent Deployment

Your next step: pick the one workflow that eats most of your time, then see how our 90-day program would build it.