Skip to content

Agentic AI

AI Escalation Paths: Who Steps In When Your Agent Can't.

JK
10 min read

An AI escalation path is a plan for handing a case from an to a person. It says who gets the case, what they can decide, what information comes with it, and how fast they must act. It also says what happens if that person is away.

Think of a new hire on day one. They handle easy tasks. They know when to knock on the manager's door. Your AI agents need the same rules, written down.

Here is how to build that plan, step by step.

TL;DR

  • An escalation path answers four things: who, what authority, what context, how fast.
  • Escalate when the action is high-impact, hard to undo, sensitive, outside policy, or unclear.
  • Do not trust confidence scores alone. Mix them with risk and reversibility.
  • Pause the work at a saved checkpoint. Resume from the human's decision.
  • Test the handoff before launch, and keep testing after.

What an AI escalation path is

An escalation path is the route a problem takes. The problem might be a strange request, a failed tool, or a risky action. It moves to someone with enough skill and authority to deal with it.

A good path covers five things:

  • Who receives the case.
  • What authority they have (can they approve a refund, or only review it?).
  • What information travels with the case.
  • How quickly they must respond.
  • What happens next if they can't solve it or don't answer.

If any one is missing, the path has a hole. Cases sit in a queue. Or the wrong person gets them.

Why you need one

Without a path, people hesitate. Strange cases get settled too narrowly. Leaders hear about serious problems too late. Nobody can tell who made which call.

The US standards body NIST treats this as core. Its AI Risk Management Framework puts human oversight, accountability and governance inside the risk system. They are not extras bolted on after launch. The same document says an AI system can decide alone, defer to a human expert, or act as a second opinion for a human decision maker.

So the choice is not "AI or human". You pick the mix for each type of task.

When should an agent hand off to a person?

The short answer: when a wrong move would cost you.

Here are triggers most teams use:

  1. High impact. Money, contracts, or a customer's account.
  2. Hard to undo. A sent email. A deleted record. A published page.
  3. Sensitive data. Health, finance, children, or anyone vulnerable.
  4. Outside policy. The request breaks a rule, or needs an exception.
  5. Unclear or thin evidence. The agent has conflicting facts, or too few.
  6. Repeated failure. A tool has failed several times in a row.
  7. The user asks. If someone wants a human, they get a human.

That list follows a widely shared practitioner framework on human escalation rules for AI agents.

Confidence is one signal, not the whole answer

Many agents give a confidence score. It tells you how sure the model is. That helps. But a sure model can still be wrong.

So never let a high score skip a rule that needs a human. A big refund is a big refund, however sure the agent feels.

Use confidence together with:

  • the size of the consequence,
  • company policy,
  • how unclear the request is,
  • whether you can reverse the action.

How to set thresholds and risk levels

A threshold is the line where the agent stops and asks. You set a different line for each risk level.

Microsoft shares a worked example in its learning module on confidence-threshold escalation. The company in it is made up. The numbers are illustrations, not rules. Still, the shape is useful.

Risk levelExample impactExample confidence needed
LowUnder $50, reversible, one customer0.60
Moderate$50 to $200, or 2 to 10 customers0.75
HighOver $200, over 10 customers, or irreversible0.88

In that example, policy exceptions and unclear situations always escalate, whatever the score.

Here is the idea. The higher the risk, the surer the agent must be. And some cases skip the score and go straight to a person.

Pick your own numbers

Your business is not that example. Here is a simple way to set your own:

  1. List every action your agents can take.
  2. Sort each into low, moderate or high risk.
  3. Set dollar and customer-count lines that fit your business.
  4. Mark the "always a human" cases.
  5. Start strict. Loosen only when the data says it is safe.

Starting strict costs a little time. Starting loose can cost you a client.

Build the ladder: who handles each stage

An escalation path works like a ladder. Each rung has a person or role. Each has a backup.

NIST's AI RMF Playbook says to hand the case to a role with the right skill and authority. It also says to have a backup, and a higher route for cases that stall or run late. It adds that human roles should be clearly defined. That includes the split between people who oversee the system and people who use it.

A four-gate model

Microsoft's AI agent runbook gives four example gates. We like them because they are easy to explain:

  • Notify. Low-risk, reversible action. The agent acts, then tells a person.
  • Confirm. Moderate consequence. A person says yes before the agent acts.
  • Commit. The agent drafts a serious output. A person signs it off.
  • Qualified review. Regulated or safety-related work. A qualified person must review.

Think of a restaurant kitchen. The line cook plates the salad alone. The head chef checks the special. The owner signs off on a new supplier. Same food, different rungs.

What stays with people

NIST's playbook says people should keep decisions about:

  • regulated or safety matters,
  • policy exceptions,
  • big money or customer impact,
  • sensitive information,
  • new situations that need judgement,
  • negotiation,
  • commitments you can't take back.

Lower-risk work that you can reverse can run on its own. Add a notification or a sample check.

Where the rungs live

Anthropic's own guide to building agents says agents can pause for human feedback at checkpoints or when they hit blockers. That is the mechanism. The rungs are the policy that tells the agent when to use it.

If you build with Claude Code, you can write the "allowed" and "not allowed" lines into its permission settings instead of hoping the agent remembers them. Rules in a file beat rules in someone's head.

Write the roles down

For each rung, write one line:

  • Role (not a name, names change).
  • What they can approve.
  • How long they have to answer.
  • Who covers when they are out.
  • Who gets it if the time runs out.

For a founder-led business, this is where the real gain sits. If every escalation lands on you, you are still the bottleneck. A good ladder sends most cases to trained team members. Only the true edge cases reach the founder.

What a good handoff carries

A handoff fails when the person receives a case and has no idea what happened. They have to ask the customer again. Trust drops.

Send a short, complete package every time:

  • What the user asked, in plain words.
  • What the agent already did.
  • What data it used.
  • Why it escalated (which trigger fired).
  • What the agent suggests.
  • What the deadline is.

Test it with one question. Could someone who has never seen this case act on it in two minutes? If not, add what is missing.

We go deeper on this in our guide to multi-agent workflows with six-field handoffs.

Pause, don't restart

For high-impact actions, the agent should stop at a saved checkpoint. When the person decides, the work picks up from that point. It should not rerun the earlier steps. A design guide on human-in-the-loop control describes this pattern.

Think of a bookmark. You don't reread the book when you come back. You open at the bookmark.

This also stops double actions, like sending the same email twice. Our piece on agent guardrails with durable approvals shows how we build it.

Safe defaults and timeouts

What if nobody answers? You need a rule for that too.

Pick a safe default for each action. Usually it is "do nothing and tell the customer we are checking". Then escalate one rung up after the time limit.

Never let silence count as a yes.

Test your escalation path

Don't wait for a real customer to find the cracks. Test first.

A good test set includes:

  • Cases that must escalate.
  • Cases where a tool fails again and again.
  • Cases where the agent says "I don't know".
  • False alarms that should not escalate.

That last one matters. If everything escalates, your team drowns and stops reading. A handoff design guide lists these test types and adds checks on timeouts, backups and safe defaults. It also says to check that the agent stops once a human takes over.

For each test, check:

  1. Did the right trigger fire?
  2. Could a stranger use the handoff package?
  3. Did the backup get the case when the owner stayed silent?
  4. Did the safe default apply?
  5. Did the work resume from the checkpoint?
  6. Did the agent stop when the human took over?

We cover test cases in more detail in AI agent testing: 10 scenarios, CI checks and replay.

Monitor and improve after launch

An escalation path is never finished. Your business changes. Your agents change. The path should too.

NIST's Generative AI Profile recommends policies for human and AI roles, feedback and recourse, independent evaluation that matches the risk, and steady improvement in how you measure risk. Its core idea is proportion. Higher-risk systems need firmer governance, clearer roles and closer monitoring than low-risk internal tools.

NIST also says to think about trust from the start, through design, launch, use and testing. Its AI RMF FAQ says the voluntary framework suits organisations of any size, in any sector.

Track the whole system

Don't just score the model. Arize, an AI observability company, suggests measuring the whole agent system in production. That means tools, retrieval, context, permissions, state, retries and stopping rules. It also says to slice results by workflow and consequence.

In plain words: look at how the whole kitchen runs, not just one cook.

Turn every failure into a test

When a real case goes wrong, save it. Turn it into a test. Run that test before every change. The same Arize guide recommends building regression sets from confirmed production failures and releasing changes gradually, with a way to roll back.

That gradual release idea works well for content too. If your agents publish pages, the same staged approach applies, as in this look at an automated content publishing pipeline built over 90 days.

A simple monthly routine:

  • Count escalations by trigger.
  • Find triggers that fire too often (false alarms).
  • Find cases that should have escalated and didn't.
  • Check how long each rung took to answer.
  • Update thresholds and write down why.

A starter escalation path you can copy

Here is a plain template. Change the numbers to fit your business.

  1. Agent handles alone: low-risk, reversible, clear, no sensitive data. A person gets a notification.
  2. Team member confirms: moderate impact, or the agent is unsure. Answer due within a set time.
  3. Team lead decides: policy exceptions, large amounts, upset customers. A named backup covers absences.
  4. Founder or qualified expert: regulated matters, irreversible commitments, anything new. This should be rare.

If the founder sees more than a few cases a week, the lower rungs need better rules or better training. Fix those, not the founder's calendar.

For the wider rule set behind this, see our guides on AI policy development and AI system governance. For the people side, read why the best systems keep humans in charge.

Our view: the founder's judgement is the escalation path

We build AI systems for founder-led education and consulting businesses. The same problem shows up every time. The founder's judgement sits in their head, and everything waits for them.

An escalation path is how you move that judgement into the system. First you write down what you would decide, and why. Then you sort the calls: which ones an agent can make, and which need you or a trained person.

That is why escalation is not a safety add-on for us. It is part of the design. We capture how a founder decides, then build it into an : a network of AI employees, built with Claude Code, that follow that standard and know when to stop and ask. If you want the build side, start with our guide to custom AI delivery systems with Claude Code.

You should be able to read the result. The rules, the thresholds and the handoffs sit in plain view, not in a black box.

Done well, your team handles more on its own, and you make fewer routine calls.

James Killick

Work with us on your escalation paths

If your business earns $1M or more and runs on your own expertise, this is the kind of system we build with you. The AI Orchestrator OS takes your judgement and turns it into agents with clear rungs, tested handoffs and a trained team.

Prefer to start smaller? Look at the self-serve platform.

Sources

Your next step: book a strategy call and we will map your first escalation path together.

FAQ

Frequently asked questions.

JK

James Killick

Founder

The AI Orchestrator. 10+ years building digital products and 200+ apps shipped, now helping $1M+ educators and consultants turn their IP into AI-powered delivery systems.

James Killick founded and runs The AI Orchestrators.

Ready to find out where your biggest AI opportunity is?

Take the assessment. It takes about 5 minutes. You'll get a clear picture of how ready your business is.