Answer or Escalate: How AI Agents Decide What They Can Handle
Every AI support agent eventually runs into a question it shouldn’t answer. That’s not a failure of the system. It’s the moment that reveals whether the system was actually designed well. The difference between an AI agent you can trust and one you can’t often comes down to exactly this: what happens when a question falls outside what it should handle on its own.
Getting this right isn’t about making the agent smarter in some abstract sense. It’s about deliberately designing the boundary between what it answers and what it hands off, and making sure that boundary holds up consistently, not just most of the time.
The two capabilities that actually matter
Strip away the complexity, and a well-built support agent really only needs to do two things well. Answer questions it has a confident, grounded basis for answering. Escalate everything else to a human. That sounds almost too simple to be the whole story, but the businesses that get AI agents right treat exactly this as the design foundation, then build the specific logic that makes each half work reliably.
This was the approach used for a biotech firm we worked with, a business operating close enough to healthcare that getting this boundary wrong carried real consequences, not just minor inconvenience. Their agent needed to reliably distinguish between a routine shipping question and anything touching on clinical interpretation, and treat those two categories completely differently.
What makes a question answerable
A question is a good candidate for the agent to answer directly when the answer exists clearly in the business’s own documentation, the question doesn’t require judgment or context outside that documentation, and getting it wrong wouldn’t cause meaningful harm to the customer. Shipping timelines, order status, general how-to questions, these tend to fit that profile well.
This is where grounding the agent in a well-structured knowledge base actually pays off. If the agent can search that knowledge base and find a clear, specific match for the question being asked, it has a legitimate basis for answering. How Your AI Agent Actually Finds the Right Answer covers how that retrieval process works, which is the mechanism that determines whether a confident answer is actually available.
What makes a question an escalation
The other side of this is just as important to define clearly. Some categories of questions should escalate every time, regardless of whether documentation technically touches on the topic. For the biotech firm, this included anything involving specific clinical interpretation or medical advice. It didn’t matter how the question was phrased or whether a related article existed. Questions in that category went straight to a human.
Beyond hard category rules, escalation should also happen anytime the agent doesn’t have a confident match in its knowledge base. This is arguably the more common scenario day to day. A customer asks something slightly outside what’s been documented, or phrases a normal question in an unusual way that doesn’t retrieve a clear answer. In both cases, the right move is the same: recognize the gap and hand off, rather than stretching a loosely related piece of documentation into an answer that isn’t quite right.
Why “I don’t know” is a feature, not a failure
There’s a natural instinct to want an AI agent that always has an answer. That instinct is worth resisting. An agent that’s built to always produce a response, even when it doesn’t have a solid basis for one, is far more dangerous than one that occasionally says “let me get you a person” when it should.
A confidently wrong answer is worse than an honest “I don’t know,” because the customer has no reason to question a confident-sounding response. They’ll act on it, and if it’s wrong, that mistake compounds. An agent that escalates instead of guessing preserves trust, even though it might feel like a less impressive result on paper. This is exactly why testing for these edge cases before launch matters so much. How to Test an AI Agent Before It Talks to Customers covers how to specifically probe for this before real customers ever interact with the agent.
Designing the actual pathways
In practice, this comes down to defining specific logic. What topics or keywords should trigger an automatic escalation, no matter what. What confidence threshold in the retrieval process should trigger a handoff instead of an answer. What happens procedurally once an escalation is triggered, does it go to a queue, does it notify someone directly, does the customer get told what’s happening next.
None of this is guesswork once you’ve done the groundwork of understanding your own support patterns. If you know from your ticket history which categories of questions are sensitive, and you know which questions your knowledge base actually covers well versus thinly, you have what you need to define these pathways deliberately rather than reactively. How to Audit Your Support Data Before Adding AI is where that groundwork actually happens, and it’s worth doing before you try to define escalation logic in detail.
This boundary needs to be revisited, not set once
The answer or escalate boundary isn’t something you define once at launch and leave alone. As your knowledge base grows and gaps get filled, questions that used to trigger an escalation because no good answer existed should start getting answered directly, because now the documentation supports it. The reverse can also happen. New product changes or new regulations might mean a category that used to be safely answerable now needs to move to the escalation side.
Keeping this boundary current means treating it as something to monitor, not something to configure once. How to Measure Whether Your AI Agent Is Actually Working covers what to actually track over time, including escalation accuracy specifically, which is one of the clearest signals for whether this boundary is still working the way you designed it.
What this looked like in practice
For the biotech team, getting this boundary right meant their agent could handle a meaningful share, around 40 percent, of inbound questions confidently and correctly, while reliably recognizing when a question needed a human’s judgment instead. That combination, real volume handled well alongside a clear, consistently enforced boundary, is what made the system trustworthy enough to actually rely on, rather than something that needed constant human oversight to catch its mistakes.
The takeaway
An AI support agent’s value isn’t just in how many questions it can answer. It’s in how reliably it knows which questions it shouldn’t try to answer at all. Designing that boundary deliberately, testing it thoroughly, and revisiting it as your business changes is what separates an agent that earns real trust from one that’s just producing answers and hoping for the best.
Listen to the full podcast episode here.