AI chatbots for customer support: deflection is the wrong target

An AI chatbot builder for customer support should be judged on resolution, not deflection. Deflection is the share of conversations that never reach a person, and it counts two opposite outcomes as one: a customer whose question was answered, and a customer who gave up. A bot tuned for deflection drifts towards the second outcome. Measuring resolution forces the handover to be good instead of rare.

What we'll cover

• Deflection counts failures as successes
• What to measure instead
• Questions a bot should answer
• Building it with CodeWords
• Questions it should never take
• The handover
• Knowing when it does not know
• What it says on your behalf
• Helping agents rather than replacing them
• Making it survive
• Limits worth knowing about
• What to build first
• Frequently asked questions

Key takeaways

• Deflection counts abandoned conversations as successes, so measure resolution, repeat contact and abandonment separately.
• A support bot should answer high-volume, low-ambiguity questions and never take complaints, money movements or upset customers.
• A good handover transfers the whole conversation and is visible at every point, which makes every other feature safe to add.
• An assistant that drafts answers for human agents is easier to adopt than a customer-facing bot, because the agent catches errors.

Deflection counts failures as successes

A contained conversation is either an answered question or an abandoned one.

Abandonment looks identical to success in the metric, and it is cheaper, so tuning the number pushes towards it.

The customer who gave up frequently contacts you through another channel, which moves the cost rather than removing it.

Or they do not contact you again at all, which is the outcome the metric rewards most and the business wants least.

Public complaints follow, since somebody who cannot reach support tells somebody else.

The number is still worth having, alongside the ones that make it interpretable.

What to measure instead

Resolution: did the customer's problem get solved, which requires asking rather than inferring.

Repeat contact within a window, which is the strongest available proxy for an unresolved issue.

Handover rate and what happens after, since a good handover is a success rather than a failure of the bot.

Time to resolution end to end, including any human stage, rather than time to first response.

Satisfaction asked after resolution, not after the bot conversation, which measures something different.

Abandonment explicitly, so the two halves of deflection are separable.

Questions a bot should answer

Factual questions with a single correct answer, such as opening hours, policy details and feature availability.

Account-specific lookups where the customer is authenticated: order status, balance, delivery date.

Routine self-service actions: reset something, update a detail, cancel within a clear policy.

Navigation, meaning getting somebody to the right place, which is a large share of contacts.

Anything the documentation answers, which the bot can find faster than a person can.

High volume and low ambiguity is the pattern, and most support volume fits it.

Building it with CodeWords

CodeWords connects to 3,000+ integrations (CodeWords, 2026), and each connection is made once and reused.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which questions are handled, which route to a person, and what context travels with them.
3. Authorize the connections: the helpdesk, the account or order system, and the knowledge base.
4. Describe the exceptions: an authenticated action, a complaint, a vulnerable customer, a question outside scope.
5. Run it against last month's real conversations and check what it would have said.

You describe the outcome; Cody builds it, connects it, and deploys it. The Free plan ($0) covers light use, with Pro at $39 per month and Business at $100 per month as usage grows (CodeWords pricing page, 2026); details are on the pricing page.

Questions it should never take

Anything where the customer is upset, which should route immediately regardless of topic.

Complaints, which have process requirements in some sectors and relationship consequences in all of them.

Anything involving money moving, such as refunds, disputes and payments, beyond stating a policy.

Cancellation, where an automated attempt to retain reads badly and is restricted in several markets.

Anything about a vulnerable customer's circumstances, which needs a person.

Anything it has already failed at once, since repeating the same answer is the single most frustrating experience a bot produces.

The handover

The handover is the most important part of a support bot, and teams usually treat it as the failure case.

Transfer the whole conversation, so the customer never repeats themselves, a common complaint about bots.

Say what the bot tried and why it escalated, so the agent starts informed.

Be honest about the wait, since an unexplained silence after a fast bot is worse than an accurate estimate.

Offer the handover early rather than making the customer discover the escape route.

Never hide the handover. A bot that makes reaching a person difficult converts a small problem into a complaint about your company rather than about the question.

Knowing when it does not know

Not knowing is an acceptable answer and should be available rather than engineered out.

The alternative is confident wrongness, which on a support channel is a statement your company has made.

Set a confidence threshold and hand over below it.

Escalate on repetition, since a customer rephrasing twice is telling you it is not working.

Watch the questions it declined, which is the list of what to add and is the most useful output of the whole system.

Never invent policy. An answer about a refund window or a warranty that the bot constructed is a commitment somebody may hold you to.

What it says on your behalf

Its answers are your statements, and are treated as such by customers and in several jurisdictions by regulators.

Ground it in your actual documentation rather than allowing general knowledge to fill gaps.

Review what it says about anything regulated: claims, guarantees, safety and eligibility.

Keep a transcript, since a disputed answer needs a record.

Tell people they are talking to a bot, which is required in some jurisdictions and is good practice everywhere.

Update it when policy changes, because a bot stating last quarter's policy is a specific and avoidable liability.

Helping agents rather than replacing them

The application with the clearest return is not customer-facing at all. A bot answering customers is visible and contested; one helping an agent answer is neither.

Suggest the answer, with the source, for the agent to send or edit. This is faster than searching and keeps the judgement human.

Assemble the context before the agent opens the conversation: account, history, recent orders, prior contacts.

Show the similar resolved case, since most contacts resemble one that was handled last month.

Draft the reply for editing, and record what agents change, which is the best available signal about where the suggestions are wrong.

Flag the policy constraint relevant to this case, so an agent does not have to remember every rule.

Agent assistants are easier to adopt, because an agent-facing error is caught by the agent and a customer-facing one is not.

Making it survive

Fail to a person, always, rather than to an error or a loop.

Handle the knowledge base being unavailable by handing over rather than answering from nothing.

Monitor for the loop, where a conversation cycles without progressing. This is the most damaging failure pattern.

Keep the escape route visible at every point in the conversation.

Report the outcome: conversations, resolved, handed over, abandoned, and the questions it could not answer.

Limits worth knowing about

Coverage is bounded by your documentation, and a bot cannot answer what nobody wrote down.

Ambiguous questions are common, and customers describe symptoms rather than causes.

Tone matters and is hard, particularly with a frustrated customer.

Languages vary in quality, which means some customers get a materially worse service.

A bot cannot fix a product problem, and deflecting contacts about a genuine fault delays finding out about it.

What to build first

Build the handover before any answering: a route to a person that carries the full conversation and is visible at every point. The handover makes everything else safe to add, because a bot whose failure mode is a fast, well-informed transfer to a human can attempt questions that one with no escape route cannot.

Two habits make the difference. Measure resolution and abandonment separately rather than reporting deflection, since the combined number rewards exactly the outcome you least want. And let it say it does not know, because confident wrongness on a support channel is a statement your company has made and may be held to.

Frequently asked questions

Why is deflection the wrong metric?

Deflection is the wrong metric because it counts an answered question and an abandoned conversation as the same outcome. Abandonment is cheaper, so tuning for the number drifts towards customers giving up, which is the result the business wants least.

What should be measured instead?

Measure resolution first, then repeat contact within a window, handover rate and what happens afterwards, end-to-end time including the human stage, satisfaction asked after resolution, and abandonment as its own number.

What should a support bot answer?

A support bot should answer factual questions with one correct answer, authenticated account lookups, routine self-service actions, navigation, and anything the documentation covers. The pattern is high volume and low ambiguity.

What should it never take?

A support bot should never take conversations with an upset customer, complaints, money movements beyond stating policy, cancellations, vulnerable customers, or any question it has already failed to answer once. Route these to a person.

Why is the handover the most important part?

The handover makes everything else safe. Transfer the full conversation so the customer never repeats themselves, say what the bot tried, be honest about the wait, and offer the handover early instead of hiding the escape route.

Should the bot be allowed to say it does not know?

Yes, the bot should be allowed to say it does not know. The alternative is confident wrongness, which on a support channel is a statement your company has made. It is most damaging when the bot invents a policy about refunds or warranties.

Do customers need to be told they are talking to a bot?

Yes, tell customers they are talking to a bot. Some jurisdictions require it, and it is good practice everywhere. The bot's answers are your statements, so keep a transcript for anything disputed.

What is the most useful output of a support bot?

The list of questions the bot could not answer is its most useful output. That list states the documentation gap precisely, and it is more valuable than the deflection figure.

Is an agent-facing assistant better than a customer-facing bot?

An agent-facing assistant is easier to adopt and safer, because the agent catches an error before it reaches a customer. Suggested answers with sources, assembled context and similar resolved cases speed agents up while the judgement stays human.

How do I tell whether the suggestions are good?

Record what agents change before sending. The pattern of edits shows where the suggestions are wrong, and collecting it costs nothing extra.

Related reading

• Automating product feedback categorisation
• Automating knowledge base updates
• Connecting Intercom to HubSpot