WhatsApp bot analytics: what to measure on a conversation

The metrics a messaging platform gives you are about messages: sent, delivered, read. The metrics a bot dashboard gives you are about containment: how many conversations did not reach a person. Neither answers the question anybody actually has, which is whether customers got what they came for.

Measuring that takes a different set of numbers and, unavoidably, reading some conversations — because the failures on this channel are visible in transcripts and invisible in aggregates.

What we'll cover

• Why the default metrics mislead
• Resolution, and how to know
• The questions it could not answer
• Building it with CodeWords
• Abandonment, which hides in containment
• Handover quality
• Delivery metrics and what they mean
• Reading transcripts on purpose
• Reporting it to people who did not build it
• Making it survive
• Limits worth knowing about
• What to measure first
• Frequently asked questions

Why the default metrics mislead

Containment counts two opposite outcomes as one: a customer who was helped and a customer who gave up.

Message volume measures activity, not outcome, and a bot sending more messages is frequently doing worse.

Read rate depends on recipient settings, so its absence means nothing.

Conversation counts tell you about demand rather than performance.

Average handling time rewards closing rather than resolving.

All of them are easy to produce, which is why they are the ones on the dashboard.

Resolution, and how to know

Ask, briefly, at the end of a conversation, which is the only direct measure.

Watch for repeat contact within a window, which is the strongest proxy and requires no question.

Check whether the action happened — the booking made, the order tracked, the detail changed — which is the concrete version.

Do not infer from silence. A conversation that stopped is ambiguous, and treating it as success is how containment misleads.

Segment by intent, since a bot excellent at order status and poor at bookings shows as mediocre in aggregate.

Track over time, because the number that matters is whether it is improving.

The questions it could not answer

The most useful output of the whole system, and the one most often discarded.

Log every refusal and every low-confidence turn, with the customer's wording.

Cluster them, since they repeat and a handful of clusters usually covers most of the volume.

That list is your documentation gap, stated precisely by the people who wanted the answer.

Review it weekly and add to the bot's sources rather than to its instructions.

Watch the rate, since a rising proportion means your customers' questions have changed.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: what to record per conversation, what to cluster, and what to report weekly.
3. Authorize the connection to your WhatsApp Business Platform account and wherever outcomes are recorded.
4. Describe the exceptions: a conversation with no resolution, a handover, a repeat contact, an unanswered question.
5. Run it for a fortnight before changing anything about the bot, so the change has a baseline.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Abandonment, which hides in containment

A conversation that stopped mid-flow is the signal to separate out.

Look at where it stopped, since abandonment concentrates at specific turns.

A stop immediately after a bot message is that message failing.

Compare against conversations that completed, which usually differ in one identifiable step.

Count it explicitly rather than letting it sit inside a containment figure.

It is the most actionable metric here, because the fix is usually a single message.

Handover quality

Rate is not quality. A high handover rate with fast resolution afterwards is a working system.

Time from handover to a human reply is the number customers experience.

Did the agent have the context, or did the customer repeat themselves, which is the most common complaint about any bot.

Handovers that bounce back to the bot indicate a routing problem.

Post-handover satisfaction measures the whole experience rather than the bot alone.

Report handover as an outcome, not a failure, or you will optimise against it.

Delivery metrics and what they mean

Sent means accepted by WhatsApp, not delivered.

Delivered means it reached a device.

Read requires the recipient's settings to allow it.

Failed carries a reason worth handling by category rather than in aggregate.

Template performance individually, since an aggregate hides the one generating blocks.

Watch delivery failures by recipient, which is how you find numbers that have stopped working.

Reading transcripts on purpose

Ten conversations a week, chosen randomly rather than by a filter.

Read as a customer, not as the author of the bot.

Note where you winced, which is more reliable than any metric for finding tone problems.

Include completed ones, not just failures, since a conversation that worked can still have read badly.

Rotate who reads them, because the same person stops noticing quickly.

This is the practice that finds what dashboards cannot, and it costs half an hour.

Reporting it to people who did not build it

The bot's numbers are usually presented to whoever sponsored it, in a shape that answers whether it was worth doing, and that shape differs from the operating metrics above.

Lead with what customers got: resolved conversations, and the proportion of contact volume handled without a person.

Say what it could not do, with the top clusters, which is more credible than a page of successes and produces the next decision.

Convert to the thing they care about — response time, staff hours returned, contacts deflected from a queue — rather than reporting conversation counts.

Report the cost beside it, including per-message charges, since that is the other half of worth.

Show a transcript or two, which does more to build confidence than any chart.

Be honest about attribution. Faster responses and higher satisfaction may have several causes, and overclaiming here is what makes the next request harder.

Making it survive

Record outcome per conversation, not just messages, which requires deciding what outcome means.

Keep the unanswered-question log permanently, since its value compounds.

Separate abandonment from resolution in every report.

Retain transcripts according to a policy, not indefinitely by default.

Report the outcome weekly: resolution, abandonment, handover and time after it, and the top unanswered clusters.

Limits worth knowing about

You cannot see blocks directly, only the quality rating that results.

Read receipts are unreliable as a measure of anything.

Satisfaction questions reduce response and are worth asking sparingly.

Attribution to the bot is loose for anything that happened after a handover.

Metrics will not tell you it sounds wrong, which is the failure that most affects how customers feel about it.

What to measure first

The questions it could not answer, logged with the customer's own wording and clustered weekly. It requires no instrumentation of outcomes, it is the most direct statement of what your customers want and are not getting, and it produces a change list rather than a score.

Two habits make the difference. Separate abandonment from resolution rather than reporting containment, since the combined number counts a customer who gave up as a success. And read ten random transcripts weekly, because tone and awkwardness are invisible in aggregates and they are what determines whether customers are glad the channel exists.

Frequently asked questions

Why is containment a poor metric?

It counts a customer who was helped and a customer who gave up as the same outcome. Since the second is cheaper, optimising containment drifts towards abandonment.

How do I know whether a conversation resolved?

Ask briefly at the end, watch for repeat contact within a window, and check whether the concrete action happened — the booking made, the order tracked. Do not infer resolution from silence.

What is the most useful thing to log?

The questions the bot could not answer, in the customer's own wording, clustered weekly. It is your documentation gap stated by the people who wanted the answer, and most implementations discard it.

What is the most actionable metric?

Abandonment, and specifically where it happens. It concentrates at particular turns, a stop immediately after a bot message means that message is failing, and the fix is usually a single message.

Should handover rate be minimised?

No — report it as an outcome rather than a failure, or you will optimise against it. A high handover rate with fast resolution afterwards is a working system.

What should I measure about handovers?

Time to a human reply, whether the agent had context or the customer repeated themselves, whether handovers bounce back to the bot, and satisfaction afterwards.

Can I see how many people blocked me?

Not directly. You see the quality rating that results, which is aggregate and delayed — another reason the unanswered-question log and transcript reading matter.

Why read transcripts if I have metrics?

Because tone, awkwardness and conversations that technically succeeded while reading badly are invisible in aggregates. Ten random ones a week, read as a customer, finds what no dashboard will.

How should bot performance be reported upward?

Resolved conversations and the share handled without a person, with the top unanswered clusters alongside — a report containing only successes is less credible and produces no next decision.

What builds confidence fastest with a sponsor?

A transcript. One real conversation does more than any chart, and it also surfaces tone problems to people who would never read the metrics.

How long should transcripts be kept for analysis?

According to a stated retention policy rather than indefinitely by default. Their analytical value declines quickly while the obligations attached to what customers volunteered do not.

Which single number would I keep if I could keep one?

The count and clustering of questions the bot could not answer. It is the only metric here that produces a change list rather than a score.

Related reading

• WhatsApp AI chatbots and conversational design
• WhatsApp bot prompt engineering
• WhatsApp lead qualification bots