How to monitor and debug your WhatsApp AI agent in production

Getting a WhatsApp AI agent live is one challenge. Keeping it reliable over time is another. Once your bot is handling real conversations, you need visibility into what it's doing, what's going wrong, and why. This guide covers the key metrics to track, the most common failure modes, and how to debug your WhatsApp AI agent in production without breaking anything.

TL;DR

• Six metrics matter most — response time, completion rate, handover rate, message volume, error rate, and cost per conversation.
• Most bugs are structural, not AI-related — the bot never replies? Check the webhook. Replies twice? You're missing deduplication.
• Log enough to debug, not enough to cause problems — avoid storing personally identifiable information; log message IDs and timestamps instead.

Key metrics to track

Response time

How long does it take for your bot to reply after a message arrives? A response time under three seconds feels instant. Over ten seconds starts to feel broken. Over 30 seconds and customers will send a follow-up message — which can cause duplicate processing bugs.

Track response time as a histogram, not just an average. A median of two seconds with a 95th percentile of 25 seconds tells you something very different from a flat two-second response across the board.

Completion rate

What percentage of conversations reach a successful outcome — an answered question, a booked appointment, a captured lead? If your completion rate is dropping, your bot is either failing to understand messages or hitting a dead end in the flow.

Compare completion rate by message type if you can. FAQ conversations might complete at 90%. Complex order flows might complete at 60%. The gaps tell you where to invest in improvement.

Handover rate

How often does the bot trigger a human handover? A high handover rate (above 20–30%) might mean your bot's knowledge base is too thin or your trigger thresholds are too aggressive. A very low handover rate might mean the bot is attempting things it shouldn't.

Track handover rate alongside completion rate. A bot with a 5% handover rate and 40% completion rate has a big problem — it's not escalating when it should.

Message volume

Track incoming messages by day and by hour. Volume patterns tell you when your bot is busiest, help you spot anomalies (a sudden spike might mean your bot went viral or got a spam flood), and inform capacity planning.

Error rate

How often does the bot fail entirely — no reply at all, an error message sent to the customer, or a crash? An error rate above 1% in production needs immediate attention. Even 0.1% matters if you're processing thousands of conversations a day.

Cost per conversation

If you're using an AI model with usage-based pricing, track how much each conversation costs. A support bot that costs £0.02 per conversation is excellent. One that costs £2 per conversation because it's sending 50-message threads to the model on every request needs to be optimised.

Common failure modes and fixes

Bot never replies

Symptoms: messages come in, nothing goes out. No error in the customer-facing chat.

Likely cause: the webhook isn't connected, the webhook URL has changed, or the WhatsApp API token has expired.

Fix: check that your webhook endpoint is receiving POST requests. Use a tool like ngrok or a logging proxy to verify. Rotate your API token if it's expired. In CodeWords, the connection status is visible on the dashboard — a disconnected indicator is your first check.

Bot replies twice

Symptoms: every message gets two identical (or near-identical) responses.

Likely cause: no deduplication. The webhook is being called twice — either because you have two webhook subscribers registered, or because WhatsApp retries delivery and your system processes both.

Fix: add a message ID check. When a message arrives, check if you've seen that message ID before (store IDs in Redis or a database for 60 seconds). If you have, skip processing. CodeWords handles this automatically.

Bot replies to old messages

Symptoms: when you reconnect the bot or restart it, it starts processing messages from hours or days ago.

Likely cause: no staleness check. The bot is processing every message in the queue regardless of when it was sent.

Fix: add a timestamp filter. If a message is more than 60 seconds old (or five minutes, depending on your tolerance), skip it. In your flow logic: if (message.timestamp < now - 300) { return; }.

Bot replies in the wrong chat

Symptoms: replies are going to different conversations than the ones that triggered them.

Likely cause: a scope filter is missing. The bot is using a stored phone number or chat ID from a previous conversation instead of the current one.

Fix: always derive the reply-to address from the incoming message, not from a stored variable. Each message payload includes the sender's phone number — use that directly.

Bot responds to its own messages or your manual replies

Symptoms: an infinite loop of bot messages, or the bot replies when you reply manually from your phone.

Likely cause: no direction gate. The bot is processing all messages, including outbound ones.

Fix: filter on message direction. Only process inbound messages (messages where the sender is not your own number). This is sometimes called an "is not from me" check, and it's essential for Personal Device connections in particular.

How to add a staleness check

The staleness check is one of the most important — and most overlooked — safeguards in a WhatsApp bot.

Here's the logic in plain English: when a message arrives, compare its timestamp to the current time. If the difference is more than five minutes (300 seconds), the message is stale — it was probably queued during a downtime and you don't want to process it now.

Tell Cody in CodeWords: "Ignore any message that was sent more than five minutes ago." Cody builds the staleness check into the flow automatically.

Logging best practices

Log enough to debug

At minimum, log:

• Message ID (for deduplication checks)
• Timestamp received
• Message direction (inbound/outbound)
• Flow outcome (completed, escalated, errored)
• Response time

These fields let you reconstruct what happened in any conversation without storing the actual message content.

Avoid PII in logs

Don't log phone numbers in plain text, and don't log message content unless you have a specific reason and appropriate data handling controls. A log file containing customer messages is a data liability.

If you need to debug a specific conversation, use the conversation ID or message ID as a lookup key — retrieve the message from WhatsApp's storage rather than keeping your own copy.

Set log retention limits

Logs older than 30 days are rarely useful for debugging. Set a retention policy and stick to it.

Testing in production safely

The safest way to test in production is to use an allowlist. Build a version of your bot that only responds to specific phone numbers — yours, your team's — and deploy that first. Once you're satisfied it's working correctly, open it up to all numbers.

For incremental changes — updating the system prompt, adding a new integration — consider a gradual rollout: run the new version for 10% of conversations, monitor the metrics, and expand if they hold.

Never test by sending yourself a message and hoping for the best. Build a test checklist: the specific messages you send, the responses you expect, and the edge cases you know have caused problems before.

How to monitor and debug your WhatsApp AI agent with CodeWords

When you build with CodeWords, many of these safeguards — deduplication, staleness checks, direction gates — are handled by Cody automatically when you describe your agent. The monitoring dashboard gives you visibility into message volume and errors without needing to build your own logging infrastructure.

For more complex monitoring needs, Cody can connect your agent to external logging services via the 3,000+ integrations available through Composio.

---

Related reading

• WhatsApp bot analytics: what to track and how to measure success
• How to add human handover to your WhatsApp AI agent
• How to build a WhatsApp AI agent in 2026 (no code)

Reading conversations, not just dashboards

The most useful monitoring practice is the least technical one: read the transcripts. Every week, someone should read fifty real conversations end to end.

Metrics tell you that something changed. Transcripts tell you what and why, and they surface the failures no metric is watching for. An agent that answers confidently and wrongly produces perfect numbers, because nothing errored, the conversation ended, and the customer went away. Only reading it reveals the problem.

Worth looking for specifically:

Customers repeating themselves. A reliable sign the agent missed something, and it rarely shows up as a failure anywhere else.

Conversations that end abruptly. Often a customer giving up. Resolution and abandonment look identical in the data and completely different on the page.

Answers that are technically correct and unhelpful. Accurate policy quoted at someone who needed a decision. These never register as errors and they generate complaints.

Anything the agent asserted that it couldn't have known. The highest-severity category, and the one only a person reading can catch.

What to do when something breaks at the worst time

Two things worth having decided before you need them.

A way to turn it off. One action that stops the agent responding and routes everything to people. Not a settings change requiring the person who built it, since problems appear at weekends. Know what that action is and who can take it.

A defined fallback message. What customers see while the agent is off. A brief, honest note that messages are being handled by the team and when to expect a reply is far better than silence, and it costs nothing to prepare in advance.

Both are ten minutes of preparation and they turn a bad afternoon into a minor one.

Frequently asked questions

What should I alert on?

Fewer things than you'd expect, and outcomes rather than errors. Volume dropping sharply, escalations spiking, and conversations ending without resolution are worth waking up for. Individual failed API calls usually aren't, and alerting on them trains everyone to ignore alerts.

How do I test a change without risking customers?

Run the new version against a set of real past conversations and compare the replies before deploying. That catches most regressions. For anything larger, route a small share of live traffic to the new version first rather than switching everyone at once.

How long should I keep conversation logs?

Long enough to investigate a complaint and no longer than your privacy commitments allow. WhatsApp conversations contain personal data, so retention is a decision to make deliberately rather than a default to inherit.

Who should own monitoring?

Whoever owns the customer experience rather than whoever built the thing. The useful signals in a WhatsApp agent are conversational rather than technical, and the person best placed to notice that replies have become subtly unhelpful is someone who knows what good service sounds like in your business.

How often should we review it once it's stable?

Monthly is enough once the first few weeks have settled, with a smaller weekly glance at the numbers. The thing that changes is rarely the agent; it's your business. New products, new policies, and seasonal questions all arrive without anyone thinking to update the automation, and a monthly read catches that drift before customers do.

What's the first sign something has gone wrong?

Usually a change in shape rather than an error: conversations getting longer, escalations rising, or customers rephrasing the same question. Those show up in transcripts before they show up in any failure metric, which is the argument for reading a sample every week even when the numbers look healthy.

What should we log, and what should we avoid logging?

Log enough to reconstruct a conversation and the decisions inside it: the messages, which tools were called, what came back, and where it escalated. Avoid logging more personal data than you need to do that, and be deliberate about how long any of it is kept, because WhatsApp conversations carry personal information by default.

How do we tell a bad agent from a bad process?

Ask whether a well-briefed new employee could have answered the message correctly. If they couldn't, because the information doesn't exist anywhere or the policy is genuinely unclear, the agent is surfacing a process problem rather than causing one. Those are worth separating, since tuning instructions will not fix a missing answer.