Context bleeding: when one customer sees another one's conversation

A customer asks about their order and is told about somebody else's. Somebody is greeted by the wrong name. A bot answers a question that was never asked in this conversation, because it was asked in another one.

This is the worst failure category in the whole subject, because it is not an inconvenience. It is disclosing one person's information to another, which is a data protection incident regardless of how minor the detail seems, and it destroys trust in a way a wrong opening hour never will.

What we'll cover

• What it looks like in practice
• The four places it comes from
• Isolation as the default
• The identity problem underneath
• Building it with CodeWords
• Concurrency, and the same person twice
• How long state should live
• Testing for something that happens rarely
• If it has already happened
• Making it survive
• Limits worth knowing about
• What to check first
• Frequently asked questions

What it looks like in practice

Rarely as dramatic as a whole conversation appearing in the wrong place. Usually smaller and harder to notice.

The wrong name, which customers report and businesses dismiss as a glitch.

A detail from elsewhere. Referring to an order, an appointment or a preference belonging to somebody else.

Answering an unasked question, because the model was given history from another conversation.

Stale identity, where somebody is treated as a customer from months ago whose number has since been reassigned.

A reply to the wrong person entirely, which is the rare and serious version.

The first three get dismissed as the system being odd. They are the same defect as the fifth, caught earlier, and treating them as cosmetic is how the serious one eventually happens.

The four places it comes from

Shared state with no key. A variable holding "the current order" or "the customer's name" at the level of the application rather than the conversation. Works perfectly with one user and fails the moment there are two.

A key that is not unique enough. Keying on something that collides: a name, a session id that gets reused, an order reference two people can both quote.

History assembled from the wrong scope. Building the model's context by fetching recent messages without filtering strictly to this conversation. The model then answers using somebody else's words, faithfully.

Reuse after reassignment. Phone numbers are recycled. A record from two years ago attached to a number now belonging to somebody else means greeting a stranger with a previous owner's details.

The first two are implementation defects. The third is a design defect and the commonest in anything using a model. The fourth is a policy question that has to be decided rather than coded around.

Isolation as the default

The principle is that nothing is shared unless sharing was a deliberate decision.

Key everything on the conversation, and make the key explicit rather than implied by whatever happened to be in scope.

Fetch history filtered at the source, not filtered afterwards. Filtering after retrieval means the unfiltered set existed, and eventually something reads it.

Never hold per-user data outside a request in anything long-lived. This is the defect that appears under load and never in testing.

Pass identity explicitly through every layer rather than relying on ambient context. Ambient context is what leaks.

Separate genuinely shared data. Opening hours and prices are shared and should be obviously shared. Anything about a person is not.

Stated simply: if a piece of data is about a person, it needs that person's identifier attached to it everywhere it travels.

The identity problem underneath

Worth being precise, because the usual answer is not good enough.

A phone number identifies an account rather than a person. Numbers get reassigned. Phones get shared within families and small businesses. Somebody may message from a work number about a personal matter. And a number is trivially spoofable in other contexts, which matters when the conversation is the only evidence of who you are dealing with.

Three practical consequences.

Do not treat the number as proof of identity for anything consequential. Verify separately before disclosing anything sensitive or acting on an instruction that matters.

Age out old associations. A record attached to a number nobody has used in a long time is more likely to be wrong than useful.

Re-confirm rather than assume. Opening with "Is that Ms Patel?" costs one exchange and prevents the case where it is not.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder, including what the bot may say about somebody's existing records and when it must confirm who it is talking to.
3. Authorize the connections to your systems and your WhatsApp Business Platform account.
4. Describe the exceptions: an unrecognised number, a number matching a very old record, two conversations from the same person, anything sensitive.
5. Test with several conversations running at once rather than one at a time, since sequential testing is exactly what hides this defect.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Concurrency, and the same person twice

Two situations that produce the same symptom.

Two people messaging simultaneously, which is the case shared state fails at. It does not appear in testing because testing is sequential, and it appears immediately in production at any volume.

One person sending two messages quickly. Extremely common, since people send a thought and then a correction. If both are processed at once, they can interleave, and the second can be answered using state the first was halfway through changing.

The second is worth handling explicitly: process messages from one conversation in order, and consider briefly waiting to see whether more are coming before replying. It makes the bot feel more attentive and removes a whole class of confusion.

How long state should live

Conversation state that lives forever is both a leak risk and a correctness problem, since a bot referring to something from four months ago is usually wrong.

Set a lifetime by what the conversation is for. An order in progress might reasonably persist for weeks; a general enquiry for hours. When it expires, the conversation starts cleanly rather than half-remembering.

Separately, apply a retention policy to stored conversations. Data protection expects a period you chose rather than accumulation by default, and a shorter history is a smaller problem if anything ever goes wrong. What genuinely matters belongs in your own records rather than in conversation state.

Testing for something that happens rarely

This defect hides from ordinary testing, so test for it deliberately.

Run conversations in parallel, several at once, with deliberately similar details. Two people called the same thing, ordering the same item, on the same day.

Send bursts from one conversation without waiting for replies.

Check what the model was actually given. Log the assembled context and inspect it for anything not belonging to this conversation. This single check finds the commonest version of the defect.

Test with a recycled number, by pointing an old record at a new conversation and confirming the bot does not assume.

Do this after every change to how context is assembled, since it is the kind of defect that gets reintroduced.

If it has already happened

Treat it as an incident rather than a bug.

Establish what was disclosed, to whom, and how many people were affected. Stop the automation if you cannot bound it. Tell the people involved plainly, because they will usually have seen it and a business that says nothing looks worse than one that explains. Consider whether it is reportable under your data protection obligations, which depends on what was disclosed rather than on how it happened. And fix the cause rather than the symptom, since one instance almost always means others went unnoticed.

The tempting response is to treat a single odd message as a glitch. It is worth resisting.

Making it survive

Key every piece of personal data on the person it belongs to.

Filter history at the source, not afterwards.

Log the assembled context so it can be inspected.

Age out old number associations rather than trusting them indefinitely.

Re-run parallel-conversation tests after any change to context handling.

Limits worth knowing about

A phone number is not an identity, and no amount of care in your code changes that.

Sequential testing cannot find this. The defect requires concurrency to appear.

Model context is the easiest place to leak, because assembling it feels like a retrieval detail rather than a boundary.

Longer memory is not better. More retained state is more to leak and more to be wrong about.

One visible instance implies others, since most cases are never reported.

What to check first

Log what the model is actually being given, and read it. Not what you believe it is given, the real assembled context for a real conversation. The most common version of this defect is history fetched with a filter that is slightly too broad, and it is invisible until somebody looks at the input.

Two habits make the difference. Test with several conversations running at once, using deliberately similar details, because sequential testing is structurally incapable of finding this. And treat a wrong name as the same defect as a wrong order, caught earlier, rather than as a cosmetic glitch.

Frequently asked questions

What is context bleeding?

State from one person's conversation appearing in another's: a wrong name, a detail from somebody else's order, or an answer to a question asked elsewhere. It is a disclosure of one customer's information to another.

What causes it?

Shared state with no key, a key that is not unique enough, model context assembled from too broad a scope, and old records attached to reassigned phone numbers.

Why does it not show up in testing?

Because testing is sequential and the defect needs concurrency. It appears immediately in production at any real volume.

Is a phone number enough to identify a customer?

No. Numbers are reassigned, phones are shared, and a number identifies an account rather than a person. Verify separately before disclosing anything sensitive or acting on a consequential instruction.

How should conversation history be handled?

Filter at the source rather than after retrieval, and log the assembled context so it can be inspected. Broad retrieval with a later filter means the unfiltered set existed.

What about two messages sent quickly by the same person?

Process them in order rather than concurrently, and consider briefly waiting to see whether more are coming. Interleaved processing produces answers based on half-updated state.

How long should conversation state live?

As long as the conversation is genuinely for, then expire it. An order might persist for weeks, a general enquiry for hours. Indefinite state is both a leak risk and usually wrong.

How do I test for it?

Run several conversations in parallel with deliberately similar details, send bursts without waiting for replies, inspect the assembled model context, and test a recycled number against an old record.

Is a wrong name a serious problem?

It is the same defect as a wrong order, caught earlier. Dismissing it as cosmetic is how the serious version eventually reaches a customer.

What should I do if it has happened?

Treat it as an incident: establish what was disclosed and to whom, stop the automation if you cannot bound it, tell the people affected, consider whether it is reportable, and fix the cause rather than the instance.

Related reading

• What you are now holding
• Adding memory to a bot
• Testing it before customers do