What is a dead letter queue, and who is actually reading it?
A dead letter queue catches what could not be processed. Unmonitored, it is a place work goes to be forgotten — which is worse than not having one, because it looks like handling.
Amman Vedi11 min read
On this page
A dead letter queue is where messages go when they cannot be processed. After a configured number of failed attempts, the message is moved out of the main queue so it stops blocking or consuming capacity, and lands somewhere it can be examined.
The mechanism is simple and the failure around it is almost universal: nobody reads it. An unmonitored dead letter queue is a directory where lost work accumulates silently, and it is worse than having none, because its existence creates the impression that failures are being handled.
What we'll cover
What it is for
Preventing one bad message from stopping everything. Without somewhere to put it, a message that always fails is retried indefinitely, consuming capacity and in ordered queues blocking everything behind it.
Preserving the message rather than discarding it, so the work is not lost and the failure can be investigated.
Separating transient from permanent failure. A message that failed because a service was briefly down should be retried; one that failed because it is malformed should not be retried forever.
Giving failure a place, which is the precondition for anybody looking at it.
It is a holding area, not a solution. The message is still unprocessed, and something has to happen to it.
The poison message
The thing dead letter queues exist for, and it comes in a few shapes.
Malformed content. The message does not parse, or is missing a field the consumer requires.
A schema change. The producer started sending a new shape and the consumer, deployed separately, cannot read it.
Referenced data that no longer exists. The message refers to a record that has since been deleted.
A bug in the consumer triggered by a specific case, which is the most common cause and the one that produces a dead letter queue full of near-identical messages.
Genuinely bad data from upstream, which needs a person rather than a retry.
Notice that most of these are not the message's fault. A dead letter queue full of one shape of message is a bug report, not a rubbish bin.
Retry before dead lettering
The policy between the first failure and the dead letter queue matters as much as the queue itself.
Distinguish transient from permanent. A timeout or a rate limit deserves a retry; a parse failure does not, and retrying it three times before dead lettering just delays the inevitable by a few seconds.
Fail fast on permanent errors. If the consumer can tell the message will never succeed, send it straight to the dead letter queue with the reason.
Use exponential backoff with jitter for transient failures, so a recovering dependency is not immediately overwhelmed by every consumer retrying in unison.
Cap the attempts. Unlimited retries are how a single message consumes a consumer indefinitely.
Record the attempts and the errors on the message when you dead letter it, because a message with no failure reason is one somebody has to reproduce before they can act.
Why they go unread
Worth naming honestly, since the fix follows from the cause.
Nobody owns it. It was configured as part of the infrastructure and assigned to no team.
No alert. Messages arrive and nothing happens, so it is discovered when somebody investigates a downstream discrepancy weeks later.
Nothing says what went wrong. Opening it reveals messages with no error attached, so understanding one means reproducing it.
No replay mechanism. Even having diagnosed the problem, getting the messages reprocessed is a manual exercise nobody wants.
It is embarrassing. A queue with four thousand messages in it is a problem nobody wants to be the one to raise.
Making one that gets used
Alert on arrival, not on depth. The first message is the signal; by the time depth is notable, the problem is old.
Attach the reason. The error, the attempt count, and the consumer version, on the message, so triage does not require reproduction.
Group by error. Four thousand messages with one cause is one fix, and presenting them as four thousand problems guarantees nobody starts.
Give it an owner, named, the same as any other operational surface.
Report age. How long the oldest message has been there is the number that says whether this is being handled.
Review it weekly even when empty, so the habit exists before the day it is not empty.
Replaying safely
Getting messages back into the system is the half that makes the whole mechanism worthwhile.
Fix the cause first. Replaying into an unchanged consumer produces the same failures and a second copy of the problem.
Replay a few, then the rest. A sample confirms the fix without committing everything.
Consumers must be idempotent, since replayed messages may have partially succeeded before failing.
Watch for ordering. Replayed messages arrive after everything that came later, so a consumer that assumes order needs to handle the out-of-sequence case.
Keep a record of what was replayed and when, because a downstream discrepancy weeks later is much easier to explain with it.
Discard deliberately, not by default. Some messages should be dropped — the record no longer exists, the business no longer cares — and that should be a decision somebody recorded rather than an expiry.
What should not go there
Expected business outcomes. A payment declined is a normal result, not a processing failure, and routing it to the dead letter queue confuses an operational surface with a business one.
Validation failures that should be rejected upstream. If the producer can detect it, the message should not be published.
Anything with a defined alternative path. An order that cannot be fulfilled goes to a fulfilment exception process, not to infrastructure.
Transient failures that have not exhausted retries. Sending them early fills the queue with things that would have succeeded.
The rule of thumb: a dead letter queue is for messages the system could not process, not for cases the business did not want. Mixing the two makes both harder to see.
Where it sits in a wider failure strategy
A dead letter queue handles one kind of failure, and it is worth knowing which.
Retries handle transient failure — something was briefly unavailable and will work shortly.
Circuit breakers handle sustained failure of a dependency, stopping the attempts rather than hammering something that is down.
Dead lettering handles individual failure — this specific message cannot be processed while others can.
Alerting handles systemic failure, where the rate itself is the signal rather than any single message.
Each covers a different case and they are not substitutes. A system with only a dead letter queue turns a dependency outage into thousands of dead lettered messages, when a circuit breaker would have paused and resumed.
Decide which failures are which, per consumer, and write it down. The distinction between "retry", "stop trying for now", and "this one will never work" is a design decision, and consumers that do not make it explicitly make it accidentally.
Handling failures with CodeWords
CodeWords connects to more than 3,000 integrations, and failure handling is part of describing an automation rather than infrastructure to configure separately.
- Open CodeWords and start a new automation.
- Describe what should happen in plain language to Cody, the automation builder.
- Describe the exceptions explicitly: what should be retried, what should stop immediately, and who should be told when something cannot be processed.
- Connect the systems involved once and reuse the connections.
You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.
What to get right first
An alert on the first message arriving, routed to a named owner. Everything else on this page is refinement; a dead letter queue nobody is told about is functionally identical to discarding the messages, except that it also creates the impression the failures are handled.
Two habits make the difference. Attach the failure reason to the message when you dead letter it, since triage that requires reproducing the error is triage nobody does. And group by cause when reporting, because four thousand messages with one root cause is a single fix and presenting it as four thousand problems guarantees it stays untouched.
Frequently asked questions
What goes in a dead letter queue?
Messages the system could not process after its retry policy was exhausted — malformed content, schema mismatches, references to deleted data, or consumer bugs. Not expected business outcomes, which belong in a business exception path.
How long should I retry before dead lettering?
Distinguish the failure type. Transient failures deserve a few attempts with exponential backoff and jitter; permanent ones such as parse errors should go straight there, since retrying a malformed message changes nothing.
Why are dead letter queues usually ignored?
No owner, no alert, no failure reason attached, and no easy replay. Each one alone is enough to guarantee nobody looks, and most implementations have all four.
When should the alert fire?
On the first message. Alerting on depth means the problem is discovered after it has been happening for a while, and the first message is the point at which the cause is most easily identified.
How do I replay messages safely?
Fix the cause first, replay a sample, then the rest. Consumers must be idempotent, since messages may have partially succeeded, and expect ordering to be wrong because replayed messages arrive after everything that followed them.
Should messages expire from the queue?
Only as a deliberate decision that somebody recorded. An expiry configured for tidiness quietly discards work, which is exactly the outcome the dead letter queue existed to prevent.
Is a dead letter queue full of similar messages bad news?
It is good news, in a sense: it is one bug rather than many. Group by error before looking at volume, because the count is frightening and the number of distinct causes is usually very small.
Is a dead letter queue enough on its own?
No. It handles individual message failure. Transient failure wants retries, a dependency outage wants a circuit breaker, and a rising failure rate wants an alert. A system with only dead lettering turns an outage into thousands of dead-lettered messages.
How do I decide which failures are which?
Per consumer, explicitly, written down: what is worth retrying, what should pause, and what will never succeed. Consumers that do not make that decision deliberately end up making it by accident.
What should be reported weekly?
Whether any messages arrived, grouped by cause, and the age of the oldest. Review it even when empty — the habit of looking is what makes it useful on the week it is not, and by then the failure is usually several days old.