Event-driven architecture explained: what you give up for decoupling

In an event-driven system, a component announces that something happened and does not know or care who reacts. That is the whole idea, and it produces a genuine benefit: adding a new consumer requires no change to the producer, which is what makes such systems easy to extend.

It also produces the characteristic cost. Nobody knows who is listening, which means nobody can tell you what a change will affect, and a request that fails somewhere downstream fails without a stack trace connecting it to what started it.

What we'll cover

• Events, commands, and the difference
• What decoupling actually buys
• What it costs
• Designing an event worth publishing
• Schema evolution, which is forever
• Eventual consistency reaches the interface
• Debugging without a call stack
• The dual write problem
• Connecting event-driven systems with CodeWords
• What to get right first
• Frequently asked questions

Events, commands, and the difference

Worth being strict about, because conflating them removes the benefit.

An event states a fact about the past. "Order placed." It is named in the past tense, it has already happened, and the publisher does not care what follows.

A command asks for something to happen. "Charge the card." It is addressed to a specific handler and the sender expects it to be carried out.

A command dressed as an event is the common mistake. An "event" called SendWelcomeEmail is a command with extra indirection — the publisher clearly knows what should happen and has simply hidden the coupling.

The test is whether you would still publish it if nobody listened. A genuine event is a fact worth recording regardless; a command is pointless with no handler.

Both are legitimate. The mistake is not using commands, it is calling them events and then being surprised that the system is coupled.

What decoupling actually buys

New consumers cost nothing to add. A team can react to orders without the order service knowing they exist, which is the single largest practical benefit.

Failure isolation. A consumer being down does not fail the publisher, because publishing succeeded and the consumer catches up.

Load absorption. A burst is buffered rather than rejected.

Replay. With a log-based transport, a new consumer can process history, and a bug can be fixed and the affected range reprocessed.

An audit trail. The event stream is a record of what happened, which is frequently valuable independent of the architecture.

What it costs

Nobody knows who is listening. Changing an event means finding every consumer, and there is no compiler to tell you.

No call stack. A failure in a consumer is disconnected from the request that caused it, so tracing must be built deliberately rather than inherited.

Eventual consistency. The world is briefly inconsistent after every event, and that inconsistency reaches users.

At-least-once delivery. Consumers must tolerate duplicates as normal behaviour, not as an edge case.

Ordering is weak. Concurrent consumers, retries, and partitioning all mean events are processed out of the order they occurred.

Local development is harder, because reproducing a flow means running or simulating the transport and every participant.

Designing an event worth publishing

The decisions here determine whether consumers can be added later without changing the producer.

Name it for the fact, in the past tense. OrderPlaced, not ProcessOrder.

Include what a consumer needs, so most can act without calling back. An event carrying only an identifier forces every consumer to fetch, which reintroduces coupling and load on the producer.

Do not include everything either. A fat event carrying the entire aggregate becomes a schema every consumer depends on, and changing it becomes impossible.

Carry identity, timestamp, and version on every event, always. The identity enables deduplication, the timestamp enables ordering, and the version enables evolution.

Publish facts, not intentions. PaymentFailed is a fact; RetryPayment is a command.

One event per fact. Combining several state changes into one event forces consumers to interpret which part concerns them.

Schema evolution, which is forever

The constraint that shapes long-lived event systems.

Events are a published contract, consumed by people you may not know about, including future consumers replaying history.

Additive changes are safe. New optional fields do not break existing consumers, provided those consumers ignore unknown fields — which they must be written to do from the start.

Removals and renames are breaking. So are type changes and tightened validation.

Old events exist forever in a replayable log, so a consumer reading history must handle every version ever published.

Version explicitly and support several versions rather than migrating a stream.

Use a schema registry or its equivalent, so a producer cannot publish a breaking change without something noticing. This is the control that actually prevents the problem rather than documenting it.

Eventual consistency reaches the interface

The part that surprises product teams rather than engineers.

A user completes an action and the result is not visible yet, because the consumer that updates the view has not processed the event.

The window is usually small and occasionally is not, particularly when a consumer is behind.

Design for it explicitly rather than hoping it is fast enough: show the optimistic result, show a pending state, or wait — all are reasonable and the choice is a product decision.

Do not read your own write from a lagging projection. A user who just changed something and sees the old value assumes the change failed and does it again.

Measure consumer lag and treat it as a user-facing metric, because it is.

Debugging without a call stack

Correlation identifiers on every event, propagated through every consumer, are not optional — they are what replaces the stack trace.

Include causation as well as correlation: which event caused this one, so a chain can be reconstructed rather than a set of coincidences.

Log the event identifier on both publish and consume, so a specific event can be followed.

Distributed tracing is the tooling answer, and it has to be instrumented before you need it rather than during an incident.

Keep a dead letter queue with an owner. Events that could not be processed accumulate somewhere, and if nobody looks, the system is losing work while appearing healthy.

The dual write problem

A failure mode specific to this architecture, common, and usually discovered in production.

Writing to your database and publishing an event are two operations with no shared transaction.

Either can fail after the other succeeds. A database write with no event means consumers never learn; an event with no write means consumers act on something that did not happen.

Retrying does not fix it, because the failure is in the gap between the two rather than in either one.

The outbox pattern is the standard answer: write the event into a table in the same transaction as the data, and a separate process publishes from that table. The transaction makes the two atomic and the publisher makes delivery at-least-once.

Change data capture is the other route, deriving events from the database's own log so there is only ever one write.

Publishing before committing is always wrong, and it is the shape people reach for first because it reads naturally in code.

Connecting event-driven systems with CodeWords

Much of the value in an event-driven system is in the consumers, and many of them are integrations rather than services. CodeWords connects to more than 3,000 integrations.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which event, and what should result.
3. Describe the exceptions: a duplicate event, one that arrives out of order, one that cannot be processed.
4. Connect the systems involved once and reuse the connections.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

What to get right first

Event design: named for the fact, in the past tense, carrying identity, timestamp, version, and enough data for a consumer to act. It is the contract everything else depends on, it is the hardest thing to change later, and getting it wrong means every consumer calls back to the producer and you have built a distributed monolith.

Two habits make the difference. Propagate a correlation identifier through every event from the first one, since retrofitting tracing into a live event system is painful and you will want it during the first incident. And write consumers to ignore unknown fields from the start, because that single property is what makes additive schema changes safe for the next several years.

Frequently asked questions

What is the difference between an event and a command?

An event states a fact about the past and the publisher does not care who reacts; a command asks a specific handler to do something. The test: would you still publish it if nobody listened? If not, it is a command.

How much data should an event carry?

Enough that most consumers can act without calling back, and not the entire aggregate. Too little reintroduces coupling and load on the producer; too much creates a schema nobody can ever change.

How do I change an event schema?

Additively — new optional fields, with consumers written to ignore unknown ones. Removals, renames, and type changes are breaking, and old events live forever in a replayable log, so version explicitly and support several versions.

Why does the user see stale data?

Eventual consistency: the consumer that updates the view has not processed the event yet. Design for it explicitly with an optimistic or pending state, and never read a user's own recent write from a lagging projection.

How do I debug a failure with no stack trace?

Correlation and causation identifiers propagated through every event, plus distributed tracing instrumented in advance. There is no retrofitting this during an incident, which is why it belongs in the first version.

Do consumers need to handle duplicates?

Yes, as normal behaviour. At-least-once delivery means retries, redeliveries after a crash, and reprocessing all produce duplicates, and a consumer that assumes exactly-once will eventually do something twice.

When is event-driven architecture the wrong choice?

When the caller needs the answer, when there is one team and one deployment, or when nobody will build the tracing and monitoring it requires. The decoupling is worth paying for only if you are actually using it.

What is the dual write problem?

Writing to your database and publishing an event are two operations with no shared transaction, so either can fail after the other succeeds — leaving consumers either uninformed or acting on something that never happened.

How do I fix it?

The outbox pattern: write the event to a table in the same transaction as the data, and publish from that table separately. Change data capture is the alternative. Publishing before committing is the intuitive shape and it is always wrong.

How do I know who consumes an event?

Only by keeping a register, because nothing in the system tells you. A schema registry plus a recorded list of consumers per event is what makes a change assessable — otherwise "who will this break" has no answer.

Related reading

• Message queues explained
• Pub/sub messaging explained
• Microservices vs monolith