BlogEngineering

Event-driven architecture explained: what you give up for decoupling

Event-driven architecture explained: the benefit and cost of decoupling, with event design, schema evolution and debugging without a call stack.

Rebecca PearsonRebecca Pearson11 min read

Summarize with AI

Event-driven architecture explained: what you give up for decoupling
On this page

Event-driven architecture is a design where a component announces that something happened and does not know which other components react. The benefit is that you can add a new consumer without changing the producer, which makes the system easy to extend.

The cost is that nobody knows who is listening. Nobody can tell you what a change will affect, and a request that fails downstream fails without a stack trace connecting it to what started it.

What we'll cover

Key takeaways

  • An event states a fact about the past, and a command asks a specific handler to act; calling a command an event hides coupling instead of removing it.
  • Decoupling lets you add consumers without changing the producer, at the cost of tracing, ordering and consistency problems.
  • Events are a published contract: additive changes are safe, while removals, renames and type changes break consumers.
  • Writing to a database and publishing an event are two separate operations, and the outbox pattern makes them atomic.
  • Add correlation identifiers to every event from the first version, because you cannot add tracing during an incident.

Events, commands, and the difference

An event states a fact about the past, and a command asks a specific handler to do something. Keep the two apart, because mixing them removes the benefit of decoupling.

An event states a fact about the past. "Order placed" is named in the past tense, has already happened, and the publisher does not care what follows.

A command asks for something to happen. "Charge the card" is addressed to a specific handler, and the sender expects it to be carried out.

A command dressed as an event is the common mistake. An "event" called SendWelcomeEmail is a command with extra indirection. The publisher knows what should happen and has hidden the coupling.

Test it by asking whether you would still publish it if nobody listened. A genuine event is a fact worth recording regardless. A command is pointless without a handler.

Both are legitimate. The mistake is calling commands events and then being surprised that the system is coupled.

What decoupling buys

Decoupling buys five things: cheap new consumers, failure isolation, load absorption, replay and an audit trail.

New consumers cost nothing to add. A team can react to orders without the order service knowing the team exists. This is the largest practical benefit.

Failure isolation. A consumer that is down does not fail the publisher, because publishing succeeded and the consumer catches up.

Load absorption. The system buffers a burst instead of rejecting it.

Replay. With a log-based transport, a new consumer can process history, and you can fix a bug and reprocess the affected range.

An audit trail. The event stream records what happened, which has value independent of the architecture.

What decoupling costs

Decoupling costs visibility and simplicity: you lose the list of consumers, the call stack, and strong consistency, ordering and delivery guarantees.

Nobody knows who is listening. Changing an event means finding every consumer, and no compiler tells you.

No call stack. A consumer failure is disconnected from the request that caused it, so you must build tracing deliberately.

Eventual consistency. The system is briefly inconsistent after every event, and that inconsistency reaches users.

At-least-once delivery. Consumers must treat duplicates as normal behaviour, not as an edge case.

Weak ordering. Concurrent consumers, retries and partitioning all mean events are processed out of the order they occurred.

Harder local development. Reproducing a flow means running or simulating the transport and every participant.

Designing an event worth publishing

A well-designed event is named for a fact, carries identity, timestamp and version, and contains enough data for consumers to act. These choices decide whether you can add consumers later without changing the producer.

Name it for the fact, in the past tense. Use OrderPlaced, not ProcessOrder.

Include what a consumer needs, so most consumers can act without calling back. An event that carries only an identifier forces every consumer to fetch, which brings back coupling and load on the producer.

Do not include everything. A fat event that carries the entire aggregate becomes a schema every consumer depends on, and you can no longer change it.

Carry identity, timestamp and version on every event. Identity enables deduplication, the timestamp enables ordering, and the version enables evolution.

Publish facts, not intentions. PaymentFailed is a fact; RetryPayment is a command.

Publish one event per fact. Combining several state changes into one event forces consumers to work out which part concerns them.

Schema evolution, which is forever

Event schemas are a published contract, so evolve them only by additive changes. Consumers include people you may not know about, and future consumers that replay history.

Treat additive changes as safe. New optional fields do not break existing consumers, provided consumers ignore unknown fields. Write consumers to do this from the start.

Treat removals and renames as breaking. So are type changes and tightened validation.

Remember old events exist forever in a replayable log, so a consumer that reads history must handle every version ever published.

Version explicitly and support several versions instead of migrating a stream.

Use a schema registry or an equivalent, so a producer cannot publish a breaking change unnoticed. This control prevents the problem instead of documenting it.

How eventual consistency reaches the interface

Eventual consistency reaches the interface when a user completes an action and the result is not visible yet, because the consumer that updates the view has not processed the event. Product teams are surprised by this more often than engineers.

Expect a small window that is occasionally large, particularly when a consumer falls behind.

Design for the window explicitly. Show the optimistic result, show a pending state, or wait. All three are reasonable, and the choice is a product decision.

Do not read a user's own write from a lagging projection. A user who changed something and sees the old value assumes the change failed and repeats it.

Measure consumer lag and treat it as a user-facing metric.

Debugging without a call stack

Replace the call stack with correlation and causation identifiers on every event, propagated through every consumer. Instrument distributed tracing before an incident, not during one.

Add correlation identifiers to every event and propagate them through every consumer. They replace the stack trace.

Include causation as well as correlation. Record which event caused this one, so you can reconstruct a chain instead of a set of coincidences.

Log the event identifier on both publish and consume, so you can follow a specific event.

Use distributed tracing as the tooling answer, and instrument it before you need it.

Keep a dead letter queue with an owner. Events that could not be processed accumulate there. If nobody looks, the system loses work while appearing healthy.

The dual write problem

The dual write problem is a failure where a database write and an event publish succeed or fail independently. It is common and usually discovered in production.

Writing to your database and publishing an event are two operations with no shared transaction.

Either can fail after the other succeeds. A database write with no event means consumers never learn. An event with no write means consumers act on something that did not happen.

Retrying does not fix it, because the failure is in the gap between the two operations.

The outbox pattern is the standard answer. Write the event into a table in the same transaction as the data, and a separate process publishes from that table. The transaction makes the two writes atomic, and the publisher makes delivery at-least-once.

Change data capture is the other route. It derives events from the database's own log, so there is only one write.

Publishing before committing is always wrong. It is the shape people reach for first because it reads naturally in code.

Connecting event-driven systems with CodeWords

Many consumers in an event-driven system are integrations, not services, and CodeWords can act as those consumers. CodeWords connects to more than 3,000 integrations (CodeWords pricing page, 2026).

  1. Open CodeWords and start a new automation.
  2. Describe to Cody, the automation builder, what should happen in plain language: which event, and what should result.
  3. Describe the exceptions: a duplicate event, one that arrives out of order, one that cannot be processed.
  4. Connect the systems involved once and reuse the connections.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows (CodeWords pricing page, 2026). Details are on the pricing page.

What to get right first

Get event design right first: name each event for the fact, in the past tense, carrying identity, timestamp, version and enough data for a consumer to act. Everything else depends on this contract, and it is the hardest thing to change later. A poor design makes every consumer call back to the producer, and you have built a distributed monolith.

Two habits make the difference. Propagate a correlation identifier through every event from the first one, because retrofitting tracing into a live event system is painful and you will want it in the first incident. And write consumers to ignore unknown fields from the start, because that property keeps additive schema changes safe for years.

Frequently asked questions

What is the difference between an event and a command?

An event states a fact about the past, and the publisher does not care who reacts. A command asks a specific handler to do something. To tell them apart, ask whether you would still publish it if nobody listened. If not, it is a command.

How much data should an event carry?

An event should carry enough data that most consumers can act without calling back, and not the entire aggregate. Too little data brings back coupling and load on the producer. Too much creates a schema that nobody can change.

How do I change an event schema?

Change it additively, with new optional fields and consumers that ignore unknown fields. Removals, renames and type changes are breaking. Old events live forever in a replayable log, so version explicitly and support several versions.

Why does the user see stale data?

Stale data appears because of eventual consistency: the consumer that updates the view has not processed the event yet. Design for this with an optimistic or pending state, and never read a user's own recent write from a lagging projection.

How do I debug a failure with no stack trace?

Propagate correlation and causation identifiers through every event, and instrument distributed tracing in advance. You cannot add these during an incident, which is why they belong in the first version of the system.

Do consumers need to handle duplicates?

Yes. At-least-once delivery means retries, redeliveries after a crash and reprocessing all produce duplicates. A consumer that assumes exactly-once delivery will eventually do something twice, so make every consumer safe to run repeatedly.

When is event-driven architecture the wrong choice?

It is the wrong choice when the caller needs the answer, when there is one team and one deployment, or when nobody will build the tracing and monitoring it requires. The decoupling is worth paying for only if you use it.

What is the dual write problem?

The dual write problem occurs because writing to a database and publishing an event are two operations with no shared transaction. Either can fail after the other succeeds, leaving consumers uninformed or acting on something that never happened.

How do I fix the dual write problem?

Use the outbox pattern: write the event to a table in the same transaction as the data, and publish from that table in a separate process. Change data capture is the alternative. Publishing before committing is always wrong.

How do I know who consumes an event?

Keep a register, because nothing in the system tells you. A schema registry plus a recorded list of consumers per event makes a change assessable. Without them, the question "who will this break" has no answer.

Get started today

Your first workflow is free to build.

Describe what you need. Cody handles the build, the connections, and the deployment.