LESSON 20.3

The outbox and inbox patterns

Publish events reliably by writing them in the same transaction as the data, relay them with polling or CDC, and deduplicate on the receiving side.

5 min read

Sagas and event-driven services depend on one thing: when a service changes its data, the matching event must be published. The outbox pattern makes that reliable without distributed transactions.

The outbox

Laying out the diagram…

The order and its event commit together. A relay moves events to the broker.

In one local transaction, insert the order and insert an OrderPlaced row into outbox (id, aggregate id, type, payload, created time).
A relay reads new outbox rows and publishes them to the broker, keyed by aggregate id so per-order ordering holds.
After the broker acknowledges, the relay marks rows sent (or a cleanup job deletes old ones).

If the service crashes before commit, neither the order nor the event exists. After commit, the event will eventually be published, however many crashes happen along the way.

Relaying: polling or CDC

Polling relay

Simple, no extra infrastructure
Polling load on the primary
Latency equals the poll interval
Care needed with multiple relays

Pick it when: Modest volume; a 100 ms to 1 s delay is fine.

CDC (Debezium-style)

Millisecond latency
No polling queries
Ordered by commit
Runs a connector and needs logical replication

Pick it when: High volume, or CDC is already in place.

The inbox

The outbox guarantees at-least-once publishing, so receivers will occasionally see the same event twice. The inbox is the mirror image: in the consumer's transaction, insert the event id into an inbox table with a unique constraint, then apply the change. A duplicate fails the insert and is acknowledged without effect.

Practical details

Keep payloads self-contained (include what consumers need) and versioned, like any public schema.
Index the outbox by unsent status and created time, and delete published rows so the table stays small.
Monitor the age of the oldest unsent row. It's the outbox equivalent of consumer lag.

Quick check

A team's relay polls the outbox with 4 instances for throughput, and consumers start seeing events for the same order out of order. What's a good fix?

Try it

E-commerce Checkout HLD

Design a checkout that stays consistent across orders, payments and inventory.

Check your understanding

1.How does the transactional outbox remove the dual-write gap?
2.The outbox relay publishes an event, then crashes before marking it as sent. What happens?
3.What's an advantage of reading the outbox through CDC instead of polling the table?

Practice it

E-commerce Checkout HLDCoordinate orders, stock and payment with idempotent steps and compensation.Change Data Capture Pipeline HLDBridge a consistent snapshot to a retained change log and replay downstream safely.

In the handbook

Multi-step processesBusiness processes that span services, minutes or days, such as orders, onboarding and payouts. State machines, sagas, durable workflow engines, timeouts and human steps.IdempotencyHandling retries safely without duplicate side effects. Idempotency keys, deduplication tables, consumer deduplication, and exactly-once processing via at-least-once delivery with deduplication.

Your notes

20. Transactions across services

LESSON 20.3

The outbox and inbox patterns

Publish events reliably by writing them in the same transaction as the data, relay them with polling or CDC, and deduplicate on the receiving side.

5 min read

The outbox

Laying out the diagram…

The order and its event commit together. A relay moves events to the broker.

In one local transaction, insert the order and insert an OrderPlaced row into outbox (id, aggregate id, type, payload, created time).
A relay reads new outbox rows and publishes them to the broker, keyed by aggregate id so per-order ordering holds.
After the broker acknowledges, the relay marks rows sent (or a cleanup job deletes old ones).

If the service crashes before commit, neither the order nor the event exists. After commit, the event will eventually be published, however many crashes happen along the way.

Relaying: polling or CDC

Polling relay

Simple, no extra infrastructure
Polling load on the primary
Latency equals the poll interval
Care needed with multiple relays

Pick it when: Modest volume; a 100 ms to 1 s delay is fine.

CDC (Debezium-style)

Millisecond latency
No polling queries
Ordered by commit
Runs a connector and needs logical replication

Pick it when: High volume, or CDC is already in place.

The inbox

Practical details

Keep payloads self-contained (include what consumers need) and versioned, like any public schema.
Index the outbox by unsent status and created time, and delete published rows so the table stays small.
Monitor the age of the oldest unsent row. It's the outbox equivalent of consumer lag.

Quick check

A team's relay polls the outbox with 4 instances for throughput, and consumers start seeing events for the same order out of order. What's a good fix?

Try it

E-commerce Checkout HLD

Design a checkout that stays consistent across orders, payments and inventory.

Check your understanding

1.How does the transactional outbox remove the dual-write gap?
2.The outbox relay publishes an event, then crashes before marking it as sent. What happens?
3.What's an advantage of reading the outbox through CDC instead of polling the table?

Practice it

In the handbook

Your notes