LESSON 19.2
Delivery guarantees
At-most-once, at-least-once and "exactly-once", what each costs, and how idempotent consumers give you effectively-once results.
6 min read
Networks lose messages and acknowledgements, and processes crash halfway through work. A messaging system can't avoid that; it can only choose which way to fail. That choice is its delivery guarantee.
Three guarantees
At-most-once
- Simple
- No duplicates
- Messages can be lost
Pick it when: Metrics or logs where losing a little is fine.
At-least-once
- Nothing is lost
- Duplicates happen; consumers must cope
Pick it when: The default for almost everything.
Exactly-once
- No loss, no duplicates
- Only within one system's boundary
- Extra coordination cost
Pick it when: Stream processing that reads and writes the same platform.
The difference is often just when you acknowledge:
- Ack (or commit the offset), then process. A crash in between loses the message: at-most-once.
- Process, then ack. A crash in between redelivers it: at-least-once.
- Make processing and the ack one atomic step, or make processing idempotent: effectively once.
Where duplicates come from
Even with careful code, duplicates arrive from several directions:
- A producer retries after a timeout, but the first send had actually succeeded.
- A consumer processes a batch and crashes before committing its offset.
- A rebalance moves a partition while its old owner is still finishing work.
- A queue's visibility timeout expires on a slow job, and another worker takes it.
So design every consumer for at-least-once input.
Idempotent consumers
The robust pattern: in one database transaction, insert the message id into a table with a unique constraint and apply the business change. A duplicate fails the unique insert and is skipped. When the change is naturally idempotent (setting a status, upserting a row by key), you may not need the table at all.
Exactly-once, honestly
Kafka offers idempotent producers (the broker drops a producer's retried duplicates) and transactions that atomically write output messages and commit input offsets. Stream processors like Flink and Kafka Streams use these for exactly-once results inside the platform. Anything outside (a database, an email, an API) still needs idempotency or a transactional sink.
Try it
Webhook Delivery HLD
Deliver webhooks reliably with retries, signing and idempotency.