LESSON 10.1
Why queues
Respond fast and do the slow work later, absorb bursts, and keep a failing dependency from taking the whole request path down.
7 min read
Some work doesn't belong in the request. Sending email, resizing images, charging cards, generating reports, calling slow partners: if the user waits for these, requests get slow, and when a dependency fails, requests fail with it. A message queue lets the request record what needs doing and return immediately, while workers do the work on their own schedule.
Three things queues buy you
- Fast responses. The request does the essential write, enqueues a job, and returns. A 3-second email send becomes a 2-millisecond enqueue.
- Absorbing bursts. If 10,000 uploads arrive in a minute, they queue up and workers process them at a steady rate instead of the whole system being sized for the peak.
- Isolating failures. If the email provider is down, jobs wait in the queue and are retried later. Sign-ups keep working.
It also decouples teams and services: producers don't need to know who consumes their messages or how many consumers there are.
The cost: work happens later
Asynchronous means the user doesn't see the result right away. That changes the product:
- Return 202 Accepted with a status the client can poll, or notify the user later (webhook, push, email).
- Decide what the user sees meanwhile: "Your video is processing."
- Accept that results arrive out of order, more than once, or after a delay, which the next lesson covers.
Backlogs and draining
A queue is only healthy if, over time, workers process at least as fast as jobs arrive. Watch the simulator: email jobs arrive faster than one worker can send them. Run it and watch the backlog chart, then click Email worker and add instances until the backlog stays flat. Notice that requests stay fast throughout, because the queue is off the critical path.
Try it under load
- Arrivals
- 900 jobs / s
- One worker
- 500 jobs / s
- Backlog growth
- 400 jobs / s, about 24,000 a minute
- Two workers
- 1,000 jobs / s: the backlog drains
Scale workers on queue depth or oldest message age, not CPU. Age is the number users feel: an email that's 20 minutes late is a problem whatever the servers' utilization.
Choosing a queue
Simple queue (SQS, RabbitMQ)
- Each message goes to one worker
- Per-message retries and dead-letter queues
- Once consumed, a message is gone
Pick it when: Background jobs and tasks.
Log / stream (Kafka, Kinesis)
- Ordered, replayable log
- Many independent consumer groups
- Ordering only within a partition
- Per-message retry is harder
Pick it when: Event streams that several systems consume, and analytics pipelines.
Database table as a queue
- No new infrastructure
- Transactional with your data
- Polling and locking costs at high volume
Pick it when: Low to moderate volume, especially with SELECT ... FOR UPDATE SKIP LOCKED.
Try it
Image Processing Pipeline HLD
Upload fast, process later: a classic queue-and-workers design.