LESSON 3.2
What one machine can do
Rough capacity numbers for servers, databases, caches and queues, so you know when one box is enough and when it is not.
6 min read
Estimates are only useful if you can compare them to something. This lesson gives you rough numbers for what common building blocks can handle, so that "12,000 reads a second" immediately means something.
Rough capacity by component
- Stateless API serverdepends heavily on the work per request
- 1k–10k req/s
- Postgres / MySQL, simple indexed readsdata in RAM; complex queries far fewer
- 5k–50k queries/s
- Postgres / MySQL, writesdepends on indexes, fsync, row size
- 1k–10k+ writes/s
- Redis, single nodesingle-key GET/SET
- 100k+ ops/s
- Kafka, per brokerhundreds of thousands of small messages/s
- 100s of MB/s
- Elasticsearch, per nodedepends on query and index size
- 100s–1,000s of searches/s
- Network cardabout 1–3 GB/s
- 10–25 Gbit/s
- Server RAMup to several TB on large instances
- 64–512 GB typical
- Local SSD100k+ random IOPS
- several TB
Reading these numbers
Most apps fit on one database for a long time. A system doing 1,000 writes a second is doing 86 million writes a day. That is a big, successful product, and one well-provisioned relational database can usually keep up, with replicas for reads. Sharding is a big step; take it when the numbers force you to, not before.
Caches change the scale of the problem by about 10x. If a single database node handles 10k reads a second and a single Redis node handles 100k+, putting hot reads in Redis buys an order of magnitude almost for free.
Servers are easy to multiply. Stateless API servers scale horizontally: if one handles 2,000 requests a second and you need 20,000, run ten to fifteen of them behind a load balancer, with headroom. The hard part is almost always the data, not the compute.
From estimate to architecture
Once you have a number, compare it to the table:
| Estimated peak | What it usually means |
|---|---|
| Under ~1k req/s | One server and one database are fine. Add a second server for availability. |
| ~1k–10k reads/s | Load balancer with several servers; a cache or read replicas. |
| ~10k–100k reads/s | Caching is essential; several replicas; CDN for anything static. |
| Writes over ~10k/s sustained | One primary database will struggle; consider sharding, a write-optimized store or batching through a queue. |
| Data beyond a few TB, growing fast | Plan for partitioning, archival or a distributed database. |
Leave headroom
Never plan to run a machine at 100%. Queues build up long before full utilization: as a server gets busier, waiting time grows sharply, and at 90% busy a small burst can double latency. Plan for peak load at roughly 50–70% utilization, which also leaves room to lose a machine.
See it for yourself. This service runs on two instances at 3,100 requests a second, about 78% busy. Press Run under load and watch the p99. Then click API service, raise its instances to 3 or 4, and run again.
Try it under load
Nothing is failing at 78%, but the tail latency is already well past the goal. Each extra instance buys headroom, and headroom is what keeps p99 flat.
Try it
Metrics Ingestion HLD
A write-heavy design where the numbers decide everything.