LESSON 3.1
Latency numbers worth knowing
How long memory, disk, network and cross-region calls take, and why the differences between them shape every design.
5 min read
You do not need exact numbers to design well. You need a feel for the orders of magnitude, because the gaps between them are enormous, and they explain why caches, CDNs, batching and data locality exist at all.
The numbers
These are rough, modern figures. Remember the order of magnitude, not the digits.
- Read from CPU cache
- ~1 ns
- Read 1 MB sequentially from memory
- ~10–50 µs
- Redis / Memcached GET (same data center)mostly network round trip
- ~0.2–0.5 ms
- Round trip within a data center
- ~0.5 ms
- Random read from an SSD
- ~0.1 ms
- Indexed lookup in Postgres (cached in RAM)including network
- ~1 ms
- Read 1 MB sequentially from SSD
- ~1 ms
- Query that scans many rows or hits disk
- 10–100+ ms
- Round trip between regions on one continent
- ~20–50 ms
- Round trip across an ocean
- ~80–150 ms
- Random seek on a spinning hard disk
- ~5–10 ms
A useful way to feel these: if reading from memory took one second, a same-datacenter network round trip would take about a week, and a cross-ocean round trip several years.
What the numbers tell you
Memory is the fast path
Anything in RAM, whether in your process or in a cache like Redis, is orders of magnitude faster than going to disk or making a complex query. That is the whole case for caching: answer the common question from memory.
The network dominates
Inside a data center, most of a cache or simple database call is network round trip, not work. That has two consequences:
- Avoid chatty patterns. Ten sequential calls of 1 ms each is 10 ms. One batched call is 1 ms. Fetch what you need in one round trip when you can.
- Parallelize independent calls. Three independent 20 ms calls take 60 ms in sequence and 20 ms in parallel.
Distance is physics
Light in fiber travels about 200 km per millisecond. Mumbai to Virginia is roughly 13,000 km, so a single round trip can never be faster than about 130 ms, and in practice is more. No amount of code optimization fixes that. Only moving data closer to users does: CDNs, edge caches, regional replicas.
Percentiles, not averages
Latency is reported as percentiles: p50 (the median), p95, p99. Averages hide the slow requests, and the slow requests are what users notice.
Tail latency matters more as systems grow. If one page load fans out to 100 backend calls, and each call is slow 1% of the time, then most page loads include at least one slow call. That is why designs talk about p99, timeouts and hedged requests, not average response time.