LESSON 8.1
Scale up or scale out
Vertical versus horizontal scaling, why stateless servers make scaling out easy, and where state should live instead.
5 min read
Sooner or later one server isn't enough: too much traffic, too much data, or too much risk riding on one machine. There are two directions to grow, and most designs use both in different places.
Vertical scaling: a bigger machine
Add CPU, memory and faster disks to the server you have. It needs no code changes and no new failure modes, and modern machines are enormous: hundreds of cores and terabytes of RAM.
Its limits:
- A ceiling. At some point you're on the largest machine available.
- Cost. The biggest machines cost disproportionately more.
- One point of failure. A bigger single server still fails, gets patched and restarts.
Vertical scaling is a great answer for databases early on and for any component that is hard to split. Saying "one larger database instance handles this for the next two years" is often more convincing than an elaborate sharding plan.
Horizontal scaling: more machines
Run many copies of the server behind a load balancer. Capacity grows by adding machines, losing one costs a fraction of capacity, and you can deploy by replacing machines a few at a time.
The catch is in that last box. Horizontal scaling only works if any server can handle any request, which means the servers must be stateless.
Moving state out of the servers
Anything a server remembers between requests is state. Each kind has a better home:
| State | Where it should live |
|---|---|
| Login sessions | A signed token (such as a JWT) the client sends back, or a session store like Redis |
| Shopping carts, drafts | The database, or Redis with persistence |
| Uploaded files | Object storage, uploaded directly from the client with a presigned URL |
| Caches shared across requests | A distributed cache |
| Background work in progress | A queue, not a thread on the web server |
Which to choose
Scale up
- No code changes
- No distributed-systems problems
- Hard ceiling
- Single point of failure
- Expensive at the top
Pick it when: Databases and stateful components early on, and anything hard to split.
Scale out
- Near-unlimited capacity
- Survives machine failures
- Rolling deploys
- Needs stateless design
- More moving parts
Pick it when: Stateless application servers, workers, caches. The default for web tiers.
In an interview, the expected default is: stateless application servers scaled horizontally behind a load balancer, with state pushed to databases, caches and object storage that have their own scaling story.
Cut a leader off from the majority and compare its local log with what it can actually commit.
Try it
Authentication Session Service HLD
Design where sessions live so any server can serve any user.