LESSON 11.1
Why and where to limit
What rate limits protect, what to key them on, where they belong in a request path, and how well-behaved clients respond to them.
6 min read
A rate limiter caps how many requests a client may make in a period: 100 a minute, 10 a second, 5 login attempts an hour. It's a small piece of infrastructure that protects everything behind it, and a staple interview question. This module builds up from why to how; the guided rate limiter lab has you build one end to end.
What limits protect
- Capacity. One buggy script or traffic spike shouldn't be able to take the API down for everyone.
- Fairness. Customers share capacity; per-customer limits stop one from crowding out the rest, and paid tiers buy bigger limits.
- Security. Slowing password guessing, card testing, scraping and spam.
- Cost. Each request to an expensive downstream (an AI model, an SMS provider) costs money.
What to key on
A limit counts requests per key. The key decides who shares a budget.
| Key | Good for | Weak against |
|---|---|---|
| API key or user id | Fair use per customer, paid tiers | Unauthenticated endpoints |
| IP address | Anonymous traffic, crude abuse | Shared IPs (offices, mobile carriers), botnets |
| Account targeted (such as username on login) | Credential stuffing across many IPs | Nothing much; pair it with IP limits |
| Route or operation | Expensive endpoints like search or export | Needs combining with a per-client key |
| Global | A hard ceiling protecting a fragile dependency | Unfair: everyone shares one budget |
Real systems layer several: per IP at the edge, per API key and route at the gateway, and a global cap in front of a fragile downstream.
Where limits live
- At the edge (CDN, WAF): cheap, coarse per-IP limits that stop floods before they reach you.
- At the API gateway: per-key, per-route limits with plan-based quotas. Most "design a rate limiter" answers live here.
- Inside services: limits that protect a specific dependency, such as calls to a payment provider or a database-heavy operation.
- In the client: courtesy throttling. Useful, never trusted, because anyone can skip it.
Telling clients they're limited
Reject with 429 Too Many Requests and say when to come back. Common headers:
HTTP/1.1 429 Too Many Requests
Retry-After: 20
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 20Include the remaining budget on successful responses too, so good clients can pace themselves. Clients should back off with jitter rather than retry in a tight loop.
Try it
Distributed API Rate Limiter HLD
Design the rate limiter in front of a public API.