Question 14 · Track 5 · Protect it and charge for it
Design a rate limiter
The algorithm is the easy half. Where it runs is the answer.
Protect it and charge for it. 4 clarifying questions to ask first, the answer in 4 moves, the follow-up that catches a memorised answer and 2 supporting topics to watch or read.
What it is really testing
Whether you know that a per-instance counter is not a rate limit, and what you return when you reject.
Ask these first
Before a single box goes on the board. The answers change the design, which is the point of asking out loud.
- Limit per user, per API key, or per IP?
- One limit, or different limits per endpoint and per tier?
- If the limiter itself is down, do we fail open or closed?
- Is a burst allowed, or is the limit strict?
The answer, in 4 moves
In this order. Each move earns the next one — say them out loud rather than drawing all four and narrating afterwards.
- 1Pick the window: fixed, sliding, or a token bucket if bursts are allowed.
- 2Keep the counter in ONE shared store (Redis), keyed by identity plus window.
- 3Enforce it at the gateway, not inside every service.
- 4Reject with 429 and a Retry-After — a limit the client cannot see is an outage.
The trap
The follow-up that separates a rehearsed answer from a real one.
Run the counter in each instance and you have not built one rate limit, you have built N of them: ten replicas at "100 per minute" let a thousand through. The shared counter is the design.
Watch or read
The pieces of this answer, each covered on its own. Take them whichever way suits you, then give the whole answer without looking.
Debouncing vs throttling
Debounce waits for silence and fires once; throttle watches the clock and fires steadily. The search box makes debounce look universal — a drag handler at 60 events/sec proves it isn't. Both mechanisms in plain JavaScript, both honest failure modes (debounce can starve forever, a naive throttle drops the final event), and a rule for picking one.
API gateway
An API gateway puts a single front door in front of all your services — one entrance every request has to pass through, including the bad ones. Here's the problem it solves, how it terminates TLS, authenticates, rate-limits, routes, and aggregates in one place, and why the same idea now guards your LLM calls too.
Part of System Design Interviews, Answered Out Loud. The fundamentals underneath it are the free CS course.