Rate limiter
Token bucket or sliding window? Where it runs, how nodes share counts and what happens when the limiter itself is down.
What is on this board
Parts
- Client Web app
- Gateway API gateway
- Limiter Rate limiter
- Counters Cache · Redis
- Rules Config file
- API Service
- Rejected requests Metrics
- Which algorithm Data table
- Token bucket Decision record
- Redis is down? Decision
- Let requests through Option
- Reject everything Option
How they connect
- Client → Gateway: request
- Gateway → Limiter: may it pass?
- Limiter → Counters: take a token
- Limiter → Rules: limit for this plan
- Gateway → API: yes
- Limiter → Rejected requests: 429
- Redis is down? → Let requests through: chosen
Notes and decisions
- Limiter: Runs in the gateway, one check per request
- Counters: Atomic with a Lua script
- Rules: Per plan, per endpoint
- Token bucket: Simple, cheap, and bursts are fine for an API.
- Let requests through: The product keeps working. Chosen
- Reject everything: Safe, but the limiter became the outage
More like this
News feed, like TwitterFan-out on write or on read? Timelines in a cache, the celebrity problem, and how a post reaches millions of followers.
News feed, like TwitterFan-out on write or on read? Timelines in a cache, the celebrity problem, and how a post reaches millions of followers.
URL shortener, like BitlyThe first system design interview question: key generation, heavy reads, a cache in front, and click analytics off the request path.
