News feed, like Twitter
Fan-out on write or on read? Timelines in a cache, the celebrity problem, and how a post reaches millions of followers.
What is on this board
Parts
- App Mobile app
- Load balancer Load balancer
- Posts Service · Takes a new post
- Timelines Service · Reads a person's feed
- Who follows whom Graph DB
- Posts Document DB · Cassandra
- Fan-out Worker pool
- New posts Event stream · Kafka
- Timelines Cache · Redis
- Images and video Object storage · S3
- Media CDN
- Search Search index · Elasticsearch
- How does a post reach a feed? Decision
- Fan-out on write Option
- Fan-out on read Option
- Both Decision record
How they connect
- App → Load balancer: HTTPS
- Load balancer → Posts: new post
- Load balancer → Timelines: my feed
- Posts → Posts: save
- Posts → Images and video: upload
- Posts → New posts: posted
- New posts → Fan-out: consume
- New posts → Search: index
- Fan-out → Who follows whom: followers?
- Fan-out → Timelines: push to each timeline
- Timelines → Timelines: read timeline
- Timelines → Posts: hydrate posts
- App → Media: images
- Media → Images and video: on a miss
- How does a post reach a feed? → Both: chosen
Notes and decisions
- Fan-out: Writes the post into each follower's timeline
- Timelines: The last 800 post IDs per person
- Fan-out on write: Fast reads. Terrible for an account with 100M followers
- Fan-out on read: Cheap writes. Slow reads for everyone
- Both: Write for normal accounts. Merge celebrities' posts at read time.
More like this
News feed, like TwitterFan-out on write or on read? Timelines in a cache, the celebrity problem, and how a post reaches millions of followers.
URL shortener, like BitlyThe first system design interview question: key generation, heavy reads, a cache in front, and click analytics off the request path.
Ride hailing app, like UberMatching riders and drivers: live locations in a geo index, dispatch, surge, trips and payments. A classic system design interview.
