← CatZep/Engineering Guides/System Design/Feeds, Messaging & Social

Designing a News Feed System

A system design interview guide to building the home feed of a social application: how a post travels from one author to the screens of millions of followers, and how the right blend of push and pull keeps both writes and reads fast.

A news feed is the continuously updating list of posts a user sees when they open a social application — status updates, photos, links, and so on, drawn from the accounts they follow. It looks simple from the outside, but it hides a sharp tension. The system must serve feeds to active users with very low latency, yet a single popular post may need to reach millions of followers. The whole design comes down to deciding when the work of assembling a feed happens: eagerly at the moment of posting, lazily at the moment of reading, or some combination of the two. This guide walks through the two core operations, the fanout models that resolve that tension, the data flow that ties them together, and the caching layers that keep the common path fast.

Concepts to understand first

Each definition includes a concrete example and the condition under which the mechanism is useful. You can read this guide without opening another guide.

1. Requirements & Scope

Every design interview starts by agreeing on what you are building. Spend the first few minutes asking questions; the answers change the design.

2. Back-of-the-Envelope Estimation

Rough numbers tell you which part of the system is the bottleneck. Assume 500 million daily active users (DAU), each opening the feed 10 times a day, posting on average once every five days, with 200 followers on average. A day has 86,400 seconds; round it to about 100,000 for mental math.

3. API Design

Two endpoints cover the core. Both are authenticated; the user id comes from the auth token, never from the request body.

POST /v1/posts
  body: { "content": "text", "media_ids": ["m123"] }
  201 → { "post_id": "7310...", "created_at": "2026-09-25T10:00:00Z" }

GET /v1/feed?limit=20&cursor=<opaque>
  200 → { "posts": [ {post_id, author, content, created_at, ...}, ... ],
          "next_cursor": "<opaque>" }        # null when no more

The feed uses cursor pagination, not ?page=3. A cursor encodes "the last item I showed you" (for example the last post id). New posts arrive at the top all the time; with page numbers, every new post shifts the pages and the user sees duplicates or skips posts. A cursor says "give me what comes after this item", which stays stable.

4. Data Model

5. Two Core Operations

Before any architecture, it helps to separate the feed into the two distinct things it actually does. They have opposite performance profiles, and almost every design decision is a tradeoff between them.

The asymmetry is the key insight. Publishing is comparatively rare — a person posts a handful of times a day at most. Retrieval is constant — the same person may open and refresh their feed dozens of times a day, and there are far more readers than authors active at any instant. A naive feed reads, for every refresh, the recent posts of every account the user follows, sorts them, and returns the top slice. That is correct but expensive on the hot path. The art of feed design is moving as much work as possible off the read path without paying for work nobody will ever look at.

6. System Architecture

At a high level the system is split into a request-handling tier, a write path for storing posts, and a separate fanout path that distributes those posts into per-user feeds. A client request first resolves through DNS to a load balancer, which spreads traffic across a pool of stateless web servers. Those servers handle cross-cutting concerns — authentication to identify the caller and rate limiting to protect downstream services — before forwarding to the appropriate internal service.

From there the request takes one of two paths:

• The publish path. A new post goes to the Post Service, which persists it in the Post DB and warms the Post cache with the full content. Persisting the post is the durable source of truth; everything after it is about distribution.
• The fanout path. Once a post is stored, the Fanout Service is responsible for getting its id into the feeds of the author's followers. It reads the follower list from the Graph DB, looks up the follower records in the User cache / User DB, and pushes distribution work onto a message queue. A pool of fanout workers drains that queue and writes the post reference into each follower's News Feed cache.

A Notification service runs alongside the fanout path, so that delivering a post can also trigger a push notification or badge update where appropriate. Decoupling fanout behind a queue is what keeps the author's publish call fast: the post is acknowledged as soon as it is durably stored, and the potentially huge job of distributing it happens asynchronously.

7. Fanout Models

"Fanout" is the act of delivering one post to many follower feeds. There are two pure strategies, and the choice between them is the single most important decision in the whole design. Each optimizes one of the two core operations at the expense of the other.

Fanout on write makes reads trivial: when a follower opens the app, their feed already exists as a list in cache, so retrieval is a single lookup. The price is paid at write time. Posting once may fan out to every follower, which is fine for someone with a few hundred followers but catastrophic for someone with tens of millions — a single post becomes tens of millions of cache writes. It also does work for followers who may not log in for weeks, spending write capacity on feeds nobody reads.

Fanout on read inverts the tradeoff. Publishing is cheap because you store the post and stop; no fanout happens. The cost moves entirely to the read path, where every feed open must gather the recent posts from each followed account and merge them on the fly. This naturally handles celebrities — a popular author causes no write storm — but it makes the most frequent operation, reading, the most expensive one.

The hybrid model is the recommended approach because it lets each kind of account use the strategy that suits it. The overwhelming majority of users have modest follower counts, so pushing their posts is cheap and gives their followers instant reads. The small number of accounts with enormous follower counts are exactly the ones that would cause a write explosion, so their posts are pulled instead. At read time, a user's feed is the precomputed push feed merged with a small live pull of the handful of celebrities they follow.

8. The Publish & Fanout Flow

Tracing a single post through the system makes the fanout path concrete. Once the Post Service has durably stored a post and the Fanout Service picks it up, the numbered steps from the architecture diagram are:

1. Get friend ids from the graph DB. The Fanout Service asks the social graph for the list of accounts that follow the author. This list is the set of feeds the post must reach.
2. Get friend data from the user cache / DB. For each follower the service may need lightweight user data — for example to skip inactive accounts, respect privacy or block settings, or apply per-user ranking signals. This comes from the user cache, falling back to the user DB on a miss.
3. Push to a message queue. Rather than writing millions of feed entries inline, the service enqueues distribution work — typically batches of (post id, follower id) deliveries. The queue absorbs bursts and decouples the author's request from the heavy work.
4. Fanout workers consume. A horizontally scalable pool of workers drains the queue. Because the work is in a queue, the system can add workers to drain a backlog without touching the publish path, and a slow spell simply means the queue grows rather than the author's request stalling.
5. Write into the news feed cache. Each worker writes the post reference into the target follower's news feed cache, prepending it so the most recent posts sit at the top of the list. After this step, that follower's next feed open is a fast cache read.

# publish: fast, synchronous part
function publish(author_id, content):
  post = post_service.store(content)        # Post DB + Post cache
  fanout_service.enqueue_fanout(post)       # hand off, return now
  return ack(post.id)                       # author's call is done

# fanout: heavy, asynchronous part
function enqueue_fanout(post):
  followers = graph_db.get_followers(post.author_id)   # step 1
  for batch in chunk(followers, size=1000):
    enrich(batch, user_cache)                          # step 2
    message_queue.push(post.id, batch)                 # step 3

function fanout_worker():                               # step 4
  while True:
    post_id, batch = message_queue.pop()
    for follower_id in batch:
      news_feed_cache.prepend(follower_id, post_id)     # step 5
      news_feed_cache.trim(follower_id, MAX_FEED_LEN)   # keep newest ~800 only

9. Caching Layers

Caching is what makes the read path cheap, and the important design choice is what to store where. The two caches play different roles.

The reason the news feed cache stores only id pairs rather than whole posts is both space and consistency. Storing references means a post exists in exactly one place — the post store — even though it appears in millions of feeds. If the post is edited or deleted, the single canonical copy changes and every feed reflects it on the next read; there is no fan of stale duplicates to chase down. The cost is one extra step at read time: after reading the list of references from the news feed cache, the system hydrates them by fetching the full content for each id from the post cache (falling back to the post DB on a miss).

function get_feed(user_id, limit):
  refs = news_feed_cache.get(user_id, limit)   # ⟨post id, user id⟩ list
  if hybrid_enabled(user_id):
    refs = merge_by_time(refs, pull_celebrity_posts(user_id))[:limit]  # hybrid read
  posts = []
  for ref in refs:
    post = post_cache.get(ref.post_id) \
        or post_db.get(ref.post_id)            # hydrate full content
    if post is None or post.deleted:           # deleted since fanout: skip
      continue
    posts.append(post)
  return posts                                 # ordered, ready to render

10. The Celebrity Problem

Push/pull move costs rather than remove them. Pull still makes a popular author's stream a hot read key; cache/replicate immutable batches and coalesce refreshes. A reader may follow many celebrities, so 'a handful' is an assumption, not a bound.

Choose policy from posting rate, active followers, measured fanout cost, and latency budgets. Mode switches need overlap/backfill/dedup to prevent gaps. Partition durable fanout tasks and limit an author's share so viral activity cannot starve ordinary work.

11. Ranking & Pagination

Chronological order uses an explicit timestamp plus deterministic tie-breaker or committed sequence. Snowflake ID order is only approximate clock order. Ranked feeds generate bounded candidates, score them, and apply diversity/visibility rules; bounded candidates entail a recall tradeoff.

For stable ranked pagination, retain a short-lived ranking snapshot and cursor into its ordered set, or explicitly handle duplicate/skip behavior when ranks change. Always recheck privacy/blocks/deletions, even for an old ranking snapshot. Bind opaque cursors to user, query, expiry, and snapshot identity.

12. Failure Modes

13. Tradeoffs & Alternatives

14. Summary

A news feed design is a series of decisions about moving work between the write path and the read path:

Worked example: a cache-miss storm during fanout

Assume 20,000 reads/s and a 99% hit rate.

Run a small, testable model

Download concepts.py. It uses only the Python 3.10+ standard library. From the directory where you saved it, run:

python3 concepts.py delivery
python3 concepts.py test

The cursor model demonstrates stable ordered paging and why maximum-seen IDs can skip delayed entries.

The program contains deterministic assertions and failure traces. It is a learning model, not a production service: it does not implement network consensus, distributed durability, authentication, or third-party delivery. Read its contracts and try changing an input before extending it.

15. 45-Minute Interview Walkthrough

16. Common Mistakes

• Choosing pure push and stopping there. The first celebrity post causes a write storm. Always discuss the hybrid.
• Doing fanout inside the publish request. The author would wait for millions of writes. Acknowledge after the durable write; fan out asynchronously.
• Uncapped feed lists. A feed cache that grows forever wastes memory on posts no one scrolls to. Cap it and rebuild older pages from the Post DB.
• Offset pagination. New posts shift the offsets, so users see duplicates. Use a cursor.
• Forgetting deletes, blocks and unfollows. Precomputed feeds go stale; filter at read time.
• Skipping numbers. Without an estimate you cannot justify why reads must be cheap or why the celebrity threshold matters.

17. Check Yourself

1. A user has 50 followers and posts twice a day. Push or pull, and why?
   Answer

   Push. Each post costs only 50 cheap cache writes, and all 50 followers get instant reads. Pull would make every one of their many feed opens do extra work.

2. With 100M posts a day and 200 followers each, how many feed inserts per second does pure push need?
   Answer

   100M × 200 = 20 billion a day; 20B ÷ 86,400 ≈ 231K inserts per second on average, more at peak.

3. Why does the feed cache store post ids instead of whole posts?
   Answer

   Space (one small reference per follower instead of a full copy) and consistency (an edit or delete changes one canonical copy; readers hydrate the latest version).

4. A fanout worker crashes after writing to half the followers of a batch. What stops duplicates when the batch is retried?
   Answer

   Idempotent writes: the feed is a set keyed by post id (for example a sorted set), so inserting the same id again does nothing.

5. Why is cursor pagination better than page numbers for a feed?
   Answer

   New posts keep arriving at the top. Page numbers shift, causing duplicates and skipped posts; a cursor ("items after post X") stays stable.

6. How does a user who follows five celebrities get their posts under the hybrid model?
   Answer

   At read time the service pulls the recent posts of those five accounts and merges them by time (or score) with the user's precomputed push feed.

18. Key Terms & Further Reading

Fanout

Fanout distributes one event to many recipients. On-write fanout precomputes recipient inbox entries; on-read fanout merges source events when a recipient asks. A hybrid chooses using measured write amplification and read cost. Example and limits above.

Fanout on write (push)

Copying a post reference into every follower's feed when it is published.

Fanout on read (pull)

Building a feed at read time by fetching and merging recent posts from followed accounts.

Hydration

Turning a list of post ids into full posts by looking each one up.

Hot key / celebrity problem

One key (an account) receiving far more traffic than others and overloading its part of the system.

Cursor pagination

Paging by "items after this marker" instead of by page number.

Idempotent

Safe to apply more than once with the same result.

• Martin Kleppmann, Designing Data-Intensive Applications, Chapter 1: the Twitter home timeline example of fanout on write vs read.
• Alex Xu, System Design Interview – An Insider's Guide, Volume 1, "Design a News Feed System".
• Silberstein et al., "Feeding Frenzy: Selectively Materializing Users' Event Feeds", SIGMOD 2010: choosing push or pull per producer/consumer pair.