A system design interview guide to building the real-time comment and presence layer for a live stream, where a single comment must reach hundreds of thousands of concurrent viewers within about a second, and the system must track who is online — all while staying up when the comment volume becomes overwhelming.
Student edition · start with the guarantee
Broadcast a bounded live comment stream while keeping durable acceptance distinct from the subset shown to viewers.
Correctness contract: State whether comments are durably accepted and whether broadcast is sampled. Every gateway/client queue has a hard budget and recovery policy.
Basic HTTP, databases, and programming are assumed. Specialized terms are explained below. Named products motivate the problem; this is an educational design, not a claim about their current internals. Capacity numbers are illustrative decimal payload estimates, not benchmarks. Add indexes, protocol overhead, replicas, headroom, and failure capacity before provisioning.
Live comments on a popular stream is one of the most demanding real-time problems in the interview canon because the fan-out ratio is brutal: one person types a comment, and that single write has to become hundreds of thousands of reads, delivered almost instantly, to viewers scattered across the world. On top of that sits presence — the running count and roster of who is currently watching — which sounds trivial until you realize that maintaining it exactly for a million viewers is its own distributed-systems headache. And both of these have to degrade gracefully rather than collapse when a stream goes viral and the comment rate spikes by orders of magnitude. This guide builds up a design around persistent connections, a pub/sub broadcast backbone, TTL-based presence, and deliberate load-shedding.
Each definition includes a concrete example and the condition under which the mechanism is useful. You can read this guide without opening another guide.
Fanout on write and read
Fanout distributes one event to many recipients. On-write fanout precomputes recipient inbox entries; on-read fanout merges source events when a recipient asks. A hybrid chooses using measured write amplification and read cost.
Example. One post to 200 followers creates up to 200 inbox inserts. A celebrity's million followers can instead read the author's shared stream when they open their feed.
Limit. Fanout is not atomic across inboxes. Persist the post and delivery work via an outbox, dedup references, catch up after misses, and recheck privacy/deletion at serving time.
Pub/sub and durable queues
Publish/subscribe routes an event to interested subscribers; it may be ephemeral or durable. A queue/log with retained records and offsets allows replay after a consumer is disconnected.
Example. An online gateway misses a comment while restarting. Ephemeral Redis Pub/Sub cannot replay it; a durable comment log can fetch after the gateway's last sequence.
Limit. Pub/sub alone does not promise persistence, global order, or exactly-once effects. Broker choice, acknowledgements, retry horizon, and client catch-up protocol define those guarantees.
Sequence numbers and replay cursors
A sequence number records committed order within a stream. A cursor is a consumer's durable position in that stream, so it can resume without skipping missing events. An opaque identity is not automatically such a position.
Example. A device receives message sequence 10 then 12. It acknowledges contiguous 10, buffers 12, and fetches 11 before advancing to 12.
Limit. A Snowflake timestamp can arrive late behind an already-acknowledged cursor. Use per-stream commit order or a durable inbox offset, define retention gaps, and use separate device cursors.
Presence and failure detection
Presence is an estimate of recently observed connection activity. Heartbeats refresh session expiry, and a timeout suspects that a session is no longer live. A user may have several sessions/devices.
Example. A phone disconnects while its laptop keeps sending heartbeats. The user remains online if the product defines online as any active session.
Limit. Timeouts cannot distinguish a dead process from a paused or partitioned one. Counts should say sessions, users, or recently active viewers; privacy and stale snapshots need explicit policy.
Backpressure and load shedding
Backpressure tells an upstream producer that the downstream stage cannot accept more work. Buffering delays that signal; shedding deliberately rejects or drops lower-priority work when a bound is reached.
Example. A sink accepts 100 events/s while input is 150/s. A 1000-event queue fills in 20 seconds. The system must then block, reject, or apply its documented drop policy.
Limit. A finite queue cannot absorb an infinite outage. Prioritized messages still need hard quotas, deadlines, and bounded client queues; otherwise an exemption can exhaust the system.
Token bucket
A counter of available tokens that refills at rate r up to capacity B. An admitted request spends its cost. The bucket permits an initial burst but bounds admissions over duration T by approximately B+rT.
Example. B=10, r=1 token/s admits 10 calls immediately, then about one per second. It can admit 70 calls over 60 seconds, so this is not a hard 60-calls-per-minute rule.
Limit. A distributed hard limit needs atomic, durable shared accounting or safely partitioned budgets. Clamp clock rollback, validate costs, and state whether rejected attempts consume tokens.
1. Requirements & Clarifying Questions
Clarifying questions to ask
How many viewers on the biggest stream? How fast do people comment? Must every viewer see every comment, or is a sample OK? Do we show a live viewer count? Do late joiners need history? Is there moderation? Assume: up to 500K viewers per stream, ~20 comments/s normally, sampling allowed when viral, approximate count, last 50 comments for late joiners.
Functional
Non-functional
Post a comment on a live stream
Comment visible to others within ~1 s
See other viewers' comments live
Same order for everyone on a stream
See a "watching now" count
Degrade gracefully (sample) under viral load
See recent comments on join
No duplicate comments on screen
2. Back-of-the-Envelope Estimation
Quantity
Assumption
Result
Viewers on one big stream
given
500K
Comment rate (normal)
given
20/s
Deliveries per second
500K × 20
10M/s
Gateways needed
500K ÷ 10K sockets each
50
Per-gateway push rate
10K sockets × 20/s
200K msg/s
Per-gateway bandwidth
200K × 200 B = 40 MB/s
~320 Mbps
Viral, no shedding
2,000/s × 500K
1B/s
Viral, capped at 30/s
30 × 500K
15M/s
Presence writes
500K ÷ 10 s heartbeat
50K/s
Takeaway
Inbound is tiny (20/s); outbound is 10M/s
fan-out dominates
Intuition
Think of a stadium announcer. One person speaks (one write), but 50 loudspeakers each repeat it to their own section (the gateways). Nobody runs to each seat. That is the two-stage fan-out.
3. API & Data Model
# WebSocket frames
client -> gw { type:"join", stream_id }
client -> gw { type:"comment", stream_id, client_msg_id, text }
gw -> client { type:"comments", items:[{seq, user, text}, ...] } # backfill or live
gw -> client { type:"count", watching: 1204331 }
client -> gw { type:"ping" } # heartbeat, every 10 s# HTTPS
GET /v1/streams/{id}/comments?before_seq=<n>&limit=50 # scroll back
Store
Key
Value
comments (append log, e.g. Cassandra or a Redis stream for the tail)
The defining characteristic of live comments is the fan-out ratio. On a stream with 500,000 concurrent viewers, every single comment is one inbound message that must turn into 500,000 outbound deliveries, and the latency budget is roughly one second end to end — slower than that and the chat stops feeling "live" and starts feeling like a delayed transcript. Multiply by even a modest comment rate and the outbound message volume dwarfs the inbound by five or six orders of magnitude.
This asymmetry is what makes the usual request/response web model the wrong tool. You cannot have half a million clients polling for new comments every second; the read amplification would melt any backend, and a poll interval long enough to be affordable would be too slow to feel live. The system has to be push-based: the server holds an open channel to every viewer and pushes new comments down it the instant they arrive. Everything else in the design — persistent connections, a broadcast backbone, presence, load-shedding — exists to make that push fast, cheap, and survivable at scale.
Dimension
Live comments
Why it is hard
Write rate
Modest — humans type.
Not the bottleneck.
Read fan-out
One write → hundreds of thousands of deliveries.
Massive amplification per message.
Latency budget
~1 second, end to end.
No time for polling or batching.
Concurrency
Hundreds of thousands of open connections per stream.
Connection state, not request throughput, is the limit.
A useful reframing for the interview: live comments is not a "messaging" problem, it is a broadcast problem. The single hardest number is the fan-out — turning one write into N near-simultaneous reads — and every component is chosen to make that fan-out cheap and bounded.
5. Persistent Connections
Because the system must push, each viewer holds a persistent connection to the server rather than making repeated requests. The standard choice is a WebSocket — a full-duplex channel that stays open for the life of the viewing session — with long-polling as a fallback for clients or networks that cannot hold a WebSocket open.
These connections terminate on a fleet of gateway nodes whose entire job is to own connection state. A gateway node accepts thousands of viewer connections, knows which streams each connection is subscribed to, and forwards a comment down every relevant socket the moment it receives one. Separating this gateway tier from the application logic is deliberate and important:
Connections are the scaling unit. The binding constraint is not CPU or request throughput but the number of simultaneous open sockets and the memory they consume. Isolating that concern in a dedicated, horizontally scalable tier lets you add capacity for connections without touching the comment service.
The application stays stateless about connections. The comment service does not need to know which physical socket a viewer is on; it just publishes a comment for a stream, and the gateways figure out delivery. This keeps the business logic simple and the gateways dumb but fast.
Graceful handling of churn. Viewers join and leave constantly. A gateway tier built around connection lifecycle handles connect, disconnect, and reconnect cleanly, which is exactly what presence also depends on.
6. End-to-End Architecture
The architecture is a layered pipeline: viewers hold WebSocket connections to gateway nodes; gateway nodes forward new comments to a comment service, which orders, dedups, and persists each one, then publishes it once to a pub/sub backbone; and every gateway subscribes to that backbone to receive broadcasts.
Viewers hold persistent WebSocket connections to gateway nodes. A gateway forwards a new comment to the comment service (1), which dedups it, assigns its sequence number, and writes it to the comment store so late joiners can backfill (2). The comment service then publishes it once to the pub/sub backbone (3), and every gateway subscribed to the stream receives it (4) and pushes it to its own viewers.
Following a comment through the system:
Viewers. Each runs a client holding one WebSocket to a gateway. The client sends comments up that socket and renders comments pushed down it.
Gateway nodes. The connection tier. When a viewer posts a comment, their gateway sends it to the comment service (a direct RPC or an ingest queue), which accepts it and then publishes it to the backbone. When any comment for a stream appears on the backbone, every gateway holding subscribers to that stream pushes it down their sockets.
Pub/Sub backbone. The broadcast bus. A comment published once is delivered to every gateway subscribed to that stream's channel. This is the mechanism that lets a comment reach viewers no matter which gateway they happen to be connected to.
Comment service. The brain. It assigns ordering, dedups, applies any load-shedding policy, and decides what gets broadcast and persisted.
Comment store. Durable storage of recent comments, so a viewer who joins mid-stream can backfill the last N comments rather than seeing an empty chat.
7. Broadcasting via Pub/Sub
The crux of the fan-out is this: a comment can be typed by a viewer connected to any gateway, and it must reach viewers connected to every gateway. With dozens or hundreds of gateway nodes, you cannot have each gateway talk to every other gateway directly — that is an N-squared mesh that becomes unmanageable. Instead, a pub/sub backbone decouples publishers from subscribers: a gateway publishes a new comment to the stream's channel exactly once, and the backbone fans it out to all gateways subscribed to that channel.
One comment is published once to the pub/sub backbone, which broadcasts it to every gateway subscribed to the stream. Each gateway then delivers it to all of its locally connected viewers. The two-stage fan-out — backbone to gateways, gateway to sockets — is what keeps a single publish from becoming an N-squared problem.
The win is a two-stage fan-out. The first stage (backbone → gateways) is small: there are only as many gateways as you run, perhaps a few hundred. The second stage (gateway → sockets) is large but local: each gateway pushes to its own few thousand connected viewers, in memory, with no cross-network coordination. The expensive fan-out happens at the edge, in parallel, on hardware sized exactly for it.
# a viewer posts a comment: the gateway forwards it, it does NOT publish itself
function on_comment(stream_id, comment):
comment_service.accept(stream_id, comment) # dedup + seq + store, then publish ONCE (see Ordering and Dedup)# every gateway runs this for streams its viewers watch
function on_broadcast(stream_id, comment):
for sock in local_subscribers(stream_id): # local, in-memory
sock.push(comment) # the big fan-out, at the edge
The pub/sub backbone is what turns "every gateway must reach every viewer" into "publish once, and each gateway fans out locally." It is the structural reason adding viewers (more sockets per gateway, or more gateways) scales linearly instead of quadratically.
8. Presence with Heartbeats
Presence answers "who is watching right now?" — both the live count and, sometimes, the roster. The challenge is that there is no clean "I left" signal you can rely on: viewers close laptops, lose signal, and drop off without a graceful disconnect. So presence cannot be a simple add-on-join, remove-on-leave set. The standard solution is heartbeats with a TTL.
Each client sends a heartbeat every few seconds; the gateway forwards it and the presence store records the viewer with a short TTL. If a heartbeat stops arriving, the key expires, the viewer drops out of the online set, and the count drops — no explicit "leave" message required.
The mechanism is simple and robust: each client sends a heartbeat (say every 10 seconds); on each heartbeat the presence store records the viewer's id under the stream with a TTL slightly longer than the interval (say 30 seconds). As long as heartbeats keep coming, the key keeps getting refreshed and the viewer counts as online. The moment heartbeats stop — for any reason — the key expires on its own and the viewer silently drops out. Crashes, network drops, and clean exits all converge to the same correct outcome without a reliable disconnect signal.
function on_heartbeat(stream_id, user_id):
presence.set(key=(stream_id, user_id), ttl=30s) # refresh on each beat
function online_count(stream_id):
return presence.count(stream_id) # keys not yet expired
At scale, exact presence becomes expensive. Maintaining and querying a precise set of a million live keys, updated every few seconds, is a heavy write and memory load. The usual trade is to relax precision where it does not matter:
Approximate counts. Showing "1.2M watching" does not require an exact integer. The cheapest exact-enough method: every gateway already knows how many of its sockets watch each stream, so each reports that number every few seconds and a counter service sums them. For "unique viewers" across devices, a HyperLogLog sketch works, but only per time window (for example one sketch per 30 s, then swap), because a HyperLogLog cannot remove a viewer who left.
Sampled or sharded heartbeats. Aggregate heartbeats per gateway and report partial counts upward, rather than tracking every individual in one global set.
Exact only when needed. Reserve a precise roster for small contexts (a private room, the host's view of named participants) where the cost is bounded and the exactness is a real feature.
9. Ordering and Dedup
Assign canonical sequence per stream through a committed authority/log partition. Gateways/clients preserve order or buffer/recover gaps. IDs deduplicate identity; independent IDs do not themselves impose shared order. Consistent hashing is placement, not an ordering/failover protocol.
Ephemeral pub/sub can lose broadcasts during disconnect. If catch-up is promised, retain the accepted log and resume after a contiguous cursor. With display sampling, distinguish accepted history from the displayed subset and communicate intentional gaps.
10. Load Shedding at Extreme Volume
Define admission separately from display. A durably accepted comment may remain in history even when not selected for live broadcast. Authors should not be told that every viewer saw it. Apply per-author limits and a hard per-stream budget, then sample/prioritize within that budget.
Priority classes need reserved bounded allowances, not unconditional bypass. Bound every gateway/client queue; coalesce/drop sampled comments, send gap markers, or disconnect and resync slow clients. Preserve moderation/removal state so replay cannot resurrect deleted comments. Sampling is a capacity tradeoff, not proof of unbiased/complete conversation coverage.
11. Persistence for Late Joiners
A viewer who joins a stream that is already in progress should not see an empty chat. To make that work, the comment service writes accepted comments to a comment store, and a newly connected client receives the most recent N comments along with the live push stream.
The access pattern is narrow and predictable, which makes the store easy to design: it is almost entirely "give me the last N comments for this stream, in order," keyed by stream and sequence number. That favors an append-optimized store with a tail-read path, and it pairs naturally with the per-stream sequence numbers from the ordering layer — if the client subscribes first and buffers live comments while it reads the backfill, it can join the two by seq with no gap and no duplication. Older comments can roll off to cheaper storage or be trimmed entirely, since the live experience only ever needs the recent tail.
function on_join(stream_id, client):
buffer = []
gateway.subscribe(client, stream_id, into=buffer) # 1. attach FIRST, hold live msgs
recent = store.tail(stream_id, n=50) # 2. backfill recent history
client.send(recent)
last = recent[-1].seq if recent else 0
client.send([c for c in buffer if c.seq > last]) # 3. flush only newer, no dupes
gateway.switch_to_direct(client) # 4. live push from now on
Why the order matters
If you read the backfill first and subscribe second, any comment accepted in the gap between those two steps is in neither: it was too new for the backfill and too early for the subscription. Subscribing first closes the gap; the seq > last filter removes the overlap.
Subscribing before reading history makes the two ranges overlap. An overlap is easy to trim; a gap cannot be repaired.
12. Failure Modes
What fails
Effect
Handling
A gateway crashes
~10K viewers disconnect.
Clients reconnect with jitter to another gateway, send their last seen seq, and get the missing tail from the store.
Pub/sub node fails
Some gateways stop receiving a stream.
Replicated brokers; gateways detect a gap in seq and fetch the missing range from the comment store.
Sequencer for a stream dies
New comments on that stream stall.
Another node takes the stream and resumes from the last stored seq + 1 (read from the store before accepting).
Slow client (bad network)
Its send buffer grows and eats gateway memory.
Bound the per-socket queue; drop the oldest comments or disconnect. One slow phone must not hurt 10K others.
Viral spike
Deliveries jump 100×.
Load shedding before sequencing; the count and comments still flow, just sampled.
Gateways subscribe only to streams they have viewers for
Subscribe per stream
Viewer count
Exact set of viewers
Sum of per-gateway counts
Sum of counts; exact only for small rooms
Viral volume
Deliver all comments
Sample and prioritize
Sample above a threshold
Ordering
Per-stream sequence (one sequencer)
No global order, client sorts by time
Per-stream sequence
14. Summary
Live comments and presence is fundamentally a broadcast problem: turning one write into a near-instant fan-out to a huge audience, while tracking who is online and surviving extreme volume. The design is a set of reinforcing decisions:
Concern
Mechanism
What makes this hard?
The fan-out: one comment must reach hundreds of thousands of viewers within ~1 second.
How do we push to viewers?
Persistent WebSocket (or long-poll) connections terminated on a dedicated gateway tier.
How does a comment reach every gateway?
A pub/sub backbone: publish once, broadcast to all subscribed gateways.
How is the big fan-out kept cheap?
Two-stage fan-out — small backbone-to-gateway step, large but local gateway-to-socket step.
Who is online?
Heartbeats with a TTL; presence keys self-expire. Approximate counts at scale.
How do we keep comments coherent?
Per-stream sequence numbers for order, unique ids for dedup.
What about a viral firehose?
Load-shedding: sampling, per-author and per-stream rate caps, prioritizing high-signal comments.
What do late joiners see?
A comment store holding the recent tail; subscribe first, backfill, then merge by seq.
The recurring theme: push instead of poll, broadcast instead of point-to-point, and decouple the connection tier from the application so the expensive fan-out happens locally at the edge. Presence rides on the same connection lifecycle via self-expiring heartbeats, and load-shedding ensures the whole thing bends instead of breaking when a stream goes viral.
Worked example: half a million connected viewers
Assume 500,000 viewers, 20 admitted comments/s, and 200 payload bytes per delivery.
Read top to bottom. The table below explains the state and guarantee at each step.
Step
State, calculation, and reasoning
1. Compute outbound work
500,000×20=10 million deliveries/s, or 2 GB/s payload before transport/TLS overhead.
2. Shard connected viewers
50 gateways with 10,000 sockets each perform 200,000 deliveries/s and 40 MB/s payload each. Benchmark CPU and queue memory, not only socket count.
3. Bound a viral burst
At 2000 comments/s, unsampled fanout is a billion deliveries/s. Cap the common broadcast at 30/s; priority classes still need bounded reserved quotas.
4. Recover a disconnected viewer
Use sequence replay if history is promised. If intentionally ephemeral/sampled, communicate gaps rather than claiming every accepted comment was displayed.
Check your understanding: Can a moderator priority bypass break a hard broadcast cap?
Yes. Priority traffic needs its own hard rate/buffer budget, or it can exhaust the system the cap protects.
Run a small, testable model
Download concepts.py. It uses only the Python 3.10+ standard library. From the directory where you saved it, run:
python3 concepts.py delivery
python3 concepts.py test
The gap-repair and duplicate-suppression model explains durable live-stream reconnection.
The program contains deterministic assertions and failure traces. It is a learning model, not a production service: it does not implement network consensus, distributed durability, authentication, or third-party delivery. Read its contracts and try changing an input before extending it.
The backbone only sends one copy per gateway (about 50), and each gateway does the big fan-out to its own sockets in memory. No single component sends 500K messages over the network.
A late joiner reads the last 50 comments, then subscribes. What can go wrong?
Answer
A comment accepted between the read and the subscribe is missed. Subscribe first, buffer, backfill, then send buffered comments with seq greater than the last backfilled one.
Why can't a single HyperLogLog show "currently watching"?
Answer
HyperLogLog supports add and count, not remove. Viewers who leave stay counted. Sum per-gateway socket counts, or use a fresh sketch per time window.
Why does the dedup set need a TTL?
Answer
Retries happen within seconds. Without a TTL the set holds every comment id forever and grows without bound.
A stream hits 2,000 comments a second. What do you do and why?
Answer
Shed load: always deliver host and moderator comments, sample the rest down to about 30/s, and let authors still see their own. That cuts deliveries from 1B/s to about 15M/s.
18. Key Terms & Further Reading
Fan-out
Fanout distributes one event to many recipients. On-write fanout precomputes recipient inbox entries; on-read fanout merges source events when a recipient asks. A hybrid chooses using measured write amplification and read cost. Example and limits above.
Gateway
A server whose job is holding client connections.
Pub/sub
Publish/subscribe routes an event to interested subscribers; it may be ephemeral or durable. A queue/log with retained records and offsets allows replay after a consumer is disconnected. Example and limits above.
Sequence number
A sequence number records committed order within a stream. A cursor is a consumer's durable position in that stream, so it can resume without skipping missing events. An opaque identity is not automatically such a position. Example and limits above.
Backfill
Sending recent history to a client that just joined.
Load shedding
Backpressure tells an upstream producer that the downstream stage cannot accept more work. Buffering delays that signal; shedding deliberately rejects or drops lower-priority work when a bound is reached. Example and limits above.
HyperLogLog
A tiny sketch that estimates how many distinct items were added.
Backpressure
Backpressure tells an upstream producer that the downstream stage cannot accept more work. Buffering delays that signal; shedding deliberately rejects or drops lower-priority work when a bound is reached. Example and limits above.
LinkedIn Engineering blog, "Now You See Me, Now You Don't: LinkedIn's Real-Time Presence Platform" (2018), and their talk on scaling live likes/comments with Akka and Play (QCon 2020, "Streaming a Million Likes/Second").
IETF RFC 6455, "The WebSocket Protocol".
Flajolet et al., "HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm" (2007).
Martin Kleppmann, Designing Data-Intensive Applications, Chapter 11: messaging systems and stream processing.
