A field guide to the recurring building blocks of system design — the handful of problems that show up in almost every large system, and the standard ways to solve each one. Each pattern is framed as a problem, the main approaches, and when to reach for which.
Student edition · start with the guarantee
Choose reusable design mechanisms from requirements and explain their correctness costs before choosing products.
Correctness contract: Every pattern states its scope, invariant, failure boundary, and recovery behavior; combining patterns does not automatically compose their guarantees.
Basic HTTP, databases, and programming are assumed. Specialized terms are explained below. Named products motivate the problem; this is an educational design, not a claim about their current internals. Capacity numbers are illustrative decimal payload estimates, not benchmarks. Add indexes, protocol overhead, replicas, headroom, and failure capacity before provisioning.
Most system-design problems are combinations of a small number of recurring sub-problems. You rarely invent a new way to push live updates or scale reads — you recognize the shape of the problem and apply a known pattern, then reason about its trade-offs for your specific constraints. This page collects the most common of those patterns. Knowing them turns a blank-page design question into a matter of selection: name the sub-problem, pick the fitting pattern, and justify the choice.
Each definition includes a concrete example and the condition under which the mechanism is useful. You can read this guide without opening another guide.
Cache-aside and invalidation
A read checks a cache, reads the authoritative store on a miss, then fills the cache. A write commits to the store and invalidates or refreshes the cached value. TTL means time to live, not a freshness proof.
Example. A reader fetches version 1, a writer commits version 2 and invalidates, then the delayed reader caches version 1. DB-first invalidation alone does not prevent this refill race.
Limit. Use invalidation-aware fill leases or version fences for stronger behavior; a TTL only bounds one entry's residence under additional assumptions. Authorization and revocation need an explicit stale-data policy.
Optimistic concurrency and compare-and-set
Update state only if the expected version or status still matches, in the same atomic operation that changes it. A failed comparison means a conflicting operation won; reread or return a conflict.
Example. UPDATE seats SET status='held', version=version+1 WHERE id=? AND status='available' allows only one concurrent claimant to update the row.
Limit. Application-level GET then unconditional SET is not equivalent. Include acquisition identity to reject stale callbacks, and couple related invariants in a transaction.
Fencing tokens
A strictly increasing grant number attached to operations. The protected resource atomically compares it with its durable highest accepted token and rejects older operations, making superseded clients unable to overwrite newer effects.
Example. A has token 33 and pauses. B acquires 34 and writes. When A sends token 33 later, the resource rejects it because its fence is already 34.
Limit. A new grant must be installed at the resource before it promises exclusion there. Token 33 can still pass until 34 is observed. Equal-token operations also need sequence/idempotency rules if they can arrive out of order.
Transactional outbox
A business transaction writes its state change and an event row to the same database. A relay publishes committed event rows later, retrying until the broker durably accepts them.
Example. An order and its OrderCreated outbox row commit together. A relay crash after publish but before marking sent publishes the same event ID again on restart.
Limit. The relay is at-least-once, not magically exactly-once. Consumers dedup transactionally; preserve per-entity sequence when order matters. CDC is change data capture: reading committed database changes.
Sagas and compensation
A durable workflow of local transactions with corrective actions for failures. Compensation is a new business operation, not a rollback of history. The workflow exposes intermediate states and must recover after crashes.
Example. Reserve a seat, authorize payment, confirm booking, then capture. A declined authorization releases the seat; an uncertain charge stays pending until reconciled.
Limit. Timeout does not mean the remote effect failed. Compensation can fail too and needs retries/escalation. A saga does not provide isolation or guarantee every external effect is reversible.
Geohash
A string encoding a rectangular latitude/longitude cell by interleaving longitude and latitude subdivision bits and representing them in base32. Longer shared prefixes identify a smaller shared cell.
Example. Two cafes 20 metres apart across a cell boundary may have unrelated prefixes. A radius search must cover all cells intersecting the query region, then filter candidates by distance.
Limit. Eight neighbors suffice only under specific cell-size/radius assumptions. Longitude wraps at the antimeridian and widths shrink toward the poles. Haversine is a spherical approximation, not ellipsoidal exactness.
Backpressure and load shedding
Backpressure tells an upstream producer that the downstream stage cannot accept more work. Buffering delays that signal; shedding deliberately rejects or drops lower-priority work when a bound is reached.
Example. A sink accepts 100 events/s while input is 150/s. A 1000-event queue fills in 20 seconds. The system must then block, reject, or apply its documented drop policy.
Limit. A finite queue cannot absorb an infinite outage. Prioritized messages still need hard quotas, deadlines, and bounded client queues; otherwise an exemption can exhaust the system.
Idempotency keys
A client-generated identity for one logical operation. The server binds it to caller scope, endpoint, request fingerprint, state, and result. A retry uses the same key and reuses the original outcome rather than repeating the effect.
Example. Payment p42 times out after the provider charged it. Retrying with p42 lets the provider return the same charge instead of creating a new one.
Limit. A unique local claim is insufficient if the business effect and saved result can diverge. Keep unknown outcomes pending, use downstream idempotency, and state the retention/retry horizon.
1. How to Use These Patterns
In an interview, patterns appear after you have the requirements and a first design. Go through the design and ask, for each box and arrow, “which of the problems below does this face?” Then name the pattern, give its cost, and tie it to a requirement.
Many patterns come down to simple math. Here is how much a cache helps a service with 20,000 reads per second:
Quantity
Working
Result
No cache
20,000 × (1 − 0)
20,000 DB reads/s
90% hit rate
20,000 × 0.10
2,000 DB reads/s
95% hit rate
20,000 × 0.05
1,000 DB reads/s
99% hit rate
20,000 × 0.01
200 DB reads/s
Think in miss rate
Going from 90% to 99% hits sounds like a 10% gain. But database load falls 10×, from 2,000 to 200 reads per second. The miss rate is what the database feels.
2. Pushing Real-Time Updates
Problem: the server has new information (a chat message, a price tick, a notification) and the client needs to see it promptly, without the user refreshing. HTTP is request-response, so the challenge is getting data to a client that did not just ask for it.
Three approaches on a spectrum from simplest to most capable: short polling, server-sent events, and WebSockets.
Approach
How it works
Use when
Short polling
Client requests on a fixed timer; server answers with new data or nothing.
Updates are infrequent and a few seconds of lag is fine. Simplest to build.
Long polling
Client request is held open until there is data (or a timeout), then immediately re-issued.
You want near-instant delivery but must stay on plain HTTP / simple infra.
Server-sent events (SSE)
One long-lived HTTP stream the server pushes events down. One-way only. The browser's EventSource reconnects on its own and sends Last-Event-ID so the server can resume.
Server-to-client streams: feeds, notifications, live dashboards.
WebSockets
A persistent, full-duplex connection (RFC 6455) that starts as an HTTP request and is then “upgraded”; both sides can send at any time over the same TCP connection.
At scale, the connection and the fan-out are separate concerns. A stateful gateway holds the many open connections, while a pub/sub layer (e.g. Redis, Kafka) decouples the publisher from the gateways so an event can reach every interested connection across the fleet.
3. Managing Long-Running Tasks
Problem: some work takes too long to finish inside a request — video transcoding, report generation, sending a million emails. Doing it synchronously ties up a server, risks timeouts, and gives the user a spinner for minutes.
Accept the work, enqueue it, return immediately, and process it asynchronously on a worker pool while the client tracks status.
The pattern is accept-and-defer: the API validates the request, writes a job record, puts it on a queue, and returns 202 Accepted with a job id. A pool of workers pulls from the queue and processes jobs in the background, writing progress and results to a status store. The client learns the outcome by polling the job status or being notified (webhook / push) when it completes.
Decoupling & durability. The queue buffers bursts and survives crashes — a worker dying just means the job is retried, not lost.
Elastic throughput. Scale workers independently of the API tier; the queue depth tells you when to add more.
Problem: multiple clients try to read-modify-write the same resource at the same time — the last seat on a flight, a counter, an account balance. Without coordination they overwrite each other and you get lost updates or oversells.
Pessimistic locking serializes access up front; optimistic concurrency lets everyone proceed and detects conflicts at write time.
Strategy
How it works
Use when
Pessimistic locking
Acquire a lock (row lock, SELECT ... FOR UPDATE) before reading; others wait.
Conflicts are frequent and retrying is expensive; short critical sections.
Optimistic concurrency (OCC)
Read a version, do the work, write only if the version is unchanged (compare-and-set). Loser retries.
Conflicts are rare; you want maximum concurrency and no held locks.
Atomic operations
Push the whole change into one atomic primitive (INCR, conditional update) the datastore serializes for you.
The update is a simple, expressible mutation (counters, sets).
Distributed lock
A lock held in an external store (Redis, ZooKeeper) coordinates across processes/machines.
The resource spans services and no single database can arbitrate.
Distributed locks are deceptively hard: you need a lease/TTL so a crashed holder cannot block forever, and a fencing token so a slow holder cannot act after its lease expired. When you can, prefer a single atomic operation or OCC over a lock you have to manage.
5. Scaling Reads
Caches, replicas, CDNs, and materialized views optimize different reads. First specify allowed staleness and authorization/revocation policy. Cache-aside invalidation has refill races; TTL is not a linearizability proof. Fresh reads can sometimes be served from coordinated replicas/versioned caches, so 'strictly current means replicas/caches cannot help' is too broad.
Read replicas still apply all writes and may be stale; use current-authority/apply barriers or explicit session dependencies when stronger reads are needed. Bound cache-miss fan-in, hot keys, and fallback traffic. Shard precomputed views only after their update/dedup/rebuild contract is clear.
6. Scaling Writes
Problem: a single node can't absorb the write throughput — too many inserts per second, or a working set too large for one machine. Writes are harder to scale than reads because you can't just add read-only copies.
The core move is partitioning: route each write to a shard by a key, so total write capacity grows with the number of shards.
Technique
How it works
Use when
Partitioning / sharding
Split data across shards by a key (hash or range); each shard takes a fraction of writes.
The dominant lever. Needs a shard key that spreads load evenly.
Batching & buffering
Accept writes into a queue/log and apply them in batches downstream.
Smooths spikes and turns many small writes into fewer large ones.
LSM-tree storage
Engines (Cassandra, RocksDB) that turn random writes into sequential appends.
Write-heavy workloads where sequential I/O is far cheaper.
Sharded / async counters
Split a hot counter into many sub-counters, or aggregate asynchronously.
A single hot row (likes, views) that would otherwise serialize all writes.
The whole game is the shard key. A poor key creates hot shards — one partition taking most of the traffic (the "celebrity" problem) — which puts you right back to a single-node bottleneck. Choose a key with high cardinality and even access.
7. Handling Large Blobs
Problem: users upload and download large binary files — images, video, documents. Streaming gigabytes through your application servers and storing them in your primary database wrecks both.
Keep the bytes out of your app: the client uploads directly to object storage with a presigned URL; the database holds only metadata.
The pattern separates bytes from metadata:
Object storage for the bytes. Files live in a blob store (S3, GCS) built for cheap, durable, massive objects — never in the relational database.
Direct, presigned transfer. The app issues a short-lived presigned URL and the client uploads (and downloads) straight to/from object storage, so the file never passes through your servers.
Metadata in the database. Your database stores only a small row: id, storage key, size, owner, content type — the things you query on.
CDN for downloads. Serve reads through a CDN so popular files are cached near users.
For very large uploads use multipart upload: the file is split into chunks uploaded in parallel and reassembled, which also makes failed uploads resumable instead of restarting from zero.
8. Multi-Step Processes
A saga is a durable sequence of local transactions plus business compensations. It has no automatic cross-service isolation. A refund is a new payment, not erasure of the charge; shipping/email may be irreversible. Compensation can fail and needs retry/reconciliation/escalation.
Persist workflow state and each step's stable identity; call downstream idempotently. Timeout is an uncertain outcome, not proof the step failed. Query/retry that same identity before compensating or moving on. Local state and emitted events share a transactional outbox/CDC boundary; the relay and consumers handle duplicate delivery.
Orchestration puts the flow in a durable coordinator; choreography distributes it across versioned event handlers. Both require explicit ordering, dedup, deadlines, pending states, and observability. 2PC provides atomic commit for participating resources and can block after prepare; it does not itself supply isolation and its coordinator can be replicated/recovered. Choose a service boundary where critical invariants fit atomically whenever practical.
9. Proximity-Based Services
A spatial index conservatively narrows two-dimensional search. Geohash encodes rectangles; quadtrees adaptively split regions; S2/H3/PostGIS provide reviewed globe/geometry tools. Shared geohash prefixes imply a shared cell, but near points can straddle boundaries.
Cover every cell intersecting the query region, not universally just eight neighbors. Refine with a documented spherical/geodesic model; driving ETA requires roads. Handle cell size, large radii, antimeridian/poles, density skew, and stale updates. A B-tree on a derived cell key can support the candidate lookup; it does not turn geographic exactness into a trivial one-prefix match.
10. Rate Limiting & Throttling
Problem: you must cap how often a client can call you — to protect against abuse and runaway loops, to enforce API quotas, and to keep one noisy tenant from starving everyone else. It pairs with nearly every other pattern here.
The token bucket: requests spend tokens that refill at a steady rate; an empty bucket means reject.
Algorithm
How it works
Character
Token bucket
Tokens refill at rate r up to a capacity; each request spends one.
Smooth average rate, allows bursts up to capacity. The common default.
Leaky bucket
Requests queue and drain at a fixed rate.
Enforces a strictly steady output rate; smooths bursts out.
Fixed window
Count requests per calendar window (e.g. per minute).
Simple, but allows 2× bursts straddling a window boundary.
Sliding window
A rolling count over the trailing interval.
Accurate, avoids the boundary spike; a bit more state to track.
Reject with 429 Too Many Requests and a Retry-After header so well-behaved clients back off. In a distributed fleet the limiter state must be shared (e.g. counters in Redis) or each node enforces only its slice of the limit.
11. Idempotency & Safe Retries
Scope an idempotency key by principal/operation and bind its payload. For a local effect, claim, apply, and store the result in one transaction. For a remote effect, use provider-supported stable identity and preserve unknown outcomes for reconciliation. A unique local insert alone does not make a remote call exactly once.
Retain financial natural identities longer than a short response-cache TTL when needed. The downloaded SQLite model is a complete transactional local example with rollback and concurrent replay tests; it does not call a real card processor. Treat a lease as recovery scheduling, not proof a paused worker stopped.
12. Worked Example: Combining Patterns
Design a concert ticket site. 50,000 seats go on sale at 10:00 and a million fans show up. Watch how the patterns stack:
Five patterns in one flow. Each box solves one sub-problem.
Rate limiting plus a virtual waiting room keeps a million clients from hitting the seat database at once.
Contention:UPDATE seats SET held_by = ?, hold_until = now() + interval '10 minutes' WHERE id = ? AND held_by IS NULL is one atomic conditional write, so two fans cannot hold one seat.
Saga and idempotency: charge the card with an idempotency key; if it fails, release the hold as the compensation.
Long-running task: the PDF ticket and email are made by a worker from a queue.
Scaling reads and real-time: the seat map is served from cache and updated by SSE, since it is read far more than written.
13. Common Mistakes
Naming a pattern without its cost. “Add a cache” is half an answer. Say what goes stale, for how long, and why that is fine.
Reaching for a distributed lock first. An atomic conditional write or optimistic concurrency is simpler and safer.
Check-then-act on idempotency keys. Claim the key atomically, or concurrent retries both do the work.
Using WebSockets for one-way updates. SSE or long polling is simpler and works through more proxies.
Forgetting the saga's in-between states. The UI must show “payment pending”, since atomicity across services is gone.
Worked example: one order with an uncertain charge
An order reserves inventory, then calls an external payment provider.
Read top to bottom. The table below explains the state and guarantee at each step.
Step
State, calculation, and reasoning
1. Reserve and record intent
The inventory state and next outbox event commit locally. Retries use the order/step identity.
2. Charge with a stable provider key
The provider may complete while its response is lost. Mark unknown/reconciling, not failed-with-no-effect.
3. Recover the workflow
Retry/query the same operation. A compensation is its own new action and may need retries/manual escalation.
4. Serve derived state
Caches/search/feed references can lag. They cannot grant new inventory or bypass current authorization merely because they contain a prior value.
Check your understanding: Do retries plus a queue guarantee correct effects?
No. Each effect needs idempotency/transactional state, bounded retention, ordering where needed, and a recoverable uncertain-outcome policy.
Run a small, testable model
Download concepts.py. It uses only the Python 3.10+ standard library. From the directory where you saved it, run:
python3 concepts.py workflow
python3 concepts.py test
The workflow model demonstrates an ambiguous remote outcome and idempotent recovery.
The program contains deterministic assertions and failure traces. It is a learning model, not a production service: it does not implement network consensus, distributed durability, authentication, or third-party delivery. Read its contracts and try changing an input before extending it.
Idempotency keys + a result store; design for naturally idempotent operations.
The meta-pattern: almost every one of these trades a little consistency or freshness for a lot of scale or availability. In an interview, the value isn't naming the pattern — it's stating that trade-off out loud and tying it to the requirements you were given.
15. Check Yourself
A live sports score page only needs server-to-client updates. Which real-time approach fits best, and why not WebSockets?
Answer
SSE. It is one-way, runs over plain HTTP, and reconnects on its own. WebSockets add two-way machinery you do not need.
Your cache hit rate drops from 99% to 95% on 20,000 reads/s. What happens to database load?
Answer
Misses go from 200/s to 1,000/s, so the database sees 5× the load.
Two buyers try to take the last item. Give a single SQL statement that prevents overselling without a lock service.
Answer
UPDATE items SET stock = stock - 1 WHERE id = ? AND stock > 0. Check the affected row count: 0 means sold out.
Why must a saga's compensating step be a new action rather than a rollback?
Answer
Each earlier step already committed in its own database. You can only issue a new corrective action, like a refund.
What problem does the transactional outbox solve?
Answer
A service could commit its change and crash before publishing the event. Writing the event to an outbox table in the same transaction makes them one unit; a relay publishes later.
Why can two points 1 meter apart have different geohash prefixes, and how do queries cope?
Answer
They sit on opposite sides of a cell edge. Queries search the cell and its 8 neighbors, then filter by exact distance.
16. Key Terms
SSE
Server-sent events: a one-way HTTP push stream.
WebSocket
A two-way connection upgraded from HTTP.
Optimistic concurrency
Update state only if the expected version or status still matches, in the same atomic operation that changes it. A failed comparison means a conflicting operation won; reread or return a conflict. Example and limits above.
Fencing token
A strictly increasing grant number attached to operations. The protected resource atomically compares it with its durable highest accepted token and rejects older operations, making superseded clients unable to overwrite newer effects. Example and limits above.
Cache-aside
A read checks a cache, reads the authoritative store on a miss, then fills the cache. A write commits to the store and invalidates or refreshes the cached value. TTL means time to live, not a freshness proof. Example and limits above.
Thundering herd
Many clients missing the same key at once.
Presigned URL
A short-lived link that grants direct upload or download.
Saga
A durable workflow of local transactions with corrective actions for failures. Compensation is a new business operation, not a rollback of history. The workflow exposes intermediate states and must recover after crashes. Example and limits above.
Transactional outbox
A business transaction writes its state change and an event row to the same database. A relay publishes committed event rows later, retrying until the broker durably accepts them. Example and limits above.
Idempotency key
A client-generated identity for one logical operation. The server binds it to caller scope, endpoint, request fingerprint, state, and result. A retry uses the same key and reuses the original outcome rather than repeating the effect. Example and limits above.
Geohash
A string encoding a rectangular latitude/longitude cell by interleaving longitude and latitude subdivision bits and representing them in base32. Longer shared prefixes identify a smaller shared cell. Example and limits above.
17. Further Reading
Martin Kleppmann, Designing Data-Intensive Applications (O'Reilly, 2017): chapters 5–9 cover replication, partitioning, transactions and consistency.
Hector Garcia-Molina and Kenneth Salem, “Sagas”, SIGMOD 1987.
Chris Richardson, Microservices Patterns (Manning, 2018), and microservices.io pages on the saga and transactional outbox patterns.
Martin Kleppmann, “How to do distributed locking” (blog post, 2016), on leases and fencing tokens.
Brandur Leach, “Implementing Stripe-like Idempotency Keys in Postgres” (2017), and Stripe's API docs on idempotent requests.
RFC 6455, The WebSocket Protocol; the WHATWG HTML Standard section on server-sent events.
