Learnings

System design, part 1 of 12

System Design Foundations

The map of the whole path, a repeatable 45-minute interview structure, the estimation arithmetic and the core trade-offs that every later page relies on. Written for an experienced backend engineer: definitions are precise, numbers are order of magnitude, and each chapter leads with the decision.

The map

Part 1 of 5 · what is asked and where it lives on this site

System design interviews come in two shapes: high-level design (HLD) of a distributed system, and low-level design (LLD) of the classes inside one service. Everything else on this path is the toolkit those two rounds draw on.

HLD, LLD and the supporting areas

The map

HLD answers "which components, how does data flow, how does it scale and fail". LLD answers "which classes and interfaces, which patterns, how is it made thread-safe and extensible". A senior candidate is expected to move between both levels in the same conversation.

Round typeWhat is askedWhat is evaluatedWhere on this site
HLD (system design)"Design a news feed / chat / rate limiter" for N usersRequirement scoping, numbers, component choice, data model, scaling, failure handling, trade-offsHLD Case Studies: Core, HLD Case Studies: Advanced
LLD / machine coding"Design a parking lot / splitwise / LRU cache" as classes, often with running codeObject modelling, interfaces, patterns, SOLID, concurrency, testability, extensibilityLow-Level Design, LLD Case Studies
Building-block probes"How does a load balancer / cache / queue behave when ..."Knowing each component's guarantees, limits and failure modesBuilding Blocks
Storage deep diveChoice of database, indexing, replication, partitioning, transactionsMatching the store to access patterns; consistency reasoningData Systems
Distributed-systems deep diveConsensus, idempotency, ordering, clocks, distributed transactionsCorrectness under partial failureDistributed Systems Patterns
API designREST / gRPC / events, pagination, versioning, real-time channelsClean contracts, backward compatibility, idempotent writesAPI and Communication Design
Architecture discussionMonolith vs microservices, event-driven, CQRS, migration pathsJudgement: fitting structure to team and load, not fashionArchitecture Styles
Operations and reliabilitySLOs, observability, deploys, overload, disaster recoveryOperational maturity: you have run systems in productionReliability and Operations
The loop itselfHow to run the session, question bank, study planDriving, prioritising, communicatingSenior Interview Playbook

Senior signal: a mid-level answer lists components. A senior answer states the requirement that forces each component, the number that sizes it and what happens when it fails.

How the 12 pages fit

The map

Read top to bottom. The toolkit pages can be read in any order after this page; the case studies assume all of them.

Pages 2 to 7 are the toolkit for the HLD case studies; the LLD track on the right is independent; both end at the playbook.

The interview framework

Part 2 of 5 · a repeatable 45-minute structure

The framework exists so that no minute is spent deciding what to do next. Requirements drive numbers, numbers drive the API and data model, and those drive the diagram.

The eight steps and the time budget

The interview framework

The only loop is between deep dives and the diagram: a deep dive often changes a box, and that is expected.
MinutesStepBudgetOutput on the board
0 to 51. Clarify requirements53 to 5 functional, 3 to 5 non-functional, out-of-scope list
5 to 92. Estimate4QPS (read, write, peak), storage, the one number that shapes the design
9 to 133. API44 to 6 endpoints or RPCs with key parameters
13 to 174. Data model4Entities, keys, main access patterns, store choice
17 to 275. High-level diagram10End-to-end path for each core use case
27 to 396. Deep dives121 to 2 areas taken to implementation depth
39 to 437. Bottlenecks, failures, trade-offs4Single points of failure, hot spots, what was traded away
43 to 458. Wrap-up2Summary against requirements, what you would do next

Interview tip: a working end-to-end design by minute 27 matters more than a perfect step 2. If the interviewer says "skip the numbers", skip them and state your assumed scale in one sentence.

Functional vs non-functional requirements

The interview framework

Functional: what the system does ("a user can post", "followers see the post"). Non-functional: how well it does it (scale, latency, availability). Functional requirements pick the boxes; non-functional requirements pick how many, where and with which guarantees.

Three lists come out of the first five minutes; the out-of-scope list protects the rest of the session.
Non-functionalQuestion to askWhat the answer changes
ScaleDAU, read:write ratio, data size, growth, peak vs average?Single node vs partitioned; cache and replica count
Latencyp99 target for the hot path? Is it user-facing or batch?Sync vs async, caching, precomputation, region placement
AvailabilityTarget nines? Is degraded service acceptable?Redundancy level, multi-AZ or multi-region, fallbacks
ConsistencyCan a read be stale? For how long? Which operations cannot be?Leader reads vs replica reads, quorum, transactions
DurabilityIs losing an acknowledged write ever acceptable?Sync replication, fsync policy, backups, queue persistence
SecurityWho may call it? Any PII, payments, multi-tenant isolation?AuthN/AuthZ placement, encryption, audit trail
CostIs there a budget or cost-per-request constraint?Storage tiering, compute choice, how much is precomputed
ComplianceData residency, retention, right to delete?Region pinning, deletion pipelines, immutable logs

Interview tip: do not ask all eight. Ask scale, latency and consistency, then propose the rest: "I'll assume 99.9% availability and no acknowledged write is lost; tell me if that is wrong".

What to produce and say at each step

The interview framework

StepProduceSayCommon mistake
1. ClarifyThree short lists: functional, non-functional, out of scope"The core use cases are A, B, C. I'm leaving out D and E unless you want them."Designing before scoping; 10 minutes of questions with no decisions
2. EstimateRead and write QPS, peak, storage per year, one design-shaping number"100:1 read-heavy, 12k reads/s at peak, so caching and replicas matter more than write scaling."Arithmetic with no conclusion; false precision
3. APIEndpoints with inputs, outputs, auth, idempotency key on writes, pagination on lists"These map one-to-one to the functional requirements."CRUD for every entity instead of the use cases; no pagination
4. Data modelEntities, primary keys, indexes for each access pattern, SQL vs NoSQL with a reason"The hot query is by user_id ordered by time, so that is the partition and sort key."Picking the database before listing queries
5. High-levelClient, edge, services, stores, async paths; trace each use case through it"Let me walk a write, then a read, through the diagram."Boxes with no data flow; adding Kafka with no stated need
6. Deep divesDetail on the 1 to 2 hardest parts: schema, algorithm, protocol, failure handling"The risky part is the fan-out. Shall I go deep there, or do you prefer the storage layer?"Shallow coverage of everything; waiting to be asked
7. BottlenecksSPOFs, hot keys, thundering herds, what breaks at 10x, monitoring"If this cache cluster is lost, the DB sees 20x load; I'd protect it with ..."Claiming the design has no weaknesses
8. Wrap-up30-second recap mapped to requirements; next steps"We meet the latency goal via X; the open risk is Y; next I would add Z."Trailing off; introducing new components at minute 44

API and data model sketch

The interview framework

The level of detail to aim for on the board, here for a short-post service. Enough to show keys, idempotency and pagination; no more.

Each table exists because of one access pattern; FEED_ITEM is a precomputed read model, not a source of truth.

Estimation

Part 3 of 5 · back-of-envelope arithmetic and the numbers to memorise

Estimation is not about the answer; it is about finding which resource is the constraint. Round hard, keep one significant figure, and end every estimate with a design consequence.

The method

Estimation

Everything derives from DAU times actions per user; peak QPS sizes compute, storage per day sizes the data tier.
QuantityFormulaShortcut
Average QPSDAU x actions per user per day / 86,400Divide by 10^5: 1M per day is about 12/s, call it 10
Peak QPSAverage x peak factor2x to 3x typical; 10x or more for flash sales and live events
Storage per dayWrites per day x bytes per record1M x 1 KB = 1 GB
5-year storagePer day x 365 x 5 x replication factorx 2,000 per replica; x 3 replicas is about x 5,500
BandwidthQPS x response size1,000/s x 1 MB = 1 GB/s = 8 Gbps
Cache memory20% of the hot working set x object size (80/20 rule)Plus about 30% to 50% overhead for keys and metadata
ServersPeak QPS / QPS per server, plus headroomAdd 30% and N+1 or N+2 for failures and deploys

80/20 rule: a rule of thumb that about 20% of the objects receive about 80% of the reads. Real workloads are often more skewed (Zipf-like), which makes caching work even better and hot keys worse.

Powers of two and time

Estimation

PowerApproximate valueUnitFeel for it
2^101 thousand (10^3)1 KBA short JSON record or a row
2^201 million (10^6)1 MBA compressed photo is 0.2 to 2 MB
2^301 billion (10^9)1 GB1M records of 1 KB
2^401 trillion (10^12)1 TB1B records of 1 KB; one large disk
2^5010^151 PBObject-storage territory
2^32about 4.3 billion IPv4 space; 32-bit id space
2^64about 1.8 x 10^19 64-bit id space; never runs out
Time factValueUse
Seconds in a day86,400, round to 10^5Per-day to per-second
Seconds in a monthabout 2.6 million (2.5 x 10^6)Monthly quotas
Seconds in a yearabout 31.5 million (pi x 10^7)Downtime budgets
1 request/sabout 86k per day, 2.6M per monthSanity checks
Days in 5 yearsabout 1,800, round to 2,000Storage horizon
ObjectTypical size (approximate)
UUID / 64-bit id / timestamp16 bytes / 8 bytes / 8 bytes
Tweet-sized text with metadataabout 1 KB
Web page HTMLabout 100 KB
Photo (compressed)about 200 KB to 2 MB
1 minute of video (720p to 1080p, compressed)about 10 to 50 MB

Worked example

Estimation

A short-post service: 100M DAU, each user writes 2 posts and loads the feed 20 times a day. A post is 1 KB; 10% carry a 500 KB image; a feed page returns 10 posts. In the interview you would round as in the right-hand column of the table below; the code shows the exact arithmetic.

QuantityInterview roundingDesign consequence
Write QPS200M / 10^5 = 2k/sOne leader per shard can take it; writes are not the hard part
Read QPS2B / 10^5 = 20k/s, peak 60k/s10:1 read-heavy: precompute feeds, cache, read replicas
Text storage200 GB/day, about 1 PB in 5 years with 3 replicasFar beyond one node: partition by user or post id from the start
Media storage10 TB/day, about 20 PB in 5 yearsObject storage plus CDN; never in the database
Egress (text)20k/s x 10 KB = 200 MB/s, about 2 GbpsFine; images dominate bandwidth, so serve them from the CDN
Cache20% of 200 GB = 40 GB, 60 GB with overheadA small replicated cache cluster, not a fleet
App servers60k / 1k per server = 60, about 80 to 90 with headroomStateless tier behind a load balancer, autoscaled

Senior signal: the last column. Every number ends in "so we need ..." or "so this is not a concern". The 1,000 QPS per server figure is an assumption; say so and move on.

Latency numbers

Estimation

Order of magnitude only; they vary by hardware and year. What matters is the ratios: memory is about 1,000x faster than SSD, a network hop inside a datacentre costs about as much as thousands of memory reads, and crossing an ocean costs hundreds of datacentre hops.

OperationTime (order of magnitude)Relative
L1 cache referenceabout 1 ns1x
Branch mispredict / L2 cache referenceabout 3 to 10 ns10x
Mutex lock and unlock (uncontended)about 20 ns20x
Main memory (RAM) referenceabout 100 ns100x
Compress 1 KB with a fast codecabout 2 to 10 us10^4 x
SSD random readabout 100 us10^5 x
Read 1 MB sequentially from memoryabout 10 to 100 us10^5 x
Round trip within the same datacentreabout 0.5 ms5 x 10^5 x
Read 1 MB sequentially from SSDabout 1 ms10^6 x
Spinning-disk seekabout 10 ms10^7 x
Read 1 MB sequentially from spinning diskabout 10 to 20 ms10^7 x
Round trip between continentsabout 100 to 150 ms10^8 x
  • A cache hit over the network (about 0.5 to 1 ms) is dominated by the network, not by the cache.
  • Sequential I/O beats random I/O by orders of magnitude on disk, which is why logs (WAL, Kafka, LSM trees) append.
  • A cross-region synchronous call adds about 100 ms per round trip; keep them off the hot path or pay for it knowingly.
  • Human-perceived "instant" is about 100 ms; a typical API p99 budget is 100 to 500 ms.

Throughput of one node

Estimation

Approximate, for sizing only. Real figures depend heavily on hardware, payload size, query shape and durability settings; state the assumption and offer to load-test.

System (one node)Approximate throughputLimited by
Application server, typical business logicabout 1k requests/s (hundreds to a few thousand)CPU per request, downstream latency, runtime
Reverse proxy / L7 load balancertens of thousands of requests/sTLS handshakes, connections, CPU
Relational DB, indexed point readsabout 10k queries/sBuffer-pool hit rate, connections
Relational DB, transactional writesabout 1k to 5k writes/sWAL fsync, index maintenance, locks
In-memory key-value store (Redis-like)about 100k operations/sSingle-threaded command loop, network
Log broker (Kafka-like)hundreds of thousands of small messages/s, on the order of 100 MB/sDisk and network throughput, replication
LSM wide-column node (Cassandra-like)about 10k writes/sCompaction, disk
Search node (Lucene-based)hundreds to low thousands of queries/sQuery complexity, shard count
  • Connections: one server can hold on the order of 100k to 1M mostly idle connections (WebSocket, long poll) if memory per connection is small; active throughput is the tighter limit.
  • Disk: a comfortable single-node dataset is up to a few TB; past that, rebuild and backup times force partitioning before capacity does.

Availability nines

Estimation

AvailabilityDowntime per yearPer 30-day monthTypically needs
99% (two nines)3.65 days7.2 hoursOne server and a restart script
99.9% (three nines)8.76 hours43 minutesRedundant instances, automated failover
99.95%4.38 hours22 minutesMulti-AZ, tested failover
99.99% (four nines)52.6 minutes4.3 minutesNo manual step in recovery; safe deploys; multi-AZ at least
99.999% (five nines)5.26 minutes26 secondsMulti-region active-active; very few systems truly need it

Interview tip: at four nines a human cannot be in the recovery loop: paging and logging in already takes longer than the monthly budget.

Core trade-offs

Part 4 of 5 · the vocabulary of every design decision

Each chapter is one axis you will be asked to choose a position on. The detail (replication protocols, quorums, consensus) lives in Data Systems and Distributed Systems Patterns.

Scalability: vertical vs horizontal

Core trade-offs

Scale up (bigger machine) until it stops being cheap or safe; scale out (more machines) when you need capacity beyond one box or redundancy. Scaling out is easy for stateless tiers and hard for state.

Horizontal scaling works because the app instances hold no state of their own; the state moved to a shared store, which is now the thing to scale.
 Vertical (scale up)Horizontal (scale out)
GainNo code change, no distributed-systems problems, strong consistency stays easyNear-linear capacity, redundancy, incremental cost
PayHard ceiling, single point of failure, downtime to resize, price grows faster than capacity at the top endLoad balancing, partitioning, coordination, partial failure, operational complexity
Use whenDatabases early on; anything under one node's limitsStateless services always; data tier once one node is not enough
  • Stateless service: any instance can serve any request because session and data live in a shared store or in the request (token). This is what makes load balancing, autoscaling and rolling deploys trivial.
  • Scalability is not performance: a system is scalable if adding resources adds capacity roughly in proportion.

Senior signal: "one large Postgres with a replica handles this for the first two years; I'd shard only when write volume or dataset size forces it" is a stronger answer than sharding on day one.

Latency, throughput and percentiles

Core trade-offs

Latency is time per request; throughput is requests per unit time. Batching and queueing raise throughput and raise latency. Near saturation, latency grows non-linearly, so plan to run at 50% to 70% utilisation.

MetricMeaningUse
p50 (median)Half of requests are fasterTypical experience; hides problems
p951 in 20 is slowerCapacity planning
p991 in 100 is slowerSLOs; heavy users hit it constantly
p99.91 in 1,000 is slowerLarge fan-out systems
AverageSum / countAvoid: one 10 s outlier hides in a 50 ms mean
Fan-out amplification: the request is as slow as its slowest backend, so backend p99 becomes the caller's common case.
  • If each backend exceeds its p99 on 1% of calls, a request touching 100 backends is slow with probability 1 - 0.99^100 = 63%. With 10 backends it is about 10%.
  • Tail-latency tools: timeouts with budgets, hedged requests (send a second copy after the p95 delay), partial results, fewer hops, avoiding shared queues.
  • Sources of tail latency: GC pauses, lock contention, cold caches, noisy neighbours, retries, queue build-up.

Interview tip: always quote a percentile: "p99 under 200 ms", never "fast" or "200 ms average".

Availability arithmetic

Core trade-offs

Components in series (all must work) multiply availabilities and make things worse. Components in parallel (any one is enough) multiply failure probabilities and make things better, assuming failures are independent.

Three stages in series; the middle stage is two instances in parallel. The weakest serial stage (the database) sets the ceiling.
StepArithmeticResult
Series ruleA = A1 x A2 x ... x AnThree 99.9% parts in series: 99.7%
Parallel ruleA = 1 - (1 - A1) x (1 - A2) x ...Two 99% parts in parallel: 99.99%
App tier above1 - 0.001 x 0.00199.9999%
Whole path0.9999 x 0.999999 x 0.9995about 99.94%, roughly 5 hours down per year
Add a DB standby with failoverDB becomes 1 - 0.0005^2, in theoryPath is now limited by the LB: about 99.99%
  • Redundancy only helps if failures are independent: same rack, same AZ, same bad deploy or same poisoned request breaks the assumption.
  • Failover is not instant or perfect; detection time and failed failovers count as downtime.
  • Every hard dependency you add in series lowers your ceiling. Make dependencies soft (cache, default, degrade) where you can.

Failure mode: correlated failure. Most large outages are a config push, a bad deploy or an overload cascade that hits all replicas at once, not independent hardware faults.

Reliability vs availability vs durability

Core trade-offs

PropertyQuestion it answersMeasured byBought with
AvailabilityDoes it answer right now?Successful requests / total; ninesRedundancy, failover, load shedding
ReliabilityDoes it do the correct thing over time, even under faults?Error rate, MTBF, correctness of resultsTesting, idempotency, safe deploys, fault isolation
DurabilityOnce acknowledged, is the data ever lost?Probability of loss per year (for example "eleven nines" for object stores)Replication, fsync, checksums, backups, restore drills
  • Available but not reliable: answers every request, sometimes with wrong or stale data.
  • Durable but not available: data is safe on disk, the service is down.
  • Related terms: fault tolerance (keeps working when a component fails), resilience (recovers quickly), MTTR (mean time to recover). Availability is about MTBF / (MTBF + MTTR), so cutting recovery time is usually cheaper than preventing every failure.

Consistency models as a ladder

Core trade-offs

Stronger models are easier to program against and cost more latency and availability. Pick the weakest model that keeps the product correct, per operation, not per system.

Top is strongest and most expensive. The two session guarantees sit side by side: each is weaker than causal, and neither implies the other.
ModelGuarantee in one lineWhen it matters
LinearizableBehaves like a single copy; every operation takes effect at one point in real time, and later reads see itLocks, leader election, unique constraints, balances, inventory decrement
SequentialAll clients see the same single order of operations, but that order may lag real timeReplicated logs, state machines fed from an ordered log
CausalOperations that are causally related are seen in order by everyone; concurrent ones may differComment threads (reply never appears before its parent), collaborative apps
Read-your-writesA client always sees its own earlier writesProfile edit then reload; post then view own timeline
Monotonic readsA client never sees time go backwards across successive readsPagination and refresh across replicas with different lag
EventualIf writes stop, all replicas converge; no bound on when, no ordering promise meanwhileLike counts, view counts, DNS, caches, search indexes
  • "Strong consistency" in interviews nearly always means linearizable reads and writes on one object. Serializability is a different axis: isolation of multi-object transactions (see Data Systems).
  • Cheap ways to get session guarantees on an eventually consistent store: read from the leader for a short window after a write, sticky routing to one replica, or pass a version token and wait for the replica to catch up.

CAP and PACELC

Core trade-offs

CAP, stated precisely: when a network partition separates replicas, a system must choose between consistency (C, meaning linearizability) and availability (A, meaning every non-failed node answers). It is not "pick two of three": partitions are not optional, so the real choice is C or A during a partition. PACELC adds the everyday case: else (E), with no partition, you still trade latency (L) against consistency (C).

PACELC as two independent choices: PA or PC under partition, EL or EC otherwise.
ClassBehaviourCommonly cited examples (depends on configuration)Fits
PA / ELAlways writable, low latency, reconcile laterDynamo-style stores such as Cassandra with low consistency levelsCarts, feeds, telemetry, session data
PC / ECConsistent always; pays latency, loses availability on the minority sideConsensus-based stores: Spanner-style databases, etcd, ZooKeeperMoney, inventory, metadata, coordination
PC / ELFast when healthy, refuses rather than diverges when partitionedLess common; some single-leader setups with async replicas and strict failoverSystems that prefer downtime to split brain
PA / ECConsistent when healthy, stays up by relaxing consistency under partitionSome configurable storesRarely a deliberate target
  • Many stores are tunable per request (consistency level, quorum size), so classify the configuration, not the product.
  • CAP's "available" is stricter than SLA availability, and its "consistent" is unrelated to the C in ACID.

Interview tip: do not say "this is a CP system" and stop. Say which operation needs which: "placing an order is CP; browsing the catalogue is AP and may be seconds stale".

ACID vs BASE, strong vs eventual

Core trade-offs

 ACIDBASE
Stands forAtomicity, Consistency (invariants hold), Isolation, DurabilityBasically Available, Soft state, Eventually consistent
PromiseA transaction is all-or-nothing and invariants are never visibly brokenThe system stays up and converges; the application handles intermediate states
CostCoordination: locks or MVCC, sync commit; hard across shards or regionsConflict resolution, compensation, idempotency, harder reasoning
Typical homeRelational and distributed SQL databasesDynamo-style key-value and wide-column stores, caches, search indexes
Eventual consistency in practice: B briefly reads a stale count. Harmless for likes, unacceptable for a balance check before a withdrawal.
DataNeedsWhy
Bank balance, wallet, ledgerStrong (linearizable, transactional)A stale read allows a double spend; invariants must hold
Inventory, seat bookingStrong on the decrement / reservationOverselling is a real-world cost
Unique username, idempotency keyStrongTwo winners is a correctness bug
Like count, view count, follower countEventualOff by a few for a few seconds is invisible
Feed, search results, recommendationsEventualFreshness in seconds is fine; availability and latency matter more
Own profile after editingRead-your-writesOthers may see it late; the author must not

Read-heavy vs write-heavy

Core trade-offs

The read:write ratio is the first number that shapes a design. Read-heavy systems move work to write time (precompute, cache, replicate). Write-heavy systems move work off the write path (append, buffer, batch, partition).

Read-heavy designs add copies in front of the database; write-heavy designs add a buffer and partitions behind the app.
 Read-heavy (for example 100:1)Write-heavy (for example 1:10)
ExamplesFeeds, product pages, URL redirects, profilesMetrics, logs, IoT, click streams, chat messages, location pings
Main toolsCache, CDN, read replicas, denormalised read models, indexesAppend-only log, queue buffering, batching, sharding, LSM storage
Storage biasB-tree, many indexesLSM-tree, few indexes
You payStaleness, invalidation complexity, slower writesRead-time work (merge, aggregate), eventual visibility
Watch forCache stampede, replica lag, hot keysBackpressure, hot partitions, compaction stalls

Push vs pull, sync vs async, stateful vs stateless

Core trade-offs

AxisOptionGainPayUse when
DeliveryPush (fan-out on write, server-sent)Fast reads, fresh dataWrite amplification, wasted work for inactive receiversFew receivers per event, low latency required
Pull (fan-out on read, polling)Cheap writes, receiver controls rateSlow or heavy reads, staleness up to the poll intervalHuge fan-out (celebrity), infrequent readers
Coupling in timeSynchronous (request-response)Simple, immediate result and errorCaller blocked; availabilities multiply; cascading failureUser needs the answer now; short call chain
Asynchronous (queue, event)Absorbs spikes, isolates failures, retries for freeEventual result, duplicates, ordering, harder debuggingSlow or unreliable work; spikes; multiple consumers
Server stateStatelessAny instance serves any request; trivial scaling and failoverEvery request pays a state lookupDefault for request-serving tiers
StatefulData locality, no lookup, long-lived connectionsSticky routing, rebalancing, careful deploys, replication of stateWebSocket gateways, databases, stream processors, game servers
Keep on the synchronous path only what the user must wait for; everything else goes behind the queue.

Trade-off: hybrids are normal. Feeds push to ordinary users and pull for celebrity accounts; APIs accept synchronously (202 plus a status URL) and process asynchronously.

Single-leader vs multi-leader vs leaderless

Core trade-offs

Where writes are accepted decides whether conflicts can exist. Details (replication lag, quorums, conflict resolution) are in Data Systems and Distributed Systems Patterns.

One writer, several writers, or every node a writer: conflicts become possible as soon as there is more than one.
 Single-leaderMulti-leaderLeaderless
Writes go toOne nodeAny leader, usually one per regionAny replica; W of N must acknowledge
Write conflictsNoneYes; need resolution (last-write-wins, CRDT, app merge)Yes; versions, read repair, anti-entropy
Consistency availableStrong from the leader; stale from async followersEventual across leadersTunable; R + W > N gives overlap, not full linearizability
Write availabilityLost until failover completesSurvives loss of a leader or regionSurvives node loss with no failover step
Write scalingOne node (shard to go further)Per region, not per keyScales with nodes
Main riskFailover: lost async writes, split brainConflict handling bugsStale reads, sloppy quorums, LWW data loss
Typical useMost OLTP: PostgreSQL, MySQL, MongoDB replica setsMulti-region writes, offline-capable clientsDynamo-style stores: Cassandra, Riak

Interview tip: default to single-leader with replicas. Move away only when you can name the requirement: writes in several regions, or write availability through node loss.

Scaling a design step by step

Part 5 of 5 · from one server to millions of users

The classic evolution. Each step is taken because a specific limit was hit, not in advance. In an interview, start at the step your numbers justify and name the trigger for the next one.

Steps and their triggers

Scaling step by step

StepChangeTriggerNew problem it brings
1Single serverStart hereOne failure domain; app and DB compete for resources
2Separate database serverCPU / memory contention; need to scale tiers independentlyNetwork hop per query
3Load balancer + stateless app tierApp CPU saturated; need zero-downtime deploys and redundancySessions must move out of the process
4Read replicasDB read load saturates the primary; need failover targetReplication lag, stale reads
5CacheSame hot rows read repeatedly; DB latency dominates p99Invalidation, stampedes, hot keys
6CDNStatic and media bytes dominate bandwidth; users are far awayPurging, cache-control correctness
7Queue + workersSlow or spiky work on the request pathDuplicates, ordering, monitoring lag
8ShardingWrite volume or dataset exceeds one primaryCross-shard queries and transactions, rebalancing, hot shards
9Multi-regionGlobal latency, region-level disaster recovery, data residencyCross-region replication, conflicts, cost

Interview tip: steps 3 to 7 are nearly free of regret and can be drawn immediately at any real scale. Steps 8 and 9 are expensive; justify them with a number.

1 and 2: one box, then a separate database

Scaling step by step

Splitting the database off lets each tier get the hardware it needs and be scaled on its own.
  • Good for: a few hundred requests per second with vertical scaling. Many products never need more than step 2 plus backups.
  • Trigger to move on: the app server is CPU-bound, or any deploy or crash is an outage.

3: load balancer and stateless app tier

Scaling step by step

Session state moves to a shared store (or a signed token), so any app instance can serve any request.
  • Gain: horizontal app capacity, rolling deploys, health-checked failover. Run the load balancer itself as a redundant pair.
  • Trigger to move on: the database is now the bottleneck, usually on reads first.

4: read replicas

Scaling step by step

Writes go to one primary; reads spread over replicas that lag by milliseconds to seconds.
  • Gain: read capacity and a promotion target if the primary fails.
  • Failure mode: read-after-write returns stale data. Route a user's reads to the primary briefly after their own write.
  • Trigger to move on: the same queries repeat, or replicas keep being added for a few hot rows.

5: cache

Scaling step by step

Cache-aside: the app checks the cache, falls back to the database on a miss and populates the cache.
  • Gain: sub-millisecond reads for hot data and an order of magnitude less database load.
  • Pay: staleness and invalidation. The database must still survive a cold cache. Patterns and failure modes are in Building Blocks.
  • Trigger to move on: bandwidth and latency for static assets and media.

6: CDN

Scaling step by step

Bytes are served from an edge near the user; only dynamic API calls reach the origin.
  • Gain: lower latency worldwide, origin offload, absorption of traffic spikes.
  • Media goes to object storage, not app-server disks; uploads use pre-signed URLs so bytes bypass the app tier.
  • Trigger to move on: slow or bursty work (email, thumbnails, fan-out) is making requests slow.

7: queue and workers

Scaling step by step

The request returns after the enqueue; workers scale on queue depth and failures park in a dead-letter queue.
  • Gain: fast responses, spike absorption, independent scaling and failure isolation of slow work.
  • Pay: at-least-once delivery means handlers must be idempotent.
  • Trigger to move on: the primary database cannot keep up with writes, or the dataset no longer fits one node comfortably.

8: sharding

Scaling step by step

Each shard is its own primary plus replicas; the shard key decides which queries stay on one shard.
  • Before sharding: bigger primary, archive cold data, split by function (move a table or service to its own database).
  • Choose the shard key from the dominant access pattern; queries that do not include it become scatter-gather.
  • Trigger to move on: users on other continents, or a requirement to survive the loss of a region.

9: multi-region

Scaling step by step

Each region is a full stack; the hard part is the dashed line between the data tiers.
ModeWritesGainPay
Active-passiveOne region; the other is a warm standbySimple; disaster recoveryFailover time; possible loss of the last async writes; far users stay slow on writes
Active-active, partitioned by user home regionEach user writes in one regionLow latency, no conflicts for most dataRouting, user migration, global data still needs a plan
Active-active, multi-leaderAny region for any dataBest write latency and availabilityConflict resolution everywhere

Cheat sheet

TopicRemember
FrameworkClarify 5, estimate 4, API 4, data model 4, high-level 10, deep dives 12, bottlenecks 4, wrap-up 2 (minutes)
RequirementsFunctional, non-functional, out of scope. Ask scale, latency, consistency; assume the rest out loud
EstimationDay = 10^5 s. 1M/day = about 10/s. 1M x 1 KB = 1 GB. Peak = 2x to 3x. 5 years = about 2,000 days
LatencyRAM 100 ns, SSD 100 us, datacentre hop 0.5 ms, disk seek 10 ms, cross-continent 100 to 150 ms
One node (approx.)App 1k rps, SQL 10k reads/s and 1k to 5k writes/s, Redis-like 100k ops/s, Kafka-like 100k+ msgs/s
Nines99.9% = 8.8 h/year, 99.99% = 53 min/year, 99.999% = 5 min/year
Availability mathsSeries multiply availabilities; parallel multiply failure probabilities; correlated failures break both
PercentilesQuote p99. Fan-out of 100 turns backend p99 into a 63% chance of a slow request
Consistency ladderLinearizable, sequential, causal, read-your-writes / monotonic reads, eventual. Weakest that is correct, per operation
CAP / PACELCDuring a partition choose C or A; otherwise choose latency or consistency
Strong vs eventualMoney, inventory, uniqueness: strong. Counts, feeds, search: eventual. Own edits: read-your-writes
Read-heavyCache, CDN, replicas, precomputed read models
Write-heavyAppend-only log, queue, batching, sharding, LSM storage
ReplicationSingle-leader by default; multi-leader for multi-region writes; leaderless for write availability
Scaling orderSeparate DB, LB + stateless apps, replicas, cache, CDN, queue, sharding, multi-region; each with a trigger

Next in the path: Building Blocks.