Learnings

System design, part 5 of 12

API and Communication Design

The "API" step of a design interview, and everything that travels over it: how to get from requirements to endpoints in two minutes, which style and transport to pick, the details interviewers probe (pagination, idempotency, versioning, webhooks), real-time delivery, service-to-service calls, and the security that sits on every one of them. For hands-on code see Web APIs with FastAPI. Numbers are approximate, order of magnitude only.

Designing an API in an interview

Part 1 of 6 · from requirements to endpoints

The API step is a contract, not a catalogue. Spend about two minutes, write 4 to 6 calls that cover the core use cases, and let them drive the data model and the high-level design that follow.

Requirements to endpoints in two minutes

Designing an API in an interview

Turn each functional requirement into a noun and a verb, then write one line per call. Stop when the core flows are covered.

Five passes over the same list. Each pass adds one column, so you never stare at a blank board.
  • Resources: the things users create and read (account, transfer, message). They usually become your tables too.
  • Operations: one line each, METHOD /path. Only the calls the requirements need.
  • Request / response: the 3 to 5 fields that matter. Identity comes from the token, never from the body.
  • Errors: name the two or three that are specific to the domain (insufficient funds, duplicate request).
  • Cross-cutting: say once, in one sentence: "all calls are authenticated, writes take an idempotency key, lists are cursor-paginated".

Interview tip: say which style you are using and why in one breath ("REST over HTTPS for the public API, gRPC between services"), then move on. Do not design every CRUD endpoint.

Worked example: a bank-transfer API

Designing an API in an interview

Requirements: move money between accounts, see a transfer's status, list an account's history. This is the whole API sketch you would write on the board.

  • The create call returns pending: settlement is asynchronous, so the status resource is part of the contract from the start.
  • The idempotency key is required, not optional: a retried payment is the first thing an interviewer will ask about. See Idempotency for writes.
  • available vs ledger balance signals you know holds and pending entries exist.

Naming and resource modelling

Designing an API in an interview

RuleDoAvoid
Plural nouns, verbs come from the methodPOST /transfersPOST /createTransfer
Nest only for true ownership, one level deep/accounts/{id}/transfers/users/1/accounts/2/transfers/3/entries
Actions that are not CRUD become sub-resources or new resourcesPOST /transfers/{id}/cancel, or POST /refundsPATCH status=cancelled with hidden side effects
Opaque, non-guessable ids with a type prefixtr_9xK2...Auto-increment integers (enumerable, leak volume)
One casing, one time format, one money formatsnake_case, RFC 3339 UTC, minor units + currencyMixed casing, local times, floats
The API is not the schemaModel the client's use caseExposing tables and join keys one to one
GET never changes stateSafe, cacheable readsGET /transfers/{id}/cancel

Senior signal: model the state machine as a resource (pending -> settled | failed) instead of a boolean. Async reality then fits the API without a breaking change later.

Choosing status codes

Designing an API in an interview

The status code is for machines (retry or not, cache or not); the body is for developers. The key split: 4xx means "do not retry unchanged", 5xx means "retry may help".

CodeUse whenClient should
200 OKRead or update succeeded, body returnedProceed
201 CreatedResource created; send LocationProceed
202 AcceptedWork queued, not done; return a status URLPoll or wait for a webhook
204 No ContentSuccess with no body (delete)Proceed
304 Not ModifiedConditional GET, client copy is currentUse cached copy
400 Bad RequestMalformed or invalid inputFix the request
401 UnauthorizedMissing or invalid credentialsAuthenticate, refresh token
403 ForbiddenAuthenticated but not allowedDo not retry
404 Not FoundNo such resource (also used to hide resources the caller may not see)Do not retry
409 ConflictState conflict: duplicate, concurrent edit, wrong stateRe-read, then decide
412 Precondition FailedIf-Match ETag no longer matchesRe-read, reapply, retry
422 UnprocessableWell-formed but violates a business ruleFix the request
429 Too Many RequestsRate limit or quota hit; send Retry-AfterBack off, then retry
500 Internal ErrorUnhandled server faultRetry with backoff if idempotent
502 / 504Gateway got a bad response / no response in time from upstreamRetry with backoff if idempotent
503 UnavailableOverloaded or shedding load; send Retry-AfterBack off, then retry

Failure mode: returning 200 with {"error": ...} in the body. Proxies, retries, alerting and client libraries all key off the status line, so every one of them now misbehaves.

Error format

Designing an API in an interview

One error shape for the whole API. A common standard is Problem Details (RFC 9457, media type application/problem+json); a house format is fine if it carries the same things.

  • Stable machine code (insufficient_funds) that clients branch on; the human text can change freely.
  • Request id echoed in a header and the body, the join key into logs and traces.
  • Field-level list for validation errors, so a form can show all problems at once.
  • No internals: no stack traces, SQL, hostnames, or "user exists" hints on login and reset endpoints.

What interviewers look for

Designing an API in an interview

TopicMid-level answerSenior answer
ScopeLists every CRUD endpoint4 to 6 calls tied to the requirements, then moves on
WritesPOST and hopeIdempotency key, defined behaviour on retry and timeout
Lists?page=3Cursor, max page size, stable sort key
Slow workBlocks the request202 + status resource, or webhook
Identityuser_id in the bodyFrom the token; object-level authorization on every call
Evolution"We will add v2"Additive changes, tolerant readers, deprecation policy
FailureHappy path onlyTimeouts, partial failure, what the client sees and does

Interview tip: the API is where you prove the system is usable under failure. For each write, be ready to answer "the client timed out, what now?".

API styles

Part 2 of 6 · REST, gRPC, GraphQL, streaming and the HTTP versions underneath

The style follows from who the caller is (browser, mobile app, partner, another service) and the shape of the traffic (request / response, streams, many views of one graph). Most real systems use two or three of them.

REST vs gRPC vs GraphQL vs WebSocket

API styles

RESTgRPCGraphQLWebSocket / streaming
TransportHTTP/1.1 or newerHTTP/2 (HTTP/3 emerging)HTTP, usually one POST endpointTCP after an HTTP upgrade
SchemaOptional (OpenAPI)Required (.proto), code generatedRequired (typed schema), introspectableNone built in; you define messages
PayloadJSON textProtobuf binaryJSON, shaped by the queryText or binary frames
StreamingNo (SSE or chunking as add-ons)Client, server and bidirectionalSubscriptions, over WebSocket or SSEFull duplex, native
CachingNative HTTP caching, CDN-friendlyNone at the HTTP layerHard: POST, per-query shape; needs persisted queriesNone
Browser supportNativeNeeds gRPC-Web and a proxyNative (it is HTTP + JSON)Native
Best forPublic and partner APIs, CRUD, cacheable readsService-to-service, low latency, polyglot back endsMany clients with different view needs over one graphChat, live updates, collaboration, games
You payOver- and under-fetching, chatty clientsOpaque on the wire, weaker browser and tooling reachQuery cost control, caching, N+1 resolversStateful connections to scale and route

REST constraints that matter

API styles

Nobody is asked to recite Fielding. These are the constraints that change the design.

ConstraintMeaning in practiceWhat it buys
StatelessEach request carries everything needed (token, cursor); no server-side conversation stateAny instance can serve any request: horizontal scaling, simple failover
Uniform interfaceResources named by URL, standard methods and status codesGeneric clients, proxies, gateways and caches work without custom code
CacheableResponses declare cacheability (Cache-Control, ETag)CDN and browser offload for reads
LayeredClient cannot tell whether it talks to origin, gateway or CDNYou can insert load balancers, WAF, gateways freely
Method semanticsGET safe; PUT and DELETE idempotent; POST neitherIntermediaries and clients know what is safe to retry

Trade-off: HATEOAS (links that drive the client) is rarely implemented fully. Returning a few links (next, status_url) gets most of the value.

gRPC and Protocol Buffers

API styles

An RPC framework: a .proto contract generates typed clients and servers in many languages; calls run as HTTP/2 streams with binary Protobuf payloads. Use it inside the data centre; keep REST or GraphQL at the edge.

All modes share one connection. Each call is its own stream, so a slow call does not block the others at the HTTP layer.
  • Why it suits service-to-service: compact payloads (often several times smaller than JSON), generated typed stubs, multiplexing on long-lived connections, streaming, and first-class deadlines and cancellation that propagate down the call chain.
  • Schema evolution: add fields with new numbers; unknown fields are ignored by old readers. Never change a field's type or number.
  • Failure mode: a layer 4 load balancer pins a long-lived HTTP/2 connection to one backend, so load never rebalances. Use layer 7 (per-request) balancing or client-side balancing; see Building Blocks.
  • You pay: not human-readable, no HTTP caching, browsers need gRPC-Web through a proxy.

GraphQL

API styles

The client sends a query naming exactly the fields it wants across a typed graph; one round trip replaces several REST calls. It fixes over-fetching (fat responses a mobile client throws away) and under-fetching (call 1 for the list, then N calls for details).

The N+1 problem: a per-field resolver runs once per parent row. A dataloader collects the keys within one tick, issues one batched query and caches per request.
Where it hurtsWhyMitigation
HTTP cachingOne endpoint, POST, a different shape per queryPersisted queries (hash sent with GET), client-side normalised cache, per-field cache hints
Rate limitingOne request can cost 1 or 10,000 database readsQuery cost analysis, depth and complexity limits, limit by cost not by request count
AbuseDeeply nested or aliased queriesDepth limit, timeout, allowlist of persisted queries for first-party clients
AuthorizationMany paths reach the same objectEnforce in the data layer per object and field, not per query
ObservabilityEverything is POST /graphql with 200Metrics per operation name and per resolver

When each is the right answer

API styles

Pick by caller first, then add a push channel only for the flows that need it. Styles combine: REST for commands plus WebSocket for live updates is the usual chat design.
  • REST: the default for anything public. Everyone can call it, debug it with curl, and cache it.
  • gRPC: internal calls where latency, typing and streaming matter and you control both ends.
  • GraphQL: product APIs with many client teams and fast-changing views. Avoid for simple CRUD, public partner APIs and file transfer.
  • WebSocket / SSE: only where the server must speak first. See Real-time communication.

HTTP/1.1 vs HTTP/2 vs HTTP/3

API styles

HTTP/1.1HTTP/2HTTP/3
TransportTCP (+ TLS)TCP (+ TLS in practice)QUIC over UDP, TLS 1.3 built in
FramingTextBinary framesBinary frames
MultiplexingNo: one request at a time per connection, browsers open about 6 per originMany streams on one connectionMany streams on one connection
Head-of-line blockingAt the HTTP layerGone at HTTP, still at TCP: one lost packet stalls every streamGone: loss stalls only the stream it belongs to
Header compressionNoneHPACKQPACK
Connection setupTCP + TLS: 2 to 3 round tripsSame as 1.11 round trip, 0-RTT on resumption
Network changeConnection breaksConnection breaksConnection id survives a Wi-Fi to mobile switch
Typical useLegacy, simple internal callsDefault for web and gRPCEdge to client on lossy mobile networks
TCP delivers bytes in order, so HTTP/2 inherits its head-of-line blocking. QUIC tracks loss per stream.

Interview tip: the usual deployment is HTTP/3 or HTTP/2 from client to edge, and HTTP/2 or HTTP/1.1 with keep-alive from edge to origin. 0-RTT data can be replayed, so allow it only for idempotent requests.

API details that get probed

Part 3 of 6 · the follow-up questions

After the endpoint list, interviewers pick one or two of these and dig. Each has a default answer worth knowing cold.

Pagination

API details that get probed

Default to cursor (keyset) pagination for anything large or changing. Offset is acceptable only for small, mostly static lists that need "jump to page N".

Offset: ?limit=50&offset=100000Cursor / keyset: ?limit=50&cursor=...
QueryORDER BY created_at DESC LIMIT 50 OFFSET 100000WHERE (created_at, id) < (:t, :id) ORDER BY created_at DESC, id DESC LIMIT 50
Cost of a deep pageScans and discards 100,000 rows: O(offset)Index seek: O(page size), the same at any depth
Rows inserted while pagingItems shift: duplicates or skipped rowsStable: continues from the last key seen
Jump to page N, total countYesNo (next / previous only)
Sharded dataEach shard must return offset + limit rowsEach shard seeks from the key, results are merged
  • The cursor is the last row's sort key, base64-encoded and opaque (sign or encrypt it if clients must not forge it). Opaque means you can change the scheme later.
  • The sort key must be unique and immutable: add id as a tie-breaker to a timestamp.
  • Enforce a maximum limit (say 100) on the server. Skip total_count unless required: it is a full scan.

Filtering and sorting

API details that get probed

  • Allowlist the filterable and sortable fields: every one needs a supporting index. Arbitrary filters are an accidental full table scan and a denial-of-service vector.
  • Simple equality and ranges as query parameters; complex search belongs in a dedicated POST /search backed by a search index, not in the primary store.
  • fields= (sparse fieldsets) trims payloads without adopting GraphQL.
  • A cursor is only valid for the filter and sort it was issued with: encode them in the cursor or reject a mismatch.

Versioning and backward compatibility

API details that get probed

The best version is no new version: evolve additively and reserve a major version for a real break.

StrategyLooks likeGainPay
URL path/v1/transfersObvious, easy to route, log and cacheCoarse: a new version copies the whole surface
Header / media typeAccept: application/vnd.bank.v2+json or API-Version: 2026-11-01Clean URLs, per-client date pinningInvisible in links and logs, harder to test and cache (Vary)
Additive evolutionNo version change; new optional fields and endpointsNo migration for clientsDiscipline; old fields live for a long time
Backward compatible (safe)Breaking (needs a new version)
Add an optional request field with a defaultAdd a required request field
Add a response fieldRemove or rename a field, change its type or meaning
Add an endpoint or an optional query parameterChange a URL, a default, or a status code
Relax a validation ruleTighten a validation rule
Add an enum value, only if clients were told to tolerate unknown valuesAdd an enum value clients switch on exhaustively

Senior signal: state the policy, not just the mechanism: tolerant readers (ignore unknown fields), expand then contract for renames, a published deprecation window with Deprecation and Sunset headers, and usage metrics per version so you know when removal is safe.

Idempotency for writes

API details that get probed

A client that times out cannot know whether the write happened. Retries are therefore mandatory, and the server must make a repeated request have the effect of one.

MethodSafe (no state change)IdempotentRetry blindly?
GET, HEAD, OPTIONSYesYesYes
PUT (full replace)NoYesYes
DELETENoYes (second call may return 404)Yes
PATCHNoDepends: "set x = 5" yes, "x += 5" noOnly with a key or If-Match
POSTNoNoOnly with an idempotency key
The retry creates no second transfer. The key (unique column, sent as the Idempotency-Key header) and the business write commit in the same transaction, so "key seen" and "money moved" can never disagree.
  • Key: a client-generated UUID per logical operation, scoped to the caller, kept for a window (commonly 24 hours).
  • Same key, different body: store a request hash and reject the mismatch (422 or 409). Same key, still in flight: return 409 or wait, never run twice.
  • Natural keys work too: a unique constraint on (merchant_id, order_id) is an idempotency key the business already has.
  • Retry only on timeouts, connection errors, 429 and 5xx, with exponential backoff and jitter. Retry storms are covered in Distributed Systems Patterns.

Failure mode: checking the key in a cache first and writing later. Two concurrent retries both miss and both execute. The uniqueness check must be atomic with the write.

Partial updates: PUT vs PATCH

API details that get probed

PUTPATCH
SemanticsReplace the whole resourceChange only the fields sent
Missing field meansReset to default or nullLeave unchanged
IdempotentYesNot guaranteed
FormatsFull representationJSON Merge Patch (RFC 7386, simple, null deletes) or JSON Patch (RFC 6902, list of operations)
RiskAn old client wipes fields it does not know aboutAmbiguity between "absent" and "null"
  • Default to PATCH with merge-patch semantics for user-edited resources.
  • Prevent lost updates with optimistic concurrency: the client sends If-Match: "etag", the server returns 412 if the resource changed. See HTTP caching.

Bulk and long-running operations

API details that get probed

Anything that can exceed a few seconds (export, video transcode, bulk import, settlement) should not hold a request open. Accept the job, return a status resource, finish in the background.

The job is a resource with its own state machine: queued, running, succeeded, failed, cancelled. Polling can be replaced by a webhook or SSE.
  • Bulk writes: cap the batch size, and choose all-or-nothing (one transaction) or partial success with a per-item result list (an overall 200 or 207 plus item-level statuses). Say which.
  • Give each item its own idempotency key or client id so a retried batch does not double-apply.
  • Large uploads bypass the API: hand out a pre-signed object-store URL, with multipart / resumable upload for big files.

Webhooks

API details that get probed

A webhook is your API calling the customer's: an HTTP POST per event to a URL they registered. It replaces polling, and it makes you a client of an unreliable server you do not control.

Delivery is at-least-once: the receiver acknowledges fast, processes asynchronously and dedupes on the event id.
ConcernDesign
SigningHMAC-SHA256 over timestamp + "." + raw_body with a per-endpoint secret, sent in a header; receiver compares in constant time. Support two active secrets for rotation.
Replay protectionReject timestamps outside a tolerance (about 5 minutes) and event ids already seen.
RetriesExponential backoff with jitter over hours to days; then a dead-letter state, an alert, and auto-disable of endpoints that keep failing.
OrderingNot guaranteed. Include a sequence number or updated_at; or send thin events ("transfer tr_9x changed") and let the receiver fetch current state.
RecoveryA GET /events?after= API so receivers can reconcile after an outage, plus manual redelivery.
Provider safetySender-side SSRF protection (block private address ranges), per-endpoint timeouts and concurrency so one slow customer cannot stall the rest.

Rate limits and quotas

API details that get probed

From the API's side, a limit is part of the contract: documented, visible in headers, and answered with a clear 429. The algorithms (token bucket, sliding window) are in Building Blocks.

Rate limitQuota
WindowSeconds to minutesDay or month
PurposeProtect the service, smooth burstsEnforce the plan, bill, cap cost
KeyAPI key, user, IP, endpointAccount or tenant
On breach429 + Retry-After429 or 403 with an upgrade message
  • Limit on several dimensions: per key, per tenant, per endpoint class (a search costs more than a read), and a global ceiling.
  • Header names vary: X-RateLimit-* is the long-standing convention, RateLimit-* the IETF draft. Retry-After is the one clients actually obey.
  • 429 means "you sent too much"; 503 means "we are overloaded". Both are retryable, with backoff and jitter.

HTTP caching: ETag, Cache-Control, conditional requests

API details that get probed

One validator, two uses: If-None-Match saves bytes on reads, If-Match prevents lost updates on writes.
DirectiveMeaningUse for
max-age=60Fresh for 60 s in any cachePublic, slowly changing reads
s-maxage=300Overrides max-age for shared caches (CDN)Longer edge TTL than browser TTL
privateBrowser may cache, shared caches must notPer-user responses
no-cacheStore, but revalidate before every useMust be current, cheap 304s
no-storeNever storeBalances, tokens, PII
stale-while-revalidate=30Serve stale while refreshing in the backgroundHiding origin latency

Failure mode: a shared cache storing an authenticated response and serving it to another user. Mark per-user responses private or no-store, and set Vary for anything that changes with a request header.

Compression and payload size

API details that get probed

  • Negotiate: the client sends Accept-Encoding: br, gzip, the server answers with Content-Encoding. JSON typically shrinks by 70 to 90 percent.
  • Skip tiny bodies (under about 1 KB, where the overhead outweighs the gain) and already-compressed media. Compression costs CPU; do it at the edge where possible.
  • Shrink before you compress: pagination, sparse fieldsets, no embedded blobs (return a URL), a binary format for internal hops.
  • Cap sizes: a maximum request body (413 beyond it), a maximum page size, a maximum decompressed size to stop decompression bombs.
  • On mobile networks the payload size often dominates latency; that is the argument for field selection and for HTTP/2 or 3 header compression.

API gateway and Backend-for-Frontend

API details that get probed

A gateway is the single entry point that handles what every API needs. A Backend-for-Frontend (BFF) is a thin per-client API that shapes and aggregates data for one kind of front end.

Gateway: cross-cutting concerns, once. BFF: client-specific composition, owned by the client team. Services stay generic.
API gatewayBFF
DoesTLS termination, authentication, rate limiting, routing, request ids, logging, canary routingAggregates several service calls, trims and reshapes payloads, handles client-specific flows
Owned byPlatform teamThe front-end team it serves
AvoidBusiness logic in the gateway; a single shared gateway config that every team must editDuplicated domain logic across BFFs; one BFF per screen

Trade-off: the gateway is an extra hop (about a millisecond or a few) and a shared failure point, so it must be stateless, horizontally scaled and deployed per zone.

Contract-first design and SDKs

API details that get probed

Write the contract (OpenAPI for REST, .proto for gRPC, the schema for GraphQL) before the code, review it like code, and generate server stubs, client SDKs, docs, mocks and a breaking-change check in CI from that one file.

Real-time communication

Part 4 of 6 · getting data to the client without being asked

HTTP is client-initiated. Every real-time technique is a way around that, trading simplicity for latency and connection state.

Polling, SSE, WebSocket and WebRTC compared

Real-time communication

Short pollingLong pollingServer-Sent EventsWebSocketWebRTC
DirectionClient pullsClient pulls, server holdsServer to clientBoth waysPeer to peer, both ways
TransportHTTPHTTPHTTP (one long response)TCP after HTTP upgradeUDP (SRTP, data channels)
LatencyUp to the poll intervalNear real timeNear real timeLowest over TCPLowest (no in-order stall)
Server stateNoneOne held request per clientOne open connection per clientOne open connection per clientSignalling, STUN / TURN servers
ReconnectNot neededBuilt into the loopAutomatic, resumes with Last-Event-IDYours to buildYours to build (ICE restart)
Proxies, firewallsWorks everywhereWorks everywhereWorks; needs buffering offMostly fine over TLS on 443Needs TURN relay when NAT blocks
Best forJob status, low-frequency dashboardsFallback, simple notificationsFeeds, tickers, progress, LLM token streamsChat, collaboration, gamesVoice, video, screen share

Short polling and long polling

Real-time communication

Short polling: most requests return nothing. Latency averages half the interval; load is clients divided by interval, whether or not anything happened.
Long polling: the server answers only when there is data or on timeout, and the client reconnects at once. The since cursor makes it lossless across reconnects.
  • Arithmetic: 1 million clients polling every 5 s is 200,000 requests per second of mostly empty responses. That number is why you move off short polling.
  • Long polling needs a server that can hold many idle requests cheaply (async I/O), and a hold time below every proxy's idle timeout.

Server-Sent Events and WebSocket

Real-time communication

SSE: one ordinary HTTP response that never ends. The browser reconnects by itself and tells the server where it left off.
WebSocket: an HTTP handshake, then a full-duplex channel with small per-frame overhead. Pings detect dead peers.
  • SSE first when only the server pushes: it is plain HTTP, so auth, gateways, HTTP/2 multiplexing and CDNs keep working. Client-to-server traffic uses normal POSTs. Over HTTP/1.1 browsers cap it at about 6 connections per origin.
  • WebSocket when the client also sends often (typing, cursors, game input) or you need binary frames.
  • Neither gives delivery guarantees. Add message ids, acks and resume-from-id at the application level.

Scaling WebSocket connections

Real-time communication

The problem is not throughput, it is state: each user is pinned to one server, and a message for them can arrive at any other. Separate the stateful connection tier from stateless business logic and join them with pub-sub.

A is on server 1, B on server 2 (a load balancer in front spreads new connections). The message is persisted first, then routed through the backplane to whichever server the registry says holds B's socket.
PieceWhat it doesNotes
Connection serversHold sockets, authenticate at handshake, forward frames. No business logic.Memory-bound: order of 10 to 50 KB per idle connection, so roughly 100k to 1M connections per tuned server
Pub-sub backplaneRoutes a message to the server holding the recipientPer-server or per-user channels (Redis pub-sub, Kafka, NATS). Per-room fan-out for large rooms
Sticky routingThe connection itself is sticky (one TCP connection). Reconnects may land anywhereSo session state lives in the registry, not in server memory
Presenceuser -> server, last_seen with a TTL refreshed by heartbeatsOnline means "heartbeat within N seconds"; fan presence out lazily, it is very chatty
HeartbeatsPing every 20 to 30 s, both to detect dead peers and to keep NAT and proxy idle timers openHalf-open TCP connections are otherwise invisible for minutes
Reconnect with resumeClient reconnects with the last message id (or sequence); server replays the gap from the storeBackoff with jitter, or a server restart causes a reconnect stampede

Failure mode: deploying the connection tier drops every socket at once. Drain gradually (send "reconnect" hints in waves), and size the fleet for the reconnect storm, including the TLS handshakes and resume queries it triggers.

Push notifications

Real-time communication

When the app is in the background there is no socket of yours to write to. The operating system keeps one shared connection to its vendor's push service; you ask that service to deliver.

Your system ends at APNs / FCM. Delivery beyond that point is best effort and outside your control.
  • Device token: issued by the OS per app install, registered with your back end, can change; the provider's response reports invalid tokens, which the workers prune.
  • Best effort: no delivery guarantee, no ordering; providers may throttle, collapse (collapse key) or drop expired messages. Payloads are small (about 4 KB).
  • Design consequence: push is a hint ("you have new messages"), the app then syncs over your API. Never make push the source of truth.
  • Dedupe per user and event, respect preferences and quiet hours, and rate limit per user before the queue.

Choosing among them

Real-time communication

RequirementChooseBecause
Status of a job, updates every 10 s or slower are fineShort pollingStateless, cacheable, nothing new to operate
Server-to-client stream: feed, prices, progress, generated textSSEPlain HTTP, auto-reconnect and resume
Frequent traffic both ways: chat, typing, collaborative editingWebSocketFull duplex, low per-message overhead
Environments where SSE and WebSocket are blockedLong polling as fallbackLooks like ordinary HTTP
Voice, video, low-latency peer dataWebRTC (signalling over WebSocket)UDP media, tolerates loss instead of stalling
User is offline or app is in the backgroundPush via APNs / FCMOnly the OS can wake the device
Server-to-server notificationWebhook or message queueNo long-lived connection needed

Senior signal: pick the simplest thing that meets the latency requirement, and state it as a number: "updates within 2 seconds" is SSE or polling, "under 100 ms both ways" is WebSocket.

Service-to-service communication

Part 5 of 6 · how services talk behind the gateway

Inside the system the choice is less about protocol and more about coupling: does the caller wait for the answer, and what happens to it when the callee is down?

Synchronous vs asynchronous

Service-to-service communication

Synchronous request / response (REST, gRPC)Asynchronous messaging (queue, log)
CouplingTemporal: both must be up at the same moment; caller knows the calleeDecoupled in time; with events, the producer does not know its consumers
LatencyLow and immediate; the caller's latency includes the callee'sCaller returns at once; end-to-end completion is later and variable
Failure behaviourFailure propagates up the chain; availability multiplies (three 99.9% hops is about 99.7%)Work waits in the queue; the consumer catches up after recovery
Load spikesHit the callee directly; needs load sheddingAbsorbed by the queue; needs lag monitoring
ConsistencyImmediate answer, read-your-write is easyEventual; needs idempotent consumers and often an outbox
DebuggingOne trace, one stackNeeds correlation ids across hops, harder to follow
Use forQueries, and commands whose result the user is waiting forSide effects, fan-out, slow or bursty work, cross-team integration
Top: the caller waits on every hop. Bottom: the caller returns after publishing. The usual shape is a short synchronous path for what the user is waiting on, and events for everything else.

Interview tip: a long synchronous chain is a red flag. Ask of every hop: "does the user need this answer before we respond?" If not, publish an event.

Request-reply over a queue

Service-to-service communication

When you want the buffering of a queue but still need an answer: send the request with a correlation id and a reply address, and match the reply when it arrives.

The correlation id is the only link between request and reply. The requester must handle a reply that is late, duplicated or never comes.
  • Use for slow, bursty or rate-limited back ends (pricing engines, document rendering) where you want backpressure and worker scaling.
  • Avoid for low-latency lookups: you have rebuilt RPC with more moving parts. If the requester blocks waiting, you keep the temporal coupling and lose the simplicity.

Events vs commands

Service-to-service communication

CommandEvent
Says"Do this": ChargeCard"This happened": PaymentCaptured
Tense, namingImperativePast tense, a fact
ReceiversExactly one ownerZero to many subscribers
Can be rejectedYesNo, it already happened
Who knows whomSender knows the receiverReceivers know the producer's event
ChannelQueue (point to point)Topic / log (publish-subscribe)
A command has one addressee and an outcome. An event is broadcast, and adding a fourth subscriber changes nothing in the producer.

Trade-off: events remove coupling but hide the flow: no single place shows "what happens when an order is placed". Choreography vs orchestration and sagas are in Architecture Styles and Distributed Systems Patterns.

Event schema and versioning

Service-to-service communication

  • id for consumer dedupe (delivery is at-least-once); subject + sequence for per-entity ordering; occurred_at is event time, not processing time; trace_id ties the hops together.
  • An event is a public API. Consumers you have never met depend on it, and old events sit in the log for days or forever, so readers must handle every version still retained.
  • Evolve additively: new optional fields with defaults only. For a real break, publish a new type (payment.captured.v2) alongside the old one and migrate consumers.
  • Schema registry with compatibility checks in CI (backward: new readers read old data; forward: old readers read new data) stops a producer deploy from breaking consumers.
  • Thin vs fat: thin events (ids only) force a callback but are never stale; fat events (full state) decouple consumers but spread data, including PII, everywhere.

Service mesh and sidecars

Service-to-service communication

A mesh moves network concerns out of application code into a proxy deployed next to every service instance (the sidecar), configured centrally by a control plane.

The apps speak plain HTTP or gRPC to localhost. The proxies form the data plane; the control plane pushes certificates, routes and policy to them, and they emit metrics and traces.
You getHow
mTLS everywhereShort-lived workload certificates issued and rotated by the control plane; service identity, not IP addresses
Traffic policyTimeouts, retries with budgets, circuit breaking, per-request load balancing, canary and traffic splitting
ObservabilityUniform request metrics (rate, errors, latency), access logs and trace spans for every hop, in any language
Authorization"Service A may call B's /transfers" as policy

Trade-off: two extra proxy hops per call (typically around a millisecond in total, more at the tail), extra CPU and memory per instance, and a complex system to operate. Worth it with many services in several languages; overkill for five services where a shared library does the job. Sidecar-less (per-node) meshes reduce the overhead.

Timeouts, retries and deadlines

Service-to-service communication

One-line pointers; the full treatment is in Distributed Systems Patterns.

  • Timeout: every network call has one, set from the callee's p99, not left at a library default of "forever".
  • Retry: only idempotent calls, only on retryable errors, with exponential backoff, jitter and a retry budget; retry at one layer, not every layer (3 retries at 3 layers is 27 calls).
  • Deadline: an absolute time passed down the chain so every hop knows the remaining budget and abandons work nobody is waiting for.
  • Circuit breaker: stop calling a failing dependency and fail fast or degrade.
  • Backpressure and load shedding: reject early (429 / 503) rather than queue without bound.

Serialization formats

Service-to-service communication

JSONProtocol BuffersAvroMessagePack
EncodingTextBinary, tagged fieldsBinary, no tags or names in the dataBinary JSON-like
Size (approx.)Baseline (largest)Often 2 to 5 times smallerSmallest for recordsSomewhat smaller than JSON; field names still included
SpeedSlowest to parseFast, generated codeFastFaster than JSON
SchemaOptional (JSON Schema)Required .proto, codegenRequired; the reader needs the writer's schema (registry or file header)None
EvolutionBy convention onlyField numbers: add new, never reuseReader / writer schema resolution with defaults, checked by a registryBy convention only
Human-readableYesNoNoNo
Typical usePublic APIs, config, logsgRPC, internal RPCKafka events, data lake filesCaches, compact internal payloads without codegen

Interview tip: JSON at the edge for reach and debuggability, a schema-based binary format inside for cost and safety. Mention that JSON numbers lose precision above 253 in many parsers, so 64-bit ids and money travel as strings or integers in minor units.

Security on the wire

Part 6 of 6 · who is calling, what may they do, and who can listen

Security is rarely the main topic of a design interview, but one sentence per layer is expected, and a design with user_id in the request body gets marked down.

Authentication vs authorization

Security on the wire

Authentication (authn) answers "who are you?" and fails with 401. Authorization (authz) answers "may you do this to that object?" and fails with 403 (or 404 to hide existence).

Authenticate once at the edge; authorize in the service that owns the data, on every request, for the specific object.
  • The gateway can check coarse rules (valid token, required scope). Only the owning service can check "does account 42 belong to this caller".
  • Internal services must not trust a bare X-User-Id header: pass a signed token or rely on mTLS identity plus a propagated, verifiable user context.

Sessions vs tokens

Security on the wire

Server-side sessionSelf-contained token (JWT)
Client holdsOpaque random id, usually in a cookieSigned claims, in a header or cookie
Server stateSession store (Redis or database)None; verify the signature locally
Per-request costOne store lookup (sub-millisecond to about 1 ms)CPU-only signature check
RevocationImmediate: delete the rowHard: valid until expiry unless you add a denylist
Size on the wireAbout 32 bytesHundreds of bytes to a few KB, on every request
Cross-service useEvery service needs the storeAny service with the public key can verify
Main riskCSRF (cookies are sent automatically)Theft via XSS if kept in JavaScript-readable storage; stale claims
FitsFirst-party web apps, when instant logout mattersAPIs, mobile, microservices, third-party access

Senior signal: the common hybrid: an opaque session or refresh token at the edge (revocable), exchanged for a short-lived JWT that internal services verify without a lookup.

JWT: anatomy, validation, expiry and revocation

Security on the wire

  • Validate, in order: algorithm against an allowlist (never trust alg from the token, reject none); signature with the key for kid from the issuer's JWKS (cached); iss; aud; exp and nbf with a small clock skew; then scopes.
  • Prefer asymmetric signing (RS256, ES256, EdDSA): services verify with a public key and cannot mint tokens. A shared HMAC secret lets every verifier forge.
  • Keep PII and secrets out of the payload; keep it small.
Short access tokens bound the damage of a leak; the refresh call is the one place revocation is checked.
Revocation approachGainPay
Short expiry (5 to 15 min) + refresh token rotationStateless verificationRevocation lags by up to the access-token lifetime
Denylist of jti or "tokens issued before T for user U", in a cacheNear-immediate logoutA lookup per request: state is back
Opaque tokens + introspectionImmediate, central controlA network call per request (cacheable for seconds)
Rotate the signing keyKills everything at onceLogs out every user; emergency only

OAuth 2.0 authorization code with PKCE

Security on the wire

OAuth 2.0 is delegated authorization: a user lets an app call an API on their behalf without giving the app their password. Authorization code with PKCE is the flow for web, mobile and single-page apps.

challenge = BASE64URL(SHA256(code_verifier)). The code travels through the browser, the tokens never do. PKCE proves that whoever redeems the code is the app that started the flow.
  • PKCE (proof key for code exchange) defeats authorization-code interception: a stolen code is useless without the verifier. Required for public clients (mobile, SPA), recommended for all.
  • state ties the callback to the request (CSRF protection); redirect URIs are matched exactly against a registered list.
  • Scopes limit what the token can do; the API still enforces object-level authorization.
  • The implicit flow and the password grant are deprecated: do not propose them.

Client credentials and OpenID Connect

Security on the wire

When there is no user, a service authenticates as itself with the client credentials grant.

The token names the calling service, not a user. Cache it; do not fetch one per request.
  • Client authentication can be a secret, a signed assertion (private_key_jwt) or mTLS; prefer the last two over long-lived shared secrets.
  • When a service acts for a user, propagate the user's context (token exchange or a signed internal token) so downstream authorization still sees the real principal.
  • OpenID Connect in one line: an identity layer on OAuth 2.0 that adds an ID token (a JWT saying who logged in, for the client) next to the access token (for the API), which is what "Sign in with ..." uses.

API keys and when they are enough

Security on the wire

An API key is a long random secret that identifies a calling application or account, not a user.

Enough whenNot enough when
Server-to-server calls from a customer's back endActing on behalf of an end user (use OAuth)
Identifying a caller for quotas, billing and analyticsThe key would ship in a browser or mobile app (it will be extracted)
Low-risk, read-mostly or public dataFine-grained, short-lived or delegated permissions are needed
  • Send in a header, never in the URL (URLs end up in logs and referrers).
  • Store only a hash; show the key once; give it a recognisable prefix (sk_live_) so secret scanners can find leaks.
  • Scope keys (read-only, per environment), allow several active keys for rotation, support expiry and optional IP allowlists.
  • For high-value calls, add request signing (HMAC over method, path, body and timestamp) so a captured request cannot be altered or replayed.

RBAC vs ABAC

Security on the wire

RBAC: role-basedABAC: attribute-basedReBAC: relationship-based
Decision fromUser -> roles -> permissionsPolicy over attributes of user, resource, action, contextA graph of relations (owner, member, parent)
Example"Tellers can view accounts""Tellers can view accounts in their branch, in business hours, under 1M""Editors of a folder can edit documents inside it"
GainSimple, auditable, easy to explainFine-grained, context-aware, no role explosionNatural for sharing and hierarchies
PayRole explosion when rules depend on dataHarder to audit ("who can see X?"), policy engine neededA graph store and careful consistency
  • Typical answer: RBAC for coarse permissions, plus an ownership / tenant check (an attribute) on every object. That hybrid covers most systems.
  • Keep policy out of scattered if statements: one policy decision point (library or service), enforced in each service, with decisions cached briefly.
  • Deny by default.

TLS and mTLS

Security on the wire

TLS gives confidentiality, integrity and server authentication. Mutual TLS (mTLS) adds a client certificate, so each side proves its identity; it is the standard for service-to-service traffic.

TLS 1.3 completes in one round trip. Asymmetric keys only authenticate and agree a secret; the traffic itself uses fast symmetric encryption.
  • Ephemeral key exchange gives forward secrecy: stealing the server's key later does not decrypt recorded traffic.
  • Where TLS ends matters: terminating at the load balancer leaves the internal hop in plain text. Re-encrypt to the back end, or use a mesh, for zero-trust networks.
  • mTLS cost is operational: issuing, rotating and revoking certificates for every workload. Use short-lived certificates (hours to days) issued automatically.
  • mTLS authenticates the service. It says nothing about the end user, so it complements tokens rather than replacing them.

Encryption and key management

Security on the wire

In transitAt restApplication / field level
Protects againstNetwork eavesdropping and tamperingStolen disks, backups, snapshotsDatabase admins, SQL injection dumps, over-broad internal access
HowTLS 1.2+ on every hop, including internalDisk, volume or database encryption (AES-256)Encrypt selected fields (card number, national id) before storing
Does not protectData at the endpointsA running system with valid accessThe application holding the key
PayHandshake CPU, certificate operationsAlmost nothingNo indexing or range queries on ciphertext; key handling in the app
Envelope encryption: a data key (DEK) encrypts the data, a key-encryption key (KEK) that never leaves the KMS encrypts the DEK.
  • Why envelope: bulk data never goes to the KMS (which is slow and rate limited); a DEK per object or tenant limits the blast radius; rotating the KEK means re-wrapping small DEKs, not re-encrypting terabytes.
  • Key management rules: keys live in a KMS or HSM, never in code or next to the data; access is by IAM policy and audited; rotate on a schedule; separate keys per environment and, where needed, per tenant.
  • Crypto-shredding: deleting a tenant's key makes all its data, including backups, unreadable. Useful for deletion requirements.
  • Use authenticated encryption (AES-GCM or ChaCha20-Poly1305). Never invent a scheme.

Password storage

Security on the wire

Passwords are never encrypted and never stored with a fast hash. Store the output of a salted, deliberately slow, memory-hard password hash.

  • Algorithm: Argon2id preferred; bcrypt or scrypt are acceptable. Not MD5, SHA-1 or plain SHA-256: those run at billions of guesses per second on GPUs.
  • Salt: unique and random per user, stored with the hash. It defeats rainbow tables and makes identical passwords look different.
  • Work factor: tune to roughly 100 ms or more per hash on your hardware, and raise it over time. That cost is also why login needs its own rate limit.
  • Pepper (optional): a secret kept in the KMS, applied to all hashes, so a database dump alone is not enough.
  • Around it: rate limit and lock out per account and per IP, check against breached-password lists, offer MFA or passkeys, give identical responses for "no such user" and "wrong password".

Secrets management

Security on the wire

  • Where: a secrets manager or vault, fetched at start-up or injected at deploy time by workload identity. Not in source control, container images, or logs.
  • Prefer no secret: workload identity (cloud IAM roles, mesh certificates) removes static credentials altogether.
  • Short-lived and dynamic: database credentials issued per service instance with a TTL beat a shared password that never changes.
  • Rotation without downtime: accept old and new together for a window (two active keys), then retire the old one. Design every credential for this from day one.
  • Least privilege and audit: each service reads only its own secrets; every read is logged.
  • Leak response: secret scanning in CI and on push, and a rehearsed revoke-and-rotate procedure. A leaked secret is rotated, not "removed from git history".

Attack classes and controls

Security on the wire

AttackWhat happensControl in the design
Injection (SQL, NoSQL, command)Input is interpreted as code or queryParameterised queries, no string-built queries or shell calls, schema validation, least-privilege database user
Broken object-level authorization (BOLA / IDOR)Caller changes /accounts/42 to /accounts/43 and it worksCheck ownership or tenant on every object access, in the data-access layer; non-guessable ids as defence in depth, never as the control
CSRFAnother site makes the user's browser send an authenticated request (cookies ride along)SameSite cookies, anti-CSRF tokens, check Origin; not an issue for bearer tokens sent in a header
XSSAttacker script runs in your page, acts as the user, steals tokensOutput encoding, Content-Security-Policy, HttpOnly cookies, no tokens in JavaScript-readable storage, correct Content-Type on API responses
SSRFServer fetches an attacker-chosen URL (webhook, image import) and reaches internal services or cloud metadataAllowlist destinations, resolve then block private and link-local ranges, no redirects, fetch from an isolated egress proxy
ReplayA captured valid request is sent againTLS, plus timestamp + nonce in signed requests, idempotency keys, short token lifetimes
DDoS and resource exhaustionVolume, or a few very expensive requestsCDN / scrubbing at the edge, rate limits and quotas, payload and page-size caps, timeouts, query cost limits, autoscaling with a ceiling
Credential stuffingLeaked passwords replayed at loginPer-account and per-IP limits, MFA, breached-password checks, bot detection
Mass assignmentClient sets fields it should not (role, balance)Explicit request schemas with an allowlist of writable fields

Interview tip: broken object-level authorization is the top item in the OWASP API Security Top 10. Saying "every read and write is scoped by the caller's id from the token" is the one security sentence worth making in every design.

Input validation and least privilege

Security on the wire

  • Validate at the boundary against a schema: types, lengths, ranges, formats, enums. Allowlist what is valid instead of blocklisting what is bad; reject unknown fields on writes.
  • Validate again where it matters: business invariants (balance never negative) live in the service and the database constraints, not only in the gateway or the client.
  • Bound everything: body size, array length, string length, nesting depth, page size, request time.
  • Least privilege: each service gets its own identity, its own database user limited to its tables, network policy limited to the services it calls, and tokens scoped to the operations it needs.
  • Defence in depth: assume one layer fails. The gateway authenticates, the service authorizes, the database user cannot drop tables, the network blocks the rest.

Multi-tenancy isolation

Security on the wire

The tenant id comes from the authenticated token, never from a request parameter, and is applied to every query, cache key, queue and object path by a shared layer (or database row-level security) rather than by each developer remembering a WHERE clause. Choose the isolation level by risk and cost: shared tables with a tenant column (cheapest, weakest), a schema or database per tenant, or a dedicated stack for the largest or most regulated tenants, with per-tenant rate limits and quotas so one noisy tenant cannot starve the rest.

Audit logging and PII handling

Security on the wire

Audit log: an append-only record of security-relevant actions: who (user, service, on whose behalf), did what, to which object, when, from where, with what outcome, and the request id. It is separate from debug logs, write-once, access-controlled, and retained per policy.

PII ruleIn practice
MinimiseDo not collect or return what the use case does not need; keep PII out of URLs, events and tokens; pass ids, not copies
MaskRedact in logs and traces at the logging layer; show partial values (last 4 digits); tokenise card numbers; field-level encryption for the most sensitive columns
RetainA retention period per data class, enforced by TTL or scheduled deletion; backups and analytics copies included
DeleteA deletion or erasure workflow that reaches every store, cache, search index and downstream consumer; crypto-shredding for backups
Control accessRole-gated, purpose-bound, and itself audited ("who looked at this customer?")
LocateKnow which region the data lives in; residency rules can force per-region storage and routing

Failure mode: the database is locked down, but the same PII sits in application logs, an analytics topic and a search index with broad access. Deletion and masking must follow the data, which is the strongest argument for thin events and central PII storage.

Cheat sheet: need, choice, reason

NeedChoiceReason
Public or partner APIREST + JSON + OpenAPIUniversal reach, cacheable, easy to debug
Internal service callsgRPC + ProtobufTyped contracts, small payloads, streaming, deadlines
Many clients, different views of one graphGraphQL or a BFF per clientClient-shaped responses in one round trip
Large or changing listCursor / keyset paginationConstant cost per page, stable under inserts
Retry-safe create or paymentIdempotency key, stored atomically with the writeA timeout cannot cause a duplicate
Concurrent editsETag + If-Match, 412 on conflictNo lost updates, no locks held
Work longer than a few seconds202 + job resource; webhook when doneNo held connections, progress is visible
Notify another company's systemSigned webhook, at-least-once, event idNo polling; receiver dedupes
Server-to-client streamSSEPlain HTTP, auto-reconnect and resume
Two-way real timeWebSocket + pub-sub backplaneFull duplex; backplane routes across servers
App in backgroundAPNs / FCM push, then syncOnly the OS can wake the device
Decouple side effectsEvents on a topicProducer does not wait for or know consumers
Evolve without breakingAdditive changes, tolerant readers, /v1 for real breaksClients upgrade on their own schedule
User login for an appOAuth 2.0 authorization code + PKCE, OIDCNo password in the app, short-lived scoped tokens
Service identitymTLS, or client credentialsMutual authentication, no shared static secrets
Instant logoutServer session, or short JWT + denylistPure JWT cannot be revoked before expiry
Protect sensitive fieldsEnvelope encryption with a KMSSmall blast radius, cheap rotation
Abuse and overloadRate limits + quotas, 429 with Retry-AfterPredictable contract, protects the back end

API review checklist

  • Shape: resources are nouns, methods carry the verbs, ids are opaque, one casing, UTC timestamps, money in minor units.
  • Identity: caller and tenant come from the token; every object access is authorized for that caller.
  • Writes: each write is idempotent by nature or by key; behaviour on timeout and retry is defined.
  • Reads: lists are paginated with a cursor and a server-side maximum; filters and sorts are allowlisted and indexed.
  • Errors: correct status class, one error format, stable machine codes, request id, no internals leaked.
  • Slow work: anything over a few seconds is a job with a status resource.
  • Limits: rate limits, quotas, payload and page caps, timeouts, all documented and visible in headers.
  • Caching: each GET is deliberately cacheable, private or no-store; ETags where clients revalidate or edit.
  • Evolution: changes are additive; versioning and deprecation policy are stated; the contract is the source of truth.
  • Real time: the push channel is the simplest that meets the latency number, with reconnect and resume.
  • Between services: timeouts and deadlines everywhere, retries bounded, async where the user is not waiting, events versioned.
  • Security: TLS on every hop, least-privilege scopes, input validated against a schema, secrets in a manager, PII minimised and masked, sensitive actions audited.

Next: how these services are arranged into a whole system, in Architecture Styles.