System design, part 5 of 12
API and Communication Design
The "API" step of a design interview, and everything that travels over it: how to get from requirements to endpoints in two minutes, which style and transport to pick, the details interviewers probe (pagination, idempotency, versioning, webhooks), real-time delivery, service-to-service calls, and the security that sits on every one of them. For hands-on code see Web APIs with FastAPI. Numbers are approximate, order of magnitude only.
Designing an API in an interview
Part 1 of 6 · from requirements to endpoints
The API step is a contract, not a catalogue. Spend about two minutes, write 4 to 6 calls that cover the core use cases, and let them drive the data model and the high-level design that follow.
Requirements to endpoints in two minutes
Designing an API in an interview
Turn each functional requirement into a noun and a verb, then write one line per call. Stop when the core flows are covered.
- Resources: the things users create and read (account, transfer, message). They usually become your tables too.
- Operations: one line each,
METHOD /path. Only the calls the requirements need. - Request / response: the 3 to 5 fields that matter. Identity comes from the token, never from the body.
- Errors: name the two or three that are specific to the domain (insufficient funds, duplicate request).
- Cross-cutting: say once, in one sentence: "all calls are authenticated, writes take an idempotency key, lists are cursor-paginated".
Interview tip: say which style you are using and why in one breath ("REST over HTTPS for the public API, gRPC between services"), then move on. Do not design every CRUD endpoint.
Worked example: a bank-transfer API
Designing an API in an interview
Requirements: move money between accounts, see a transfer's status, list an account's history. This is the whole API sketch you would write on the board.
- The create call returns
pending: settlement is asynchronous, so the status resource is part of the contract from the start. - The idempotency key is required, not optional: a retried payment is the first thing an interviewer will ask about. See Idempotency for writes.
availablevsledgerbalance signals you know holds and pending entries exist.
Naming and resource modelling
Designing an API in an interview
| Rule | Do | Avoid |
|---|---|---|
| Plural nouns, verbs come from the method | POST /transfers | POST /createTransfer |
| Nest only for true ownership, one level deep | /accounts/{id}/transfers | /users/1/accounts/2/transfers/3/entries |
| Actions that are not CRUD become sub-resources or new resources | POST /transfers/{id}/cancel, or POST /refunds | PATCH status=cancelled with hidden side effects |
| Opaque, non-guessable ids with a type prefix | tr_9xK2... | Auto-increment integers (enumerable, leak volume) |
| One casing, one time format, one money format | snake_case, RFC 3339 UTC, minor units + currency | Mixed casing, local times, floats |
| The API is not the schema | Model the client's use case | Exposing tables and join keys one to one |
| GET never changes state | Safe, cacheable reads | GET /transfers/{id}/cancel |
Senior signal: model the state machine as a resource (pending -> settled | failed) instead of a boolean. Async reality then fits the API without a breaking change later.
Choosing status codes
Designing an API in an interview
The status code is for machines (retry or not, cache or not); the body is for developers. The key split: 4xx means "do not retry unchanged", 5xx means "retry may help".
| Code | Use when | Client should |
|---|---|---|
| 200 OK | Read or update succeeded, body returned | Proceed |
| 201 Created | Resource created; send Location | Proceed |
| 202 Accepted | Work queued, not done; return a status URL | Poll or wait for a webhook |
| 204 No Content | Success with no body (delete) | Proceed |
| 304 Not Modified | Conditional GET, client copy is current | Use cached copy |
| 400 Bad Request | Malformed or invalid input | Fix the request |
| 401 Unauthorized | Missing or invalid credentials | Authenticate, refresh token |
| 403 Forbidden | Authenticated but not allowed | Do not retry |
| 404 Not Found | No such resource (also used to hide resources the caller may not see) | Do not retry |
| 409 Conflict | State conflict: duplicate, concurrent edit, wrong state | Re-read, then decide |
| 412 Precondition Failed | If-Match ETag no longer matches | Re-read, reapply, retry |
| 422 Unprocessable | Well-formed but violates a business rule | Fix the request |
| 429 Too Many Requests | Rate limit or quota hit; send Retry-After | Back off, then retry |
| 500 Internal Error | Unhandled server fault | Retry with backoff if idempotent |
| 502 / 504 | Gateway got a bad response / no response in time from upstream | Retry with backoff if idempotent |
| 503 Unavailable | Overloaded or shedding load; send Retry-After | Back off, then retry |
Failure mode: returning 200 with {"error": ...} in the body. Proxies, retries, alerting and client libraries all key off the status line, so every one of them now misbehaves.
Error format
Designing an API in an interview
One error shape for the whole API. A common standard is Problem Details (RFC 9457, media type application/problem+json); a house format is fine if it carries the same things.
- Stable machine code (
insufficient_funds) that clients branch on; the human text can change freely. - Request id echoed in a header and the body, the join key into logs and traces.
- Field-level list for validation errors, so a form can show all problems at once.
- No internals: no stack traces, SQL, hostnames, or "user exists" hints on login and reset endpoints.
What interviewers look for
Designing an API in an interview
| Topic | Mid-level answer | Senior answer |
|---|---|---|
| Scope | Lists every CRUD endpoint | 4 to 6 calls tied to the requirements, then moves on |
| Writes | POST and hope | Idempotency key, defined behaviour on retry and timeout |
| Lists | ?page=3 | Cursor, max page size, stable sort key |
| Slow work | Blocks the request | 202 + status resource, or webhook |
| Identity | user_id in the body | From the token; object-level authorization on every call |
| Evolution | "We will add v2" | Additive changes, tolerant readers, deprecation policy |
| Failure | Happy path only | Timeouts, partial failure, what the client sees and does |
Interview tip: the API is where you prove the system is usable under failure. For each write, be ready to answer "the client timed out, what now?".
API styles
Part 2 of 6 · REST, gRPC, GraphQL, streaming and the HTTP versions underneath
The style follows from who the caller is (browser, mobile app, partner, another service) and the shape of the traffic (request / response, streams, many views of one graph). Most real systems use two or three of them.
REST vs gRPC vs GraphQL vs WebSocket
| REST | gRPC | GraphQL | WebSocket / streaming | |
|---|---|---|---|---|
| Transport | HTTP/1.1 or newer | HTTP/2 (HTTP/3 emerging) | HTTP, usually one POST endpoint | TCP after an HTTP upgrade |
| Schema | Optional (OpenAPI) | Required (.proto), code generated | Required (typed schema), introspectable | None built in; you define messages |
| Payload | JSON text | Protobuf binary | JSON, shaped by the query | Text or binary frames |
| Streaming | No (SSE or chunking as add-ons) | Client, server and bidirectional | Subscriptions, over WebSocket or SSE | Full duplex, native |
| Caching | Native HTTP caching, CDN-friendly | None at the HTTP layer | Hard: POST, per-query shape; needs persisted queries | None |
| Browser support | Native | Needs gRPC-Web and a proxy | Native (it is HTTP + JSON) | Native |
| Best for | Public and partner APIs, CRUD, cacheable reads | Service-to-service, low latency, polyglot back ends | Many clients with different view needs over one graph | Chat, live updates, collaboration, games |
| You pay | Over- and under-fetching, chatty clients | Opaque on the wire, weaker browser and tooling reach | Query cost control, caching, N+1 resolvers | Stateful connections to scale and route |
REST constraints that matter
Nobody is asked to recite Fielding. These are the constraints that change the design.
| Constraint | Meaning in practice | What it buys |
|---|---|---|
| Stateless | Each request carries everything needed (token, cursor); no server-side conversation state | Any instance can serve any request: horizontal scaling, simple failover |
| Uniform interface | Resources named by URL, standard methods and status codes | Generic clients, proxies, gateways and caches work without custom code |
| Cacheable | Responses declare cacheability (Cache-Control, ETag) | CDN and browser offload for reads |
| Layered | Client cannot tell whether it talks to origin, gateway or CDN | You can insert load balancers, WAF, gateways freely |
| Method semantics | GET safe; PUT and DELETE idempotent; POST neither | Intermediaries and clients know what is safe to retry |
Trade-off: HATEOAS (links that drive the client) is rarely implemented fully. Returning a few links (next, status_url) gets most of the value.
gRPC and Protocol Buffers
An RPC framework: a .proto contract generates typed clients and servers in many languages; calls run as HTTP/2 streams with binary Protobuf payloads. Use it inside the data centre; keep REST or GraphQL at the edge.
- Why it suits service-to-service: compact payloads (often several times smaller than JSON), generated typed stubs, multiplexing on long-lived connections, streaming, and first-class deadlines and cancellation that propagate down the call chain.
- Schema evolution: add fields with new numbers; unknown fields are ignored by old readers. Never change a field's type or number.
- Failure mode: a layer 4 load balancer pins a long-lived HTTP/2 connection to one backend, so load never rebalances. Use layer 7 (per-request) balancing or client-side balancing; see Building Blocks.
- You pay: not human-readable, no HTTP caching, browsers need gRPC-Web through a proxy.
GraphQL
The client sends a query naming exactly the fields it wants across a typed graph; one round trip replaces several REST calls. It fixes over-fetching (fat responses a mobile client throws away) and under-fetching (call 1 for the list, then N calls for details).
| Where it hurts | Why | Mitigation |
|---|---|---|
| HTTP caching | One endpoint, POST, a different shape per query | Persisted queries (hash sent with GET), client-side normalised cache, per-field cache hints |
| Rate limiting | One request can cost 1 or 10,000 database reads | Query cost analysis, depth and complexity limits, limit by cost not by request count |
| Abuse | Deeply nested or aliased queries | Depth limit, timeout, allowlist of persisted queries for first-party clients |
| Authorization | Many paths reach the same object | Enforce in the data layer per object and field, not per query |
| Observability | Everything is POST /graphql with 200 | Metrics per operation name and per resolver |
When each is the right answer
- REST: the default for anything public. Everyone can call it, debug it with curl, and cache it.
- gRPC: internal calls where latency, typing and streaming matter and you control both ends.
- GraphQL: product APIs with many client teams and fast-changing views. Avoid for simple CRUD, public partner APIs and file transfer.
- WebSocket / SSE: only where the server must speak first. See Real-time communication.
HTTP/1.1 vs HTTP/2 vs HTTP/3
| HTTP/1.1 | HTTP/2 | HTTP/3 | |
|---|---|---|---|
| Transport | TCP (+ TLS) | TCP (+ TLS in practice) | QUIC over UDP, TLS 1.3 built in |
| Framing | Text | Binary frames | Binary frames |
| Multiplexing | No: one request at a time per connection, browsers open about 6 per origin | Many streams on one connection | Many streams on one connection |
| Head-of-line blocking | At the HTTP layer | Gone at HTTP, still at TCP: one lost packet stalls every stream | Gone: loss stalls only the stream it belongs to |
| Header compression | None | HPACK | QPACK |
| Connection setup | TCP + TLS: 2 to 3 round trips | Same as 1.1 | 1 round trip, 0-RTT on resumption |
| Network change | Connection breaks | Connection breaks | Connection id survives a Wi-Fi to mobile switch |
| Typical use | Legacy, simple internal calls | Default for web and gRPC | Edge to client on lossy mobile networks |
Interview tip: the usual deployment is HTTP/3 or HTTP/2 from client to edge, and HTTP/2 or HTTP/1.1 with keep-alive from edge to origin. 0-RTT data can be replayed, so allow it only for idempotent requests.
API details that get probed
Part 3 of 6 · the follow-up questions
After the endpoint list, interviewers pick one or two of these and dig. Each has a default answer worth knowing cold.
Pagination
Default to cursor (keyset) pagination for anything large or changing. Offset is acceptable only for small, mostly static lists that need "jump to page N".
Offset: ?limit=50&offset=100000 | Cursor / keyset: ?limit=50&cursor=... | |
|---|---|---|
| Query | ORDER BY created_at DESC LIMIT 50 OFFSET 100000 | WHERE (created_at, id) < (:t, :id) ORDER BY created_at DESC, id DESC LIMIT 50 |
| Cost of a deep page | Scans and discards 100,000 rows: O(offset) | Index seek: O(page size), the same at any depth |
| Rows inserted while paging | Items shift: duplicates or skipped rows | Stable: continues from the last key seen |
| Jump to page N, total count | Yes | No (next / previous only) |
| Sharded data | Each shard must return offset + limit rows | Each shard seeks from the key, results are merged |
- The cursor is the last row's sort key, base64-encoded and opaque (sign or encrypt it if clients must not forge it). Opaque means you can change the scheme later.
- The sort key must be unique and immutable: add
idas a tie-breaker to a timestamp. - Enforce a maximum
limit(say 100) on the server. Skiptotal_countunless required: it is a full scan.
Filtering and sorting
- Allowlist the filterable and sortable fields: every one needs a supporting index. Arbitrary filters are an accidental full table scan and a denial-of-service vector.
- Simple equality and ranges as query parameters; complex search belongs in a dedicated
POST /searchbacked by a search index, not in the primary store. fields=(sparse fieldsets) trims payloads without adopting GraphQL.- A cursor is only valid for the filter and sort it was issued with: encode them in the cursor or reject a mismatch.
Versioning and backward compatibility
The best version is no new version: evolve additively and reserve a major version for a real break.
| Strategy | Looks like | Gain | Pay |
|---|---|---|---|
| URL path | /v1/transfers | Obvious, easy to route, log and cache | Coarse: a new version copies the whole surface |
| Header / media type | Accept: application/vnd.bank.v2+json or API-Version: 2026-11-01 | Clean URLs, per-client date pinning | Invisible in links and logs, harder to test and cache (Vary) |
| Additive evolution | No version change; new optional fields and endpoints | No migration for clients | Discipline; old fields live for a long time |
| Backward compatible (safe) | Breaking (needs a new version) |
|---|---|
| Add an optional request field with a default | Add a required request field |
| Add a response field | Remove or rename a field, change its type or meaning |
| Add an endpoint or an optional query parameter | Change a URL, a default, or a status code |
| Relax a validation rule | Tighten a validation rule |
| Add an enum value, only if clients were told to tolerate unknown values | Add an enum value clients switch on exhaustively |
Senior signal: state the policy, not just the mechanism: tolerant readers (ignore unknown fields), expand then contract for renames, a published deprecation window with Deprecation and Sunset headers, and usage metrics per version so you know when removal is safe.
Idempotency for writes
A client that times out cannot know whether the write happened. Retries are therefore mandatory, and the server must make a repeated request have the effect of one.
| Method | Safe (no state change) | Idempotent | Retry blindly? |
|---|---|---|---|
| GET, HEAD, OPTIONS | Yes | Yes | Yes |
| PUT (full replace) | No | Yes | Yes |
| DELETE | No | Yes (second call may return 404) | Yes |
| PATCH | No | Depends: "set x = 5" yes, "x += 5" no | Only with a key or If-Match |
| POST | No | No | Only with an idempotency key |
Idempotency-Key header) and the business write commit in the same transaction, so "key seen" and "money moved" can never disagree.- Key: a client-generated UUID per logical operation, scoped to the caller, kept for a window (commonly 24 hours).
- Same key, different body: store a request hash and reject the mismatch (422 or 409). Same key, still in flight: return 409 or wait, never run twice.
- Natural keys work too: a unique constraint on
(merchant_id, order_id)is an idempotency key the business already has. - Retry only on timeouts, connection errors, 429 and 5xx, with exponential backoff and jitter. Retry storms are covered in Distributed Systems Patterns.
Failure mode: checking the key in a cache first and writing later. Two concurrent retries both miss and both execute. The uniqueness check must be atomic with the write.
Partial updates: PUT vs PATCH
| PUT | PATCH | |
|---|---|---|
| Semantics | Replace the whole resource | Change only the fields sent |
| Missing field means | Reset to default or null | Leave unchanged |
| Idempotent | Yes | Not guaranteed |
| Formats | Full representation | JSON Merge Patch (RFC 7386, simple, null deletes) or JSON Patch (RFC 6902, list of operations) |
| Risk | An old client wipes fields it does not know about | Ambiguity between "absent" and "null" |
- Default to PATCH with merge-patch semantics for user-edited resources.
- Prevent lost updates with optimistic concurrency: the client sends
If-Match: "etag", the server returns 412 if the resource changed. See HTTP caching.
Bulk and long-running operations
Anything that can exceed a few seconds (export, video transcode, bulk import, settlement) should not hold a request open. Accept the job, return a status resource, finish in the background.
- Bulk writes: cap the batch size, and choose all-or-nothing (one transaction) or partial success with a per-item result list (an overall 200 or 207 plus item-level statuses). Say which.
- Give each item its own idempotency key or client id so a retried batch does not double-apply.
- Large uploads bypass the API: hand out a pre-signed object-store URL, with multipart / resumable upload for big files.
Webhooks
A webhook is your API calling the customer's: an HTTP POST per event to a URL they registered. It replaces polling, and it makes you a client of an unreliable server you do not control.
| Concern | Design |
|---|---|
| Signing | HMAC-SHA256 over timestamp + "." + raw_body with a per-endpoint secret, sent in a header; receiver compares in constant time. Support two active secrets for rotation. |
| Replay protection | Reject timestamps outside a tolerance (about 5 minutes) and event ids already seen. |
| Retries | Exponential backoff with jitter over hours to days; then a dead-letter state, an alert, and auto-disable of endpoints that keep failing. |
| Ordering | Not guaranteed. Include a sequence number or updated_at; or send thin events ("transfer tr_9x changed") and let the receiver fetch current state. |
| Recovery | A GET /events?after= API so receivers can reconcile after an outage, plus manual redelivery. |
| Provider safety | Sender-side SSRF protection (block private address ranges), per-endpoint timeouts and concurrency so one slow customer cannot stall the rest. |
Rate limits and quotas
From the API's side, a limit is part of the contract: documented, visible in headers, and answered with a clear 429. The algorithms (token bucket, sliding window) are in Building Blocks.
| Rate limit | Quota | |
|---|---|---|
| Window | Seconds to minutes | Day or month |
| Purpose | Protect the service, smooth bursts | Enforce the plan, bill, cap cost |
| Key | API key, user, IP, endpoint | Account or tenant |
| On breach | 429 + Retry-After | 429 or 403 with an upgrade message |
- Limit on several dimensions: per key, per tenant, per endpoint class (a search costs more than a read), and a global ceiling.
- Header names vary:
X-RateLimit-*is the long-standing convention,RateLimit-*the IETF draft.Retry-Afteris the one clients actually obey. - 429 means "you sent too much"; 503 means "we are overloaded". Both are retryable, with backoff and jitter.
HTTP caching: ETag, Cache-Control, conditional requests
If-None-Match saves bytes on reads, If-Match prevents lost updates on writes.| Directive | Meaning | Use for |
|---|---|---|
max-age=60 | Fresh for 60 s in any cache | Public, slowly changing reads |
s-maxage=300 | Overrides max-age for shared caches (CDN) | Longer edge TTL than browser TTL |
private | Browser may cache, shared caches must not | Per-user responses |
no-cache | Store, but revalidate before every use | Must be current, cheap 304s |
no-store | Never store | Balances, tokens, PII |
stale-while-revalidate=30 | Serve stale while refreshing in the background | Hiding origin latency |
Failure mode: a shared cache storing an authenticated response and serving it to another user. Mark per-user responses private or no-store, and set Vary for anything that changes with a request header.
Compression and payload size
- Negotiate: the client sends
Accept-Encoding: br, gzip, the server answers withContent-Encoding. JSON typically shrinks by 70 to 90 percent. - Skip tiny bodies (under about 1 KB, where the overhead outweighs the gain) and already-compressed media. Compression costs CPU; do it at the edge where possible.
- Shrink before you compress: pagination, sparse fieldsets, no embedded blobs (return a URL), a binary format for internal hops.
- Cap sizes: a maximum request body (413 beyond it), a maximum page size, a maximum decompressed size to stop decompression bombs.
- On mobile networks the payload size often dominates latency; that is the argument for field selection and for HTTP/2 or 3 header compression.
API gateway and Backend-for-Frontend
A gateway is the single entry point that handles what every API needs. A Backend-for-Frontend (BFF) is a thin per-client API that shapes and aggregates data for one kind of front end.
| API gateway | BFF | |
|---|---|---|
| Does | TLS termination, authentication, rate limiting, routing, request ids, logging, canary routing | Aggregates several service calls, trims and reshapes payloads, handles client-specific flows |
| Owned by | Platform team | The front-end team it serves |
| Avoid | Business logic in the gateway; a single shared gateway config that every team must edit | Duplicated domain logic across BFFs; one BFF per screen |
Trade-off: the gateway is an extra hop (about a millisecond or a few) and a shared failure point, so it must be stateless, horizontally scaled and deployed per zone.
Contract-first design and SDKs
Write the contract (OpenAPI for REST, .proto for gRPC, the schema for GraphQL) before the code, review it like code, and generate server stubs, client SDKs, docs, mocks and a breaking-change check in CI from that one file.
Real-time communication
Part 4 of 6 · getting data to the client without being asked
HTTP is client-initiated. Every real-time technique is a way around that, trading simplicity for latency and connection state.
Polling, SSE, WebSocket and WebRTC compared
| Short polling | Long polling | Server-Sent Events | WebSocket | WebRTC | |
|---|---|---|---|---|---|
| Direction | Client pulls | Client pulls, server holds | Server to client | Both ways | Peer to peer, both ways |
| Transport | HTTP | HTTP | HTTP (one long response) | TCP after HTTP upgrade | UDP (SRTP, data channels) |
| Latency | Up to the poll interval | Near real time | Near real time | Lowest over TCP | Lowest (no in-order stall) |
| Server state | None | One held request per client | One open connection per client | One open connection per client | Signalling, STUN / TURN servers |
| Reconnect | Not needed | Built into the loop | Automatic, resumes with Last-Event-ID | Yours to build | Yours to build (ICE restart) |
| Proxies, firewalls | Works everywhere | Works everywhere | Works; needs buffering off | Mostly fine over TLS on 443 | Needs TURN relay when NAT blocks |
| Best for | Job status, low-frequency dashboards | Fallback, simple notifications | Feeds, tickers, progress, LLM token streams | Chat, collaboration, games | Voice, video, screen share |
Short polling and long polling
since cursor makes it lossless across reconnects.- Arithmetic: 1 million clients polling every 5 s is 200,000 requests per second of mostly empty responses. That number is why you move off short polling.
- Long polling needs a server that can hold many idle requests cheaply (async I/O), and a hold time below every proxy's idle timeout.
Server-Sent Events and WebSocket
- SSE first when only the server pushes: it is plain HTTP, so auth, gateways, HTTP/2 multiplexing and CDNs keep working. Client-to-server traffic uses normal POSTs. Over HTTP/1.1 browsers cap it at about 6 connections per origin.
- WebSocket when the client also sends often (typing, cursors, game input) or you need binary frames.
- Neither gives delivery guarantees. Add message ids, acks and resume-from-id at the application level.
Scaling WebSocket connections
The problem is not throughput, it is state: each user is pinned to one server, and a message for them can arrive at any other. Separate the stateful connection tier from stateless business logic and join them with pub-sub.
| Piece | What it does | Notes |
|---|---|---|
| Connection servers | Hold sockets, authenticate at handshake, forward frames. No business logic. | Memory-bound: order of 10 to 50 KB per idle connection, so roughly 100k to 1M connections per tuned server |
| Pub-sub backplane | Routes a message to the server holding the recipient | Per-server or per-user channels (Redis pub-sub, Kafka, NATS). Per-room fan-out for large rooms |
| Sticky routing | The connection itself is sticky (one TCP connection). Reconnects may land anywhere | So session state lives in the registry, not in server memory |
| Presence | user -> server, last_seen with a TTL refreshed by heartbeats | Online means "heartbeat within N seconds"; fan presence out lazily, it is very chatty |
| Heartbeats | Ping every 20 to 30 s, both to detect dead peers and to keep NAT and proxy idle timers open | Half-open TCP connections are otherwise invisible for minutes |
| Reconnect with resume | Client reconnects with the last message id (or sequence); server replays the gap from the store | Backoff with jitter, or a server restart causes a reconnect stampede |
Failure mode: deploying the connection tier drops every socket at once. Drain gradually (send "reconnect" hints in waves), and size the fleet for the reconnect storm, including the TLS handshakes and resume queries it triggers.
Push notifications
When the app is in the background there is no socket of yours to write to. The operating system keeps one shared connection to its vendor's push service; you ask that service to deliver.
- Device token: issued by the OS per app install, registered with your back end, can change; the provider's response reports invalid tokens, which the workers prune.
- Best effort: no delivery guarantee, no ordering; providers may throttle, collapse (collapse key) or drop expired messages. Payloads are small (about 4 KB).
- Design consequence: push is a hint ("you have new messages"), the app then syncs over your API. Never make push the source of truth.
- Dedupe per user and event, respect preferences and quiet hours, and rate limit per user before the queue.
Choosing among them
| Requirement | Choose | Because |
|---|---|---|
| Status of a job, updates every 10 s or slower are fine | Short polling | Stateless, cacheable, nothing new to operate |
| Server-to-client stream: feed, prices, progress, generated text | SSE | Plain HTTP, auto-reconnect and resume |
| Frequent traffic both ways: chat, typing, collaborative editing | WebSocket | Full duplex, low per-message overhead |
| Environments where SSE and WebSocket are blocked | Long polling as fallback | Looks like ordinary HTTP |
| Voice, video, low-latency peer data | WebRTC (signalling over WebSocket) | UDP media, tolerates loss instead of stalling |
| User is offline or app is in the background | Push via APNs / FCM | Only the OS can wake the device |
| Server-to-server notification | Webhook or message queue | No long-lived connection needed |
Senior signal: pick the simplest thing that meets the latency requirement, and state it as a number: "updates within 2 seconds" is SSE or polling, "under 100 ms both ways" is WebSocket.
Service-to-service communication
Part 5 of 6 · how services talk behind the gateway
Inside the system the choice is less about protocol and more about coupling: does the caller wait for the answer, and what happens to it when the callee is down?
Synchronous vs asynchronous
Service-to-service communication
| Synchronous request / response (REST, gRPC) | Asynchronous messaging (queue, log) | |
|---|---|---|
| Coupling | Temporal: both must be up at the same moment; caller knows the callee | Decoupled in time; with events, the producer does not know its consumers |
| Latency | Low and immediate; the caller's latency includes the callee's | Caller returns at once; end-to-end completion is later and variable |
| Failure behaviour | Failure propagates up the chain; availability multiplies (three 99.9% hops is about 99.7%) | Work waits in the queue; the consumer catches up after recovery |
| Load spikes | Hit the callee directly; needs load shedding | Absorbed by the queue; needs lag monitoring |
| Consistency | Immediate answer, read-your-write is easy | Eventual; needs idempotent consumers and often an outbox |
| Debugging | One trace, one stack | Needs correlation ids across hops, harder to follow |
| Use for | Queries, and commands whose result the user is waiting for | Side effects, fan-out, slow or bursty work, cross-team integration |
Interview tip: a long synchronous chain is a red flag. Ask of every hop: "does the user need this answer before we respond?" If not, publish an event.
Request-reply over a queue
Service-to-service communication
When you want the buffering of a queue but still need an answer: send the request with a correlation id and a reply address, and match the reply when it arrives.
- Use for slow, bursty or rate-limited back ends (pricing engines, document rendering) where you want backpressure and worker scaling.
- Avoid for low-latency lookups: you have rebuilt RPC with more moving parts. If the requester blocks waiting, you keep the temporal coupling and lose the simplicity.
Events vs commands
Service-to-service communication
| Command | Event | |
|---|---|---|
| Says | "Do this": ChargeCard | "This happened": PaymentCaptured |
| Tense, naming | Imperative | Past tense, a fact |
| Receivers | Exactly one owner | Zero to many subscribers |
| Can be rejected | Yes | No, it already happened |
| Who knows whom | Sender knows the receiver | Receivers know the producer's event |
| Channel | Queue (point to point) | Topic / log (publish-subscribe) |
Trade-off: events remove coupling but hide the flow: no single place shows "what happens when an order is placed". Choreography vs orchestration and sagas are in Architecture Styles and Distributed Systems Patterns.
Event schema and versioning
Service-to-service communication
- id for consumer dedupe (delivery is at-least-once); subject + sequence for per-entity ordering; occurred_at is event time, not processing time; trace_id ties the hops together.
- An event is a public API. Consumers you have never met depend on it, and old events sit in the log for days or forever, so readers must handle every version still retained.
- Evolve additively: new optional fields with defaults only. For a real break, publish a new type (
payment.captured.v2) alongside the old one and migrate consumers. - Schema registry with compatibility checks in CI (backward: new readers read old data; forward: old readers read new data) stops a producer deploy from breaking consumers.
- Thin vs fat: thin events (ids only) force a callback but are never stale; fat events (full state) decouple consumers but spread data, including PII, everywhere.
Service mesh and sidecars
Service-to-service communication
A mesh moves network concerns out of application code into a proxy deployed next to every service instance (the sidecar), configured centrally by a control plane.
| You get | How |
|---|---|
| mTLS everywhere | Short-lived workload certificates issued and rotated by the control plane; service identity, not IP addresses |
| Traffic policy | Timeouts, retries with budgets, circuit breaking, per-request load balancing, canary and traffic splitting |
| Observability | Uniform request metrics (rate, errors, latency), access logs and trace spans for every hop, in any language |
| Authorization | "Service A may call B's /transfers" as policy |
Trade-off: two extra proxy hops per call (typically around a millisecond in total, more at the tail), extra CPU and memory per instance, and a complex system to operate. Worth it with many services in several languages; overkill for five services where a shared library does the job. Sidecar-less (per-node) meshes reduce the overhead.
Timeouts, retries and deadlines
Service-to-service communication
One-line pointers; the full treatment is in Distributed Systems Patterns.
- Timeout: every network call has one, set from the callee's p99, not left at a library default of "forever".
- Retry: only idempotent calls, only on retryable errors, with exponential backoff, jitter and a retry budget; retry at one layer, not every layer (3 retries at 3 layers is 27 calls).
- Deadline: an absolute time passed down the chain so every hop knows the remaining budget and abandons work nobody is waiting for.
- Circuit breaker: stop calling a failing dependency and fail fast or degrade.
- Backpressure and load shedding: reject early (429 / 503) rather than queue without bound.
Serialization formats
Service-to-service communication
| JSON | Protocol Buffers | Avro | MessagePack | |
|---|---|---|---|---|
| Encoding | Text | Binary, tagged fields | Binary, no tags or names in the data | Binary JSON-like |
| Size (approx.) | Baseline (largest) | Often 2 to 5 times smaller | Smallest for records | Somewhat smaller than JSON; field names still included |
| Speed | Slowest to parse | Fast, generated code | Fast | Faster than JSON |
| Schema | Optional (JSON Schema) | Required .proto, codegen | Required; the reader needs the writer's schema (registry or file header) | None |
| Evolution | By convention only | Field numbers: add new, never reuse | Reader / writer schema resolution with defaults, checked by a registry | By convention only |
| Human-readable | Yes | No | No | No |
| Typical use | Public APIs, config, logs | gRPC, internal RPC | Kafka events, data lake files | Caches, compact internal payloads without codegen |
Interview tip: JSON at the edge for reach and debuggability, a schema-based binary format inside for cost and safety. Mention that JSON numbers lose precision above 253 in many parsers, so 64-bit ids and money travel as strings or integers in minor units.
Security on the wire
Part 6 of 6 · who is calling, what may they do, and who can listen
Security is rarely the main topic of a design interview, but one sentence per layer is expected, and a design with user_id in the request body gets marked down.
- Authentication vs authorization, sessions vs tokens, JWT
- OAuth 2.0 authorization code with PKCE, client credentials and OpenID Connect, API keys
- RBAC vs ABAC
- TLS and mTLS, encryption and key management
- Password storage, secrets management
- Attack classes and controls, input validation and least privilege
- Multi-tenancy isolation, audit logging and PII
Authentication vs authorization
Authentication (authn) answers "who are you?" and fails with 401. Authorization (authz) answers "may you do this to that object?" and fails with 403 (or 404 to hide existence).
- The gateway can check coarse rules (valid token, required scope). Only the owning service can check "does account 42 belong to this caller".
- Internal services must not trust a bare
X-User-Idheader: pass a signed token or rely on mTLS identity plus a propagated, verifiable user context.
Sessions vs tokens
| Server-side session | Self-contained token (JWT) | |
|---|---|---|
| Client holds | Opaque random id, usually in a cookie | Signed claims, in a header or cookie |
| Server state | Session store (Redis or database) | None; verify the signature locally |
| Per-request cost | One store lookup (sub-millisecond to about 1 ms) | CPU-only signature check |
| Revocation | Immediate: delete the row | Hard: valid until expiry unless you add a denylist |
| Size on the wire | About 32 bytes | Hundreds of bytes to a few KB, on every request |
| Cross-service use | Every service needs the store | Any service with the public key can verify |
| Main risk | CSRF (cookies are sent automatically) | Theft via XSS if kept in JavaScript-readable storage; stale claims |
| Fits | First-party web apps, when instant logout matters | APIs, mobile, microservices, third-party access |
Senior signal: the common hybrid: an opaque session or refresh token at the edge (revocable), exchanged for a short-lived JWT that internal services verify without a lookup.
JWT: anatomy, validation, expiry and revocation
- Validate, in order: algorithm against an allowlist (never trust
algfrom the token, rejectnone); signature with the key forkidfrom the issuer's JWKS (cached);iss;aud;expandnbfwith a small clock skew; then scopes. - Prefer asymmetric signing (RS256, ES256, EdDSA): services verify with a public key and cannot mint tokens. A shared HMAC secret lets every verifier forge.
- Keep PII and secrets out of the payload; keep it small.
| Revocation approach | Gain | Pay |
|---|---|---|
| Short expiry (5 to 15 min) + refresh token rotation | Stateless verification | Revocation lags by up to the access-token lifetime |
Denylist of jti or "tokens issued before T for user U", in a cache | Near-immediate logout | A lookup per request: state is back |
| Opaque tokens + introspection | Immediate, central control | A network call per request (cacheable for seconds) |
| Rotate the signing key | Kills everything at once | Logs out every user; emergency only |
OAuth 2.0 authorization code with PKCE
OAuth 2.0 is delegated authorization: a user lets an app call an API on their behalf without giving the app their password. Authorization code with PKCE is the flow for web, mobile and single-page apps.
challenge = BASE64URL(SHA256(code_verifier)). The code travels through the browser, the tokens never do. PKCE proves that whoever redeems the code is the app that started the flow.- PKCE (proof key for code exchange) defeats authorization-code interception: a stolen code is useless without the verifier. Required for public clients (mobile, SPA), recommended for all.
- state ties the callback to the request (CSRF protection); redirect URIs are matched exactly against a registered list.
- Scopes limit what the token can do; the API still enforces object-level authorization.
- The implicit flow and the password grant are deprecated: do not propose them.
Client credentials and OpenID Connect
When there is no user, a service authenticates as itself with the client credentials grant.
- Client authentication can be a secret, a signed assertion (
private_key_jwt) or mTLS; prefer the last two over long-lived shared secrets. - When a service acts for a user, propagate the user's context (token exchange or a signed internal token) so downstream authorization still sees the real principal.
- OpenID Connect in one line: an identity layer on OAuth 2.0 that adds an ID token (a JWT saying who logged in, for the client) next to the access token (for the API), which is what "Sign in with ..." uses.
API keys and when they are enough
An API key is a long random secret that identifies a calling application or account, not a user.
| Enough when | Not enough when |
|---|---|
| Server-to-server calls from a customer's back end | Acting on behalf of an end user (use OAuth) |
| Identifying a caller for quotas, billing and analytics | The key would ship in a browser or mobile app (it will be extracted) |
| Low-risk, read-mostly or public data | Fine-grained, short-lived or delegated permissions are needed |
- Send in a header, never in the URL (URLs end up in logs and referrers).
- Store only a hash; show the key once; give it a recognisable prefix (
sk_live_) so secret scanners can find leaks. - Scope keys (read-only, per environment), allow several active keys for rotation, support expiry and optional IP allowlists.
- For high-value calls, add request signing (HMAC over method, path, body and timestamp) so a captured request cannot be altered or replayed.
RBAC vs ABAC
| RBAC: role-based | ABAC: attribute-based | ReBAC: relationship-based | |
|---|---|---|---|
| Decision from | User -> roles -> permissions | Policy over attributes of user, resource, action, context | A graph of relations (owner, member, parent) |
| Example | "Tellers can view accounts" | "Tellers can view accounts in their branch, in business hours, under 1M" | "Editors of a folder can edit documents inside it" |
| Gain | Simple, auditable, easy to explain | Fine-grained, context-aware, no role explosion | Natural for sharing and hierarchies |
| Pay | Role explosion when rules depend on data | Harder to audit ("who can see X?"), policy engine needed | A graph store and careful consistency |
- Typical answer: RBAC for coarse permissions, plus an ownership / tenant check (an attribute) on every object. That hybrid covers most systems.
- Keep policy out of scattered
ifstatements: one policy decision point (library or service), enforced in each service, with decisions cached briefly. - Deny by default.
TLS and mTLS
TLS gives confidentiality, integrity and server authentication. Mutual TLS (mTLS) adds a client certificate, so each side proves its identity; it is the standard for service-to-service traffic.
- Ephemeral key exchange gives forward secrecy: stealing the server's key later does not decrypt recorded traffic.
- Where TLS ends matters: terminating at the load balancer leaves the internal hop in plain text. Re-encrypt to the back end, or use a mesh, for zero-trust networks.
- mTLS cost is operational: issuing, rotating and revoking certificates for every workload. Use short-lived certificates (hours to days) issued automatically.
- mTLS authenticates the service. It says nothing about the end user, so it complements tokens rather than replacing them.
Encryption and key management
| In transit | At rest | Application / field level | |
|---|---|---|---|
| Protects against | Network eavesdropping and tampering | Stolen disks, backups, snapshots | Database admins, SQL injection dumps, over-broad internal access |
| How | TLS 1.2+ on every hop, including internal | Disk, volume or database encryption (AES-256) | Encrypt selected fields (card number, national id) before storing |
| Does not protect | Data at the endpoints | A running system with valid access | The application holding the key |
| Pay | Handshake CPU, certificate operations | Almost nothing | No indexing or range queries on ciphertext; key handling in the app |
- Why envelope: bulk data never goes to the KMS (which is slow and rate limited); a DEK per object or tenant limits the blast radius; rotating the KEK means re-wrapping small DEKs, not re-encrypting terabytes.
- Key management rules: keys live in a KMS or HSM, never in code or next to the data; access is by IAM policy and audited; rotate on a schedule; separate keys per environment and, where needed, per tenant.
- Crypto-shredding: deleting a tenant's key makes all its data, including backups, unreadable. Useful for deletion requirements.
- Use authenticated encryption (AES-GCM or ChaCha20-Poly1305). Never invent a scheme.
Password storage
Passwords are never encrypted and never stored with a fast hash. Store the output of a salted, deliberately slow, memory-hard password hash.
- Algorithm: Argon2id preferred; bcrypt or scrypt are acceptable. Not MD5, SHA-1 or plain SHA-256: those run at billions of guesses per second on GPUs.
- Salt: unique and random per user, stored with the hash. It defeats rainbow tables and makes identical passwords look different.
- Work factor: tune to roughly 100 ms or more per hash on your hardware, and raise it over time. That cost is also why login needs its own rate limit.
- Pepper (optional): a secret kept in the KMS, applied to all hashes, so a database dump alone is not enough.
- Around it: rate limit and lock out per account and per IP, check against breached-password lists, offer MFA or passkeys, give identical responses for "no such user" and "wrong password".
Secrets management
- Where: a secrets manager or vault, fetched at start-up or injected at deploy time by workload identity. Not in source control, container images, or logs.
- Prefer no secret: workload identity (cloud IAM roles, mesh certificates) removes static credentials altogether.
- Short-lived and dynamic: database credentials issued per service instance with a TTL beat a shared password that never changes.
- Rotation without downtime: accept old and new together for a window (two active keys), then retire the old one. Design every credential for this from day one.
- Least privilege and audit: each service reads only its own secrets; every read is logged.
- Leak response: secret scanning in CI and on push, and a rehearsed revoke-and-rotate procedure. A leaked secret is rotated, not "removed from git history".
Attack classes and controls
| Attack | What happens | Control in the design |
|---|---|---|
| Injection (SQL, NoSQL, command) | Input is interpreted as code or query | Parameterised queries, no string-built queries or shell calls, schema validation, least-privilege database user |
| Broken object-level authorization (BOLA / IDOR) | Caller changes /accounts/42 to /accounts/43 and it works | Check ownership or tenant on every object access, in the data-access layer; non-guessable ids as defence in depth, never as the control |
| CSRF | Another site makes the user's browser send an authenticated request (cookies ride along) | SameSite cookies, anti-CSRF tokens, check Origin; not an issue for bearer tokens sent in a header |
| XSS | Attacker script runs in your page, acts as the user, steals tokens | Output encoding, Content-Security-Policy, HttpOnly cookies, no tokens in JavaScript-readable storage, correct Content-Type on API responses |
| SSRF | Server fetches an attacker-chosen URL (webhook, image import) and reaches internal services or cloud metadata | Allowlist destinations, resolve then block private and link-local ranges, no redirects, fetch from an isolated egress proxy |
| Replay | A captured valid request is sent again | TLS, plus timestamp + nonce in signed requests, idempotency keys, short token lifetimes |
| DDoS and resource exhaustion | Volume, or a few very expensive requests | CDN / scrubbing at the edge, rate limits and quotas, payload and page-size caps, timeouts, query cost limits, autoscaling with a ceiling |
| Credential stuffing | Leaked passwords replayed at login | Per-account and per-IP limits, MFA, breached-password checks, bot detection |
| Mass assignment | Client sets fields it should not (role, balance) | Explicit request schemas with an allowlist of writable fields |
Interview tip: broken object-level authorization is the top item in the OWASP API Security Top 10. Saying "every read and write is scoped by the caller's id from the token" is the one security sentence worth making in every design.
Input validation and least privilege
- Validate at the boundary against a schema: types, lengths, ranges, formats, enums. Allowlist what is valid instead of blocklisting what is bad; reject unknown fields on writes.
- Validate again where it matters: business invariants (balance never negative) live in the service and the database constraints, not only in the gateway or the client.
- Bound everything: body size, array length, string length, nesting depth, page size, request time.
- Least privilege: each service gets its own identity, its own database user limited to its tables, network policy limited to the services it calls, and tokens scoped to the operations it needs.
- Defence in depth: assume one layer fails. The gateway authenticates, the service authorizes, the database user cannot drop tables, the network blocks the rest.
Multi-tenancy isolation
The tenant id comes from the authenticated token, never from a request parameter, and is applied to every query, cache key, queue and object path by a shared layer (or database row-level security) rather than by each developer remembering a WHERE clause. Choose the isolation level by risk and cost: shared tables with a tenant column (cheapest, weakest), a schema or database per tenant, or a dedicated stack for the largest or most regulated tenants, with per-tenant rate limits and quotas so one noisy tenant cannot starve the rest.
Audit logging and PII handling
Audit log: an append-only record of security-relevant actions: who (user, service, on whose behalf), did what, to which object, when, from where, with what outcome, and the request id. It is separate from debug logs, write-once, access-controlled, and retained per policy.
| PII rule | In practice |
|---|---|
| Minimise | Do not collect or return what the use case does not need; keep PII out of URLs, events and tokens; pass ids, not copies |
| Mask | Redact in logs and traces at the logging layer; show partial values (last 4 digits); tokenise card numbers; field-level encryption for the most sensitive columns |
| Retain | A retention period per data class, enforced by TTL or scheduled deletion; backups and analytics copies included |
| Delete | A deletion or erasure workflow that reaches every store, cache, search index and downstream consumer; crypto-shredding for backups |
| Control access | Role-gated, purpose-bound, and itself audited ("who looked at this customer?") |
| Locate | Know which region the data lives in; residency rules can force per-region storage and routing |
Failure mode: the database is locked down, but the same PII sits in application logs, an analytics topic and a search index with broad access. Deletion and masking must follow the data, which is the strongest argument for thin events and central PII storage.
Cheat sheet: need, choice, reason
| Need | Choice | Reason |
|---|---|---|
| Public or partner API | REST + JSON + OpenAPI | Universal reach, cacheable, easy to debug |
| Internal service calls | gRPC + Protobuf | Typed contracts, small payloads, streaming, deadlines |
| Many clients, different views of one graph | GraphQL or a BFF per client | Client-shaped responses in one round trip |
| Large or changing list | Cursor / keyset pagination | Constant cost per page, stable under inserts |
| Retry-safe create or payment | Idempotency key, stored atomically with the write | A timeout cannot cause a duplicate |
| Concurrent edits | ETag + If-Match, 412 on conflict | No lost updates, no locks held |
| Work longer than a few seconds | 202 + job resource; webhook when done | No held connections, progress is visible |
| Notify another company's system | Signed webhook, at-least-once, event id | No polling; receiver dedupes |
| Server-to-client stream | SSE | Plain HTTP, auto-reconnect and resume |
| Two-way real time | WebSocket + pub-sub backplane | Full duplex; backplane routes across servers |
| App in background | APNs / FCM push, then sync | Only the OS can wake the device |
| Decouple side effects | Events on a topic | Producer does not wait for or know consumers |
| Evolve without breaking | Additive changes, tolerant readers, /v1 for real breaks | Clients upgrade on their own schedule |
| User login for an app | OAuth 2.0 authorization code + PKCE, OIDC | No password in the app, short-lived scoped tokens |
| Service identity | mTLS, or client credentials | Mutual authentication, no shared static secrets |
| Instant logout | Server session, or short JWT + denylist | Pure JWT cannot be revoked before expiry |
| Protect sensitive fields | Envelope encryption with a KMS | Small blast radius, cheap rotation |
| Abuse and overload | Rate limits + quotas, 429 with Retry-After | Predictable contract, protects the back end |
API review checklist
- Shape: resources are nouns, methods carry the verbs, ids are opaque, one casing, UTC timestamps, money in minor units.
- Identity: caller and tenant come from the token; every object access is authorized for that caller.
- Writes: each write is idempotent by nature or by key; behaviour on timeout and retry is defined.
- Reads: lists are paginated with a cursor and a server-side maximum; filters and sorts are allowlisted and indexed.
- Errors: correct status class, one error format, stable machine codes, request id, no internals leaked.
- Slow work: anything over a few seconds is a job with a status resource.
- Limits: rate limits, quotas, payload and page caps, timeouts, all documented and visible in headers.
- Caching: each GET is deliberately cacheable, private or no-store; ETags where clients revalidate or edit.
- Evolution: changes are additive; versioning and deprecation policy are stated; the contract is the source of truth.
- Real time: the push channel is the simplest that meets the latency number, with reconnect and resume.
- Between services: timeouts and deadlines everywhere, retries bounded, async where the user is not waiting, events versioned.
- Security: TLS on every hop, least-privilege scopes, input validated against a schema, secrets in a manager, PII minimised and masked, sensitive actions audited.
Next: how these services are arranged into a whole system, in Architecture Styles.