Backend System Design

They differ in who fills the cache and when the database is written. Cache-aside and read-through fill the cache on a miss; write-through writes cache and database together; write-behind writes the cache now and the database later, trading durability for write speed. Cache-aside: read the cache, on a miss load the database and store the result with a TTL (give every key one); on a write update the database and delete the key. The default: the cache stays optional. Read-through: the same flow inside a cache library with a loader function, so callers only ask the cache. Write-through: every write goes to the database and the cache in one path, so the next read is hot, but every write now depends on Redis. Write-behind: the write lands in the cache and is flushed in batches. Fine for view counters; a crash before the flush loses writes the client was told succeeded, so never for orders or money.

Strong consistency: every read sees the latest committed write. Eventual: replicas converge once writes stop, so reads can be stale meanwhile. CAP: during a network partition you keep either consistency or availability. PACELC adds that even without a partition you trade latency for consistency. CAP in practice: partitions happen whether you like it or not, so the real choice is CP or AP. CP refuses requests that cannot reach a majority (etcd, a Postgres primary with synchronous standbys); AP answers from any reachable replica and reconciles later (Cassandra at consistency level ONE). PACELC: the everyday trade-off: a synchronous replica or a quorum read costs a round trip on every request, even when nothing is broken. Choose per piece of data, not per system: an account balance, stock count or unique username needs a strong read from the primary; a news feed, like counter or search index can lag by seconds. Guarantees between the two: read-your-writes (users see their own change), monotonic reads (time never goes backwards for one user), causal (a reply is never visible before its post). Often what the product needs, and far cheaper than strong consistency everywhere. Put a bound on "eventually" ("search reflects a change within 5 seconds") and monitor it through replication and consumer lag.

Consistent hashing puts nodes and keys on one hash ring, and a key belongs to the first node clockwise from it, so adding an eleventh node moves about 1/11 of the keys. With hash(key) % N the owner depends on N itself: going from 10 to 11 nodes moves about 91% of them. Why modulo fails: a key stays put only if h % 10 == h % 11, true for about 1 key in 11. A cache cluster that grows or loses a node misses on nearly everything at once, and the database takes the whole read load. The ring: every node is hashed onto the same 64-bit circle. A new node takes over only the arc just before it; a removed node's arc passes to its successor. In a simulation with 200,000 keys, an 11th node moved about 9% of them. Virtual nodes: one point per node gives very uneven arcs: in the same simulation the busiest of 10 nodes typically held 2 to 3 times its share. 100 to 200 points per node bring every node within roughly 10 to 20% of the average and spread a dead node's keys over all the survivors. Alternatives: rendezvous hashing (score each node with hash(key, node), take the highest), jump consistent hash for numbered shards, or fixed slots mapped to nodes by a table, as in Redis Cluster. Where you meet it: memcached clients (ketama), Cassandra's token ring, Envoy's RING_HASH and MAGLEV balancers for sticky routing to a cache, groupcache. What it does not fix: moved keys start cold, a hot key still lives on one node, and a database, unlike a cache, has to copy the moved data.

Fixed window counts requests per clock interval: cheapest, but it lets a double burst through at the boundary. Sliding windows fix that with more memory or an approximation; a token bucket allows a set burst, then a steady rate; a leaky bucket queues requests and lets them out at a constant rate. Fixed window: one counter per key per minute (INCR plus EXPIRE). A limit of 100 a minute lets 100 through at 12:00:59 and 100 more at 12:01:00: 200 in two seconds. Sliding window log: store the timestamp of every request (a Redis sorted set), drop the ones older than the window, count the rest. Exact, but memory grows with the limit: 10,000 an hour means up to 10,000 entries per client. Sliding window counter: weight the previous window by how much of it still overlaps: prev*(1 - elapsed/window) + curr. Two counters per key, accurate to a few percent under steady traffic. Token bucket: refill r tokens a second up to b; each request takes one. Two numbers per key and an explicit burst; this is what golang.org/x/time/rate implements and what most API limits mean; the default for a public API. Leaky bucket: requests enter a queue that drains at a fixed rate. It smooths the output to a fragile downstream (a provider that allows 50 calls a second) at the cost of latency, and drops requests when the queue is full.

Backpressure is the slow side making the fast side slow down instead of buffering without limit. In a process it is a bounded channel or semaphore that blocks or rejects; across a broker it is a capped backlog watched through consumer lag, with producers told 429 or 503 when it is full. Unbounded buffers only delay the failure: a go handle(job) per request or a million-slot channel turns a slow consumer into an OOM kill that loses everything buffered. Block or reject: a blocking send passes the slowdown upstream, which is right between pipeline stages. At the edge of a service, reject at once: a blocked handler still holds a connection and a goroutine while the client's timer runs out. A queue is a buffer too: Kafka keeps messages on disk, and a lagging consumer never slows producers (only quotas and a full producer buffer do), so you watch consumer lag (messages or seconds behind) and act before retention deletes unread data. RabbitMQ throttles publishers under memory pressure and can cap a queue with x-max-length and x-overflow: reject-publish. Then add capacity where it helps: more partitions and consumers, batched database writes, more worker concurrency, up to what the downstream database can take. Shed by priority: when something must be dropped, drop the least valuable work first (analytics before payments), and make clients retry with backoff.

Sort by the column plus a unique tie-breaker and continue from the last row seen with a row comparison: WHERE (created_at, id) < ($1, $2) ORDER BY created_at DESC, id DESC. Every page is an index range scan however deep you go; you give up jumping to page N and a cheap total count. Why the tie-breaker: with created_at alone, rows that share the boundary timestamp are skipped or repeated. Adding id makes the order total. The index must match the order: (created_at DESC, id DESC), the same columns and directions. A row comparison only works when all columns sort the same way; for mixed directions expand it to a < $1 OR (a = $1 AND b > $2). Opaque cursor: base64 of the last row's sort values. Clients never build it, so you can change what is inside later. Validate it on decode, and tie the filters and the sort order to it so a cursor cannot be replayed against a different query. **limit+1**: fetch one extra row to learn whether a next page exists without a COUNT(*). What you lose: no "page 37 of 120", and an exact total is a separate, expensive COUNT; show "load more" or an estimate. OFFSET stays fine for small admin tables. What you gain: stability under writes: a new post does not shift the next page and repeat an item, as it does with OFFSET 40.

A bigint identity is 8 bytes and always inserts at the right edge of the index. A random UUIDv4 (16 bytes) lands on a random leaf page, so the index fragments and stops fitting in memory. UUIDv7 starts with a timestamp, so it inserts almost like a sequence yet needs no database to create it. **bigint GENERATED ALWAYS AS IDENTITY**: the smallest key, repeated in every foreign key and index. Only the database hands it out, so the client learns the id after the insert, merged databases collide, and /orders/10452 reveals your volume. UUIDv4: created anywhere (in Go before the insert, offline on a phone) and unguessable, but each insert hits a random leaf: pages split and stay part-empty, more full-page images go to the WAL, and once the index outgrows memory inserts become disk reads. With 2 million rows on Postgres 18 the v4 index was 76 MB with leaves 71% full; v7 was 60 MB at 90%, bigint 43 MB. UUIDv7 (RFC 9562): a 48-bit millisecond timestamp, then random bits: uuidv7() in Postgres 18, uuid.NewV7() in github.com/google/uuid. Anyone can read the creation time from it. **Store it as uuid** (16 bytes), never as text (37 bytes, slower comparisons). Snowflake-style ids (64 bits of timestamp, machine id and sequence): time-ordered like v7 in half the space, which matters across billions of rows and every index. The price: a unique machine id per generator and a clock that never steps back. A common split: a bigint key for joins inside the database and a UUIDv7 public id in URLs; or UUIDv7 everywhere when ids are created outside the database.

The check and the insert are separate steps, so two requests with the same key both find nothing and both charge; the key row is written only after the charge. The key has to be claimed atomically, before any side effect happens. Check-then-act race: SELECT returns sql.ErrNoRows to both requests; the second INSERT then fails on the primary key, but the card has already been charged twice. A second bug: any SELECT error other than "no rows" (a timeout, a dropped connection) is treated as "never seen" and charges again. Claim, act, record: INSERT ... ON CONFLICT DO NOTHING with status pending: exactly one request sees one row affected and goes on. The others replay the stored response, get 409 while it is still pending, or 422 if the same key arrives with a different body (compare a request hash). Pass the key to the provider: the card processor's own idempotency key closes the gap where the process dies after charging and before saving the response. Recovery: a crash leaves the row pending; a reaper re-runs rows older than a minute with the same provider key, so the client is not stuck on 409 forever. Retention: keep keys at least as long as clients retry (Stripe prunes them after 24 hours), then delete them in batches.

Serial dependencies multiply: 0.999⁴ ≈ 99.6%, about 2.9 hours of downtime a month before any bug of your own. To promise more, make dependencies optional or redundant (a cache, a fallback, async work, a second replica) so their failure stops being yours. Serial: availability is the product of everything on the request path: the load balancer, your code, the database, every hard dependency. Ten hops at 99.9% cap you at about 99%. Parallel: two replicas at 99% each give 1 - 0.01² = 99.99%, but only if they fail independently. A shared database, zone or config push makes their failures correlated, and the math no longer holds. Soft dependencies: with a timeout and a fallback (a cached price, default recommendations, "reviews unavailable") a dependency drops out of the equation. Set the timeout well below your own deadline, so a slow dependency cannot eat the whole budget, and a circuit breaker so a dead one is not retried on every request. Ask of every call: does the user's request fail when this fails? Async: work moved off the request path (the email goes through a queue) turns the dependency's outage into a delay, not an error.

Add a small in-process cache when Redis round trips or a few hot keys dominate: it answers in nanoseconds and spreads a hot key's load across pods. The price is 40 independent copies, so keep their TTL short, broadcast invalidations and accept seconds of staleness. Hot key: one Redis key read 200,000 times a second lands on one shard, a single CPU core, however big the cluster is. A local copy per pod, or the key split into key#0..key#7 replicas read at random, spreads it. Bounded size: a plain map with no eviction is a memory leak. Use an LRU or a cost-bounded cache (ristretto, otter) and keep only small, hot, rarely changing values: config, feature flags, product cards. Invalidation: on a write, publish the key on a Redis Pub/Sub channel and every pod drops it. Pub/Sub is fire-and-forget: a pod that was reconnecting misses messages, so a short local TTL (5–30 s) is the backstop, not an option. Redis client-side caching: since Redis 6, CLIENT TRACKING makes the server send invalidations for keys a connection has read; rueidis uses it out of the box. Misses per pod: a singleflight per key on a local miss, so 40 pods send at most 40 loads, not thousands. Not for read-your-writes data: the user who just changed something may land on a pod with the old copy, unless the writing pod updates its own copy and requests stick to it.

Delete-on-write still loses a race: a reader misses, loads the old row, the writer commits and deletes the key, then the reader stores the old value, which then lives for the full TTL. Replica lag and lost deletes widen the window. Where stale values come from Read-then-set race: a load slower than the write; the late SET wins. Rare per key, certain at scale. Replica lag: the miss path reads a replica that has not applied the write yet, so the refill after the delete is stale. Lost delete: a Redis timeout or a crash between the commit and DEL, and nothing retries it. Delete before commit: deleting inside the transaction lets a reader refill from the old committed row before the commit lands. Fixes, cheapest first TTL as the bound: every key has one, and it caps how long any of these bugs lives. A TTL of hours means stale data for hours. Delayed double delete: delete after the commit and again about a second later from a queue job, covering the reader that was in flight. A cheap patch, not a guarantee. Versioned writes: instead of deleting, the writer stores the new value with its row version, and every fill goes through a Lua script that writes only if its version is higher than the cached one. The late reader's old version is refused. Invalidation from the change stream: a consumer of the database's changes (Debezium, or the outbox) deletes keys after every committed change and retries on failure. A lost DEL gets retried, and writes from services that never heard of the cache invalidate it too. Refill from the primary for data where staleness costs money, or do not cache it at all.

Send a user's reads to the primary only while their own write may not have reached the replica: remember the write's position (or time) for that user, and use a replica only once it has replayed that position. Everyone else keeps reading replicas. Time window: the simplest: after a write, set a wrote_at cookie and read from the primary for N seconds, above the usual lag. It breaks when lag spikes past N. WAL position: return pg_current_wal_lsn() after the write; before reading a replica, check pg_last_wal_replay_lsn() >= $lsn and fall back to the primary if it is behind. Exact, for one cheap query (or cache the replica's position for a few milliseconds). Where the token lives: a cookie or header the client sends back works across pods; server memory does not, because the next request lands on another pod. Caches count too: a cache refilled from a replica brings the problem back; on a write, delete the user's keys and refill from the primary. Monotonic reads: pinning a user to one replica stops them seeing a newer value and then an older one when two replicas lag by different amounts. Across devices: a cookie lives in one browser. For "edit on the phone, check on the laptop", keep the position per user on the server (in Redis), or always read a user's own profile from the primary.

Set the timeout from the dependency's measured latency (a little above its p99 and inside the caller's remaining deadline), not a round number. Hedging sends a second copy of a slow idempotent read after about the p95, so one straggling replica no longer sets your p99. From data: if p99 is 80 ms and p99.9 is 200 ms, 250 ms cuts off only real hangs. A 30 s default lets a stuck dependency hold your goroutines and connections for 30 s each. A budget, not a number per hop: the edge sets the total deadline; each hop uses the smaller of its own timeout and what is left in ctx, leaving room for its own work and one retry. Fan-out amplifies the tail: a request that waits for 100 shards, each slow 1% of the time, is slow 63% of the time (1 - 0.99¹⁰⁰). This is where hedging pays. Hedging rules: idempotent reads only; send the copy to another replica; cap hedges at a few percent of traffic, or a sick dependency doubles its own load; cancel the losing attempt. Outside the code: Envoy can hedge on a per-try timeout (hedge_on_per_try_timeout), so the policy lives in the mesh instead of every client.

Move to CDC when polling's latency or database load starts to hurt, or when you need every change rather than only the events the code remembers to write. The price is a new component reading the WAL, and a stalled replication slot makes Postgres keep WAL until the disk fills. Polling relay Simple: SELECT ... FOR UPDATE SKIP LOCKED LIMIT 100, publish, mark (all Go code you own). Latency is the poll interval, and at high volume the constant updates and deletes create dead tuples and vacuum work. Ordering needs care: several relay workers with SKIP LOCKED can publish two events of one order out of order. Partition the relay by aggregate id, or run one active relay. CDC The connector reads committed changes from the WAL in commit order, with near-zero latency and no polling queries. Debezium's outbox event router turns outbox inserts into events; the outbox rows can be deleted right after insert, because the WAL already holds them. Replication slot: Postgres keeps WAL until the slot's consumer confirms it. A stopped connector means a growing pg_wal and in the end a full disk on the primary. Alert on slot lag in pg_replication_slots, and cap it with max_slot_wal_keep_size; past the cap the slot is invalidated and the connector needs a new snapshot. Failover: before Postgres 17 a logical slot did not survive promotion of a standby; Postgres 17 adds failover slots: create the slot with failover = true and set sync_replication_slots = on on the standby, which also needs primary_slot_name, hot_standby_feedback = on and a dbname in primary_conninfo. Naming that physical slot in synchronized_standby_slots on the primary is strongly recommended, so the connector cannot run ahead of the standby; the connector must also use such a slot. Delivery: still at least once: after a restart the connector replays from its last stored offset, so consumers dedupe by event id exactly as before. Schema coupling: CDC on raw tables turns internal columns into a public contract; publishing only the outbox table keeps the event schema deliberate.

Make the check and the decrement one atomic statement (UPDATE ... SET stock = stock - $2 WHERE stock >= $2), so overselling is impossible. Then keep the crowd away from that one hot row: admit buyers through Redis, sell reservations with an expiry, and take payment afterwards. Why read-then-write oversells: under Read Committed two transactions read stock = 1, both pass the check, both write 0. The conditional UPDATE re-checks stock after it gets the row lock, so the second one sees 0 and updates no rows. The hot row: every buyer waits on the same row lock, one at a time, and each waiter holds a pooled connection. 50,000 of them exhaust the pool long before stock runs out. Redis gate: DECR stock:{sku} is atomic: a result of 0 or more means admitted, a negative one means sold out, so the other 49,900 get "sold out" without touching the database. The database's conditional update stays the source of truth, because Redis can lose writes on failover. Reservations, not sales: an admitted buyer gets a reservation that expires in 10 minutes and pays after that. An expired or failed payment returns the unit to both counters. Split the row: for large stock without a gate, spread it over N bucket rows and decrement a random non-empty one, so buyers contend on N locks instead of one. Serializable is not the answer: it also prevents the oversell, but aborts most transactions with 40001 under this contention.

Record every movement as an immutable transfer whose entries sum to zero (minus on one account, plus on the other), written in one database transaction with the cached balances. A mistake is fixed by a reversing transfer, never by an edit, and reconciliation checks that the sums still hold. Model Amounts: signed integers in minor units (cents, tiyn), with the currency on the account; never float64. An exchange is two transfers through a conversion account. Money from outside is an account too: a card top-up debits psp:incoming and credits the user, so all accounts together always sum to zero. System accounts may go negative, customer accounts may not. Append-only: the service's database role gets no UPDATE or DELETE on entries. The history is the audit trail, and a past balance is the sum up to that moment. Balances Summing every entry on each read gets slow, so accounts.balance is updated in the same transaction, with balance >= $amount in the WHERE as the overdraft check. Lock both accounts in id order, as in the deadlock question. A merchant receiving thousands of payments a second turns its balance row into a hot spot: split it into sub-accounts whose sum is the balance, or credit it asynchronously, since an incoming credit needs no overdraft check. Idempotency and the outside world The transfer row holds the request's idempotency key under a unique constraint, so a retry finds the first transfer (the claim-first pattern from the payment question). Never call a bank or a card provider inside the ledger transaction. Post a hold to a pending account, call the provider, then settle or release the hold when it answers. Each step is idempotent, and a stuck hold shows up in a report. Reconciliation Internal and continuous: every transfer sums to zero, every cached balance equals the sum of its entries, all accounts together sum to zero. External and daily: match the ledger against the bank's and the provider's settlement files. A mismatch becomes a case for a person, never an automatic fix.

Allocate a unique number from a counter (a database sequence or ID ranges per app instance), encode it in base62, store code → URL keyed by the code and serve redirects from a cache: reads outnumber writes about 100 to 1. Most of the design is ID generation and the read path. Estimates 100 million links a month is about 40 writes a second on average (about 400 at peak) and, at 100:1, about 4,000 redirects a second on average (about 40,000 at peak). 500 bytes a row × 1.2 billion links a year ≈ 600 GB a year: one partitioned Postgres or a key-value store holds years of it. 62⁷ ≈ 3.5 trillion codes: 7 characters last for centuries at this rate. API and IDs POST /links {url} → 201 {code}; GET /{code} → a redirect with Location. Use 302 if you count clicks, since browsers cache a 301 and stop asking. IDs from a database sequence, or ranges of it handed to each instance, encoded in base62. A raw Snowflake ID is 64 bits and encodes to about 11 characters; for 7-character codes use a counter below 62⁷, or a random 7-character code with a unique constraint and retry. Sequential codes are guessable; if that matters, shuffle the counter through a bijection. Hashing the URL (first 7 characters of a digest) brings collisions to handle, and makes one URL always map to one code, which may or may not be wanted. Read path Cache code → URL in Redis, and in process for the hottest links; most traffic hits a small set of codes. A miss is a primary-key lookup. Cache "not found" briefly, so scans of random codes do not reach the database. Count clicks through a queue and aggregate them asynchronously, never with an UPDATE per redirect. Worth naming Custom aliases (a unique constraint), expiry (a column plus a cleanup job), abuse: rate-limit link creation per user and check URLs against malware lists.

Never one UPDATE per view. Each API pod adds views up in memory and flushes the deltas every second to Redis or to a log such as Kafka, and a job folds the totals into Postgres in batches. A view count may be a little late and a little approximate, and that is what makes 50,000 a second cheap. Why the obvious design fails UPDATE videos SET views = views + 1 takes the row lock, so a viral video's increments queue on one row, each with its own WAL record and dead tuple, while the waiters hold pooled connections. One hot row tops out far below 50,000 updates a second. Counting Aggregate in the process: a map[videoID]int64 per pod, flushed by a ticker. With 20 pods a viral video costs 20 writes a second instead of 50,000. Flush target: one Redis pipeline of INCRBY per flush, or one message per video per flush on a topic keyed by video id, which a consumer sums. Redis answers "how many views now"; at extreme rates split one video's key into a few subkeys and add them up on read. Persist in batches: every 30 to 60 s a job copies the totals into Postgres in one statement (UPDATE videos SET views = GREATEST(views, t.n) FROM (VALUES ...) AS t(id, n) WHERE videos.id = t.id). Totals rather than deltas make the job safe to repeat, and a Redis that lost its last second cannot move the stored count backwards. What you can lose or double A crashed pod loses its unflushed second; flush on SIGTERM, so a deploy loses nothing. A Redis failover can drop the last acknowledged increments. State the budget ("at most about a second of views per failure"); the product usually accepts it. A flush retried after a timeout may count twice. If that matters, tag each flush with the pod and a sequence number and skip ones already seen. Unique viewers need a different counter: a HyperLogLog (PFADD / PFCOUNT) counts distinct users in 12 KB per key with about 0.81% error, where a set grows with every viewer. Bots and refreshes: count a view once per user per video per time window, or after a few seconds of playback, before it reaches the counter.

An API that validates and enqueues, a router that applies preferences and templates, and per-channel queues drained by workers that call the providers within their rate limits. Delivery is at least once, so every message carries an idempotency key and is deduped before sending. Flow Other services call POST /notifications or publish an event: user, type, data, idempotency key. The API stores it and answers 202. A router resolves preferences, opt-outs and quiet hours, renders the template in the user's language, and writes one message per channel to the email, sms and push topics. Channel workers call the providers (an email API, an SMS gateway, APNs and FCM) with a concurrency limit each. Reliability Dedupe: insert a unique (idempotency_key, channel) row before sending, so a redelivered message is not sent twice; pass the key to the provider where it accepts one. Retries: transient provider errors retry with backoff through delay topics; permanent ones (an invalid device token, a hard bounce) disable the address and stop. After N attempts the message goes to a DLQ with a replay tool. Isolation: separate queues per channel and per priority: a 10-million-email marketing blast must not delay a login code sent by SMS. Scale and limits Provider limits: a token bucket per provider in the workers; when it is empty, messages wait in the queue instead of failing. User limits: at most N pushes an hour per user, and duplicates collapsed ("3 new comments"). Bulk sends: a scheduler expands a campaign into batches, instead of the caller sending one request per user. Status: track each message (queued, sent, delivered, failed) from provider webhooks, for support and analytics.

Clients keep a WebSocket to a gateway tier. A message is stored first, with the next sequence number of its conversation, then pushed to the recipients' gateways via a presence registry; offline devices get a push and, on reconnect, fetch everything after the last number they have. Sending and ordering The client sends conversation_id, a client_msg_id it generated (a UUID) and the text. The server increments the conversation's last_seq, inserts the message with that number in the same transaction and acks with seq. Order comes from seq, not timestamps: device and server clocks disagree, and two messages in one millisecond still need an order. Only order within a conversation is needed, never a global one. A resend after a lost ack hits the unique (conversation_id, client_msg_id): the transaction rolls back without using up a number, and the server acks with the stored seq. Delivery Online: a presence registry (user to gateway nodes, one entry per device, refreshed by heartbeats with a TTL) says where each recipient's sockets are; the chat service sends the message to those gateways. Storage first: the real-time path may drop messages, because nothing lives only there. A client that sees 41 and then 43 asks for 42; on reconnect it sends its last seq per conversation and pulls the gap. Offline: devices without a socket get an APNs or FCM push. Gateways hold state: drain them gradually on a deploy, or a million clients reconnect at once. Receipts and groups Store one last_read_seq per member, not a row per message per reader. "Read" means seq <= last_read_seq, unread is last_seq - last_read_seq, and the value never moves backwards. A channel with 100,000 members gets a light "new messages" signal instead of every message, and open clients pull. Storage The hot queries are "the last 50 messages" and "everything after seq N", so messages are keyed by (conversation_id, seq). Postgres with partitions goes a long way; at Discord's scale this became a wide-column store partitioned by channel and time bucket. Media goes to object storage, and the message keeps a link.

Fan-out on write copies each new post id into every follower's stored feed, so a read is one lookup; fan-out on read merges the latest posts of everyone the user follows at request time. Real systems mix them: write fan-out for most authors, a read-time merge for the few with millions of followers. Fan-out on write A worker reads the author's followers and prepends the post id to each follower's feed: a Redis list or sorted set trimmed to the newest 500 to 800 ids, or a feed table. A read is one lookup plus a batch fetch of the posts. It runs from a queue after the post is committed, so posting returns at once; authors see their own post because their posts are merged in at read time. The cost is on the write. 300 followers means 300 small writes. Ten million followers means ten million writes per post, minutes before the last follower sees it, and most of them land in feeds of people who will not open the app this week. Fan-out on read On request, fetch the recent posts of every followed account (each author's timeline is cached) and merge them by time. Posting costs nothing extra. Reads are expensive: following 2,000 accounts means 2,000 timeline reads every time the feed opens, and feed opens outnumber posts by orders of magnitude. The hybrid Fan out on write for authors below a follower threshold; for celebrities skip the fan-out and merge their recent posts in when a follower reads. Twitter's home timeline worked this way. Fan out only to users active in the last few weeks; a returning user's feed is rebuilt by pull. Feeds store ids, not post bodies: an edit or a deletion changes one post in the post cache instead of millions of feed copies, and deleted ids are filtered out at read time. Page with a cursor on the post id or score, never OFFSET: new posts arrive at the top while the user scrolls. A ranked feed adds a scoring step over the same candidates; the storage design stays the same.

Keep a token bucket per API key in Redis and update it with one Lua script, so the check and the update are atomic across every replica and cost one round trip. Bound the Redis call with a short timeout, and on failure fall back to a local limiter at each pod's share instead of dropping the limit. Design Check in middleware or at the gateway, before any work is done; over the limit, answer 429 with Retry-After. The script reads the tokens and the last refill time, adds elapsed × rate, caps at the burst, takes one token and writes back with an expiry. It uses Redis's TIME rather than each pod's clock, so clock skew between pods does not matter. All of one key's state lives in one Redis key, so in Redis Cluster the script never crosses hash slots. Cost and scale One round trip per request, well under a millisecond in the same zone; at 100,000 requests a second that is a Redis cluster to size and monitor. A huge customer is a hot key on one shard. Taking tokens in batches (a pod reserves 10 and spends them locally) cuts round trips at the cost of some precision. Failure A Redis timeout of 10–20 ms. On an error, fail open but bounded: an in-process rate.Limiter per key at limit / replicas. Failing closed turns a Redis outage into a full API outage; failing fully open removes the protection exactly when something is already wrong. Limits that guard against abuse, such as login attempts, may fail closed instead.