Agent Memory: Building Memory-Aware Agents

Presented by Oracle

Free short course on DeepLearning. By the end, you’ll have assembled a fully stateful Memory Aware Agent that loads prior context at startup, assembles relevant context, state, tools, and outputs and improves across sessions.

Start Building

How Database Indexing Really Works

Slow queries have a reflex fix: add an index.

It works often enough that teams keep doing it; until write latency creeps up, storage balloons, and the query planner starts making strange choices.

That’s the index tax. Real, measurable, and entirely predictable once you understand what an index actually costs.

The goal isn’t to index everything. It’s to index the right things, and know when to stop.

What an index really does

At the simplest level, an index maps a search key to where the row lives, much like a book index maps a chapter to page numbers.

The goal is to cut down how much of the table the engine must look through to answer a query.

That helps in a few common cases:

• Point lookups → WHERE id = 42 becomes a targeted search instead of a full scan.

• Range queries → BETWEEN, <, and > work well when keys stay in order.

• Joins → An index on join keys helps the engine find matching rows faster.

• Ordering → A tree index can return rows in sorted order without an extra sort step.

• Uniqueness → Unique indexes help enforce “this value must appear only once.”

The catch is that every INSERT, UPDATE, and DELETE must also update the index. So every helpful index also comes with a write tax. Add too many, and your database starts carrying extra baggage on every row change.

Index types

Not all indexes work the same way. Each structure solves a different query pattern.

Most relational databases use B-tree style indexes by default, often implemented internally as B+ trees.

B-trees are the safe first choice for most OLTP queries:

• Equality filters → Great for exact matches.

• Range filters → Great for time windows and ordered values.

• Ordered reads → Helpful for ORDER BY.

• General-purpose use → Good default when your data is sortable and your workload is mixed.

B-trees are the workhorse, while other index families exist for more specialized conditions and data shapes.

How the optimizer decides

This is what catches most engineers: just because the index exists, doesn’t mean it’ll be used.

The optimizer decides to use an index when its cost model estimates that the index path is cheaper than reading the table directly.

That decision depends heavily on statistics: row counts, value distribution, selectivity, and sometimes histograms. If those statistics are stale, the optimizer can make a bad decision.

This is why keeping statistics fresh matters.

PostgreSQL uses ANALYZE to populate pg_statistic. MySQL manages histograms via ANALYZE TABLE. If statistics are stale, the optimizer can pick a costly plan even when a good index exists.

The optimizer assigns a cost to each candidate plan and picks the cheapest. In PostgreSQL, you can nudge this by tuning random_page_cost relative to seq_page_cost; lowering the ratio makes the planner more willing to use index scans.

When not to add an index

You shouldn’t add an index just because a column appears in a WHERE clause once. Indexes pay off when they serve a repeated access pattern better than their write and storage cost.

Avoid or remove an index when:

• It is rarely used → Unused indexes still slow writes.

• It duplicates another index → Same leading columns often mean overlapping value.

• The predicate is not selective → If most rows match, scanning may still be cheaper.

• Writes dominate the workload → More indexes means more maintenance on every change.

• The query prevents index use → Wrapping a column in a function can block index use unless you build the matching expression index.

There is also a physical side to this.

Indexes can fragment, bloat, and lose page density over time, especially under heavy updates. That’s why rebuild, reorganize, or reindex operations exist across major databases. An index can be conceptually correct and still operationally unhealthy.

The right mental model

The best indexing strategy is not “index everything.”

It is “match the structure to the predicate, then verify with real plans.”

That means:

• Start with the query pattern → Index for how you filter, join, and sort.

• Start with B-trees → Switch only when the data shape clearly demands another family.

• Keep stats fresh → The optimizer can only choose well with up-to-date statistics.

• Prefer fewer, sharper indexes → Each one should earn its keep.

• Re-check over time → Workloads shift, and yesterday’s useful index becomes tomorrow’s write penalty.

Indexes are best understood as a trade: you spend extra work on writes so reads can skip unnecessary work. Once you see that trade clearly, index tuning stops feeling like superstition and starts feeling like engineering.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Write Software: Private, Persistent, Share with a Link

Presented by exe.dev

Exe.dev gives you instant VMs over SSH. Spin up a full VM in seconds over SSH with built-in HTTPS exposure, persistent storage, and zero cloud configuration. No networking setup, no rebuilding environments, no YAML. Optimized for side projects, internal tools, and AI agents. Start a new sandbox, systemd and all, in a second.

Try it Out

Circuit Breakers: How to Stop Small Failures from Spreading

Retries make your system more resilient. Until they make it worse.

When a dependency is genuinely down, retrying doesn’t help.

That is how a small outage turns into a bigger one: threads pile up, retries multiply, queues grow, and healthy parts of the system start failing just because they are waiting on something unhealthy.

A circuit breaker exists to stop that spread.

It fails fast on purpose, because a quick “no” is often safer than a slow collapse.

What a circuit breaker actually is

A circuit breaker is a proxy that sits between your code and a dependency like an API, database, or downstream service.

It monitors every call for failures, timeouts, and slow responses. When things look bad enough, it stops forwarding requests entirely and starts failing fast instead of letting threads pile up waiting for a service that won’t recover on its own.

Most health-based circuit breakers use three states:

• Closed → Requests flow normally, and the breaker tracks failures, timeouts, or slow calls.

• Open → Requests fail immediately instead of touching the unhealthy dependency.

• Half-open → A small number of trial calls test whether the dependency has recovered.

  

What triggers a trip

Not all failures look the same, so not all circuit breakers trip for the same reason.

Some open after a run of consecutive failures. That works well for low-traffic calls where even a few failures are meaningful.

Some open on failure rate. Instead of asking “Did five calls fail in a row?” they ask “Is too large a share of recent traffic failing?” That works better for high-volume services because it filters out noise.

Some open on slow-call rate. This is important because a dependency does not need to be fully down to be dangerous. A slow database can burn your connection pool long before it starts returning errors.

And some proxy or mesh-level breakers focus on capacity rather than health. They trip on too many active requests, pending requests, retries, or connections. These don’t ask whether the dependency is broken. They ask whether the system can safely take more load.

Here is the key difference:

That is why teams often use both. One protects against unhealthy behavior. The other protects against unhealthy volume.

Why circuit breakers help so much

The biggest benefit is not speed. It is containment.

When a dependency is clearly struggling, the breaker stops your service from wasting resources on calls that are likely to fail anyway.

That gives the rest of your system room to breathe.

It also makes failure easier to shape.

• Fail fast → Users get a quick error or fallback instead of waiting through long timeouts.

• Protect capacity → Threads, sockets, and pools stay available for work that still has a chance to succeed.

• Reduce blast radius → One broken service is less likely to drag healthy services down with it.

• Support graceful degradation → You can return cached data, defaults, or partial results instead of total failure.

  

This is why circuit breakers often pair well with fallbacks. If a recommendation service is down, your app can still show popular items. If a profile service is slow, you can still render most of the page.

The user sees degraded behavior, not a frozen system.

Observability is non-negotiable

A circuit breaker without monitoring is a black box that silently shapes your availability.

At minimum, track:

• State transitions (Closed → Open → Half-Open)

• Failure rate and slow-call rate gauges

• Not-permitted calls, requests the breaker rejected without forwarding

• Overflow counters if you’re using Envoy (e.g. upstream_cx_overflow)

Alert on state transitions when they exceed a rate threshold, and alert on sustained not-permitted calls correlated with user-impact SLIs (service level indicators).

One-off blips don’t warrant pages; sustained opens that burn your error budget do.

When not to lean on a circuit breaker

Circuit breakers are powerful, but they are not a substitute for basic resilience hygiene.

Do not use circuit breakers as a bandage for missing timeouts. If your calls can hang indefinitely, a circuit breaker will only react after damage is already done. Always set clear, enforced timeouts first, because they limit how long resources stay blocked.

Separate retries from circuit breakers. They solve different failure patterns. Retries help with brief, transient failures. Circuit breakers help when failure lasts long enough that retrying becomes harmful.

Avoid adding breakers everywhere. Not every dependency needs one. Each breaker introduces configuration, monitoring, and edge cases. Use them where failure is costly, not where it is merely inconvenient.

And be careful with shared breakers. If one breaker covers several independent shards or providers, failures in one can block traffic to the healthy ones too.

Wrapping up

Cascading failure doesn’t start with a catastrophic event.

It starts with one slow service and nothing to stop the pressure from spreading.

A circuit breaker is that stop.

That is the real win: a smaller blast radius when something goes wrong. And in distributed systems, survival often depends less on avoiding failure and more on containing it.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Unblocked: Context That Saves You Time and Tokens

Presented by Unblocked

Stop babysitting your coding agents. Unblocked gives them the organizational knowledge to generate mergeable code without the back and forth. It pulls context from across your engineering stack, resolves conflicts, and cuts the rework cycle by delivering only what agents need for the task at hand.

See how it works

REST APIs Principles Explained Clearly

What makes an API RESTful?

If your first thought is “resources and CRUD,” you’re missing half the picture.

REST is an architectural style with specific constraints, and most APIs only follow a subset. Knowing the full model helps you design better interfaces; and know when to choose something else.

What is REST

REST (Representational State Transfer) is an architectural style for distributed systems. It’s not a protocol, not “just JSON,” and not a synonym for “HTTP API.”

To be truly RESTful, a system follows six constraints:

• Client–server separation → UI and data/service concerns are split so each can evolve independently

• Stateless interactions → Every request carries the full context; the server stores no client session

• Cacheable responses → Responses say whether they can be cached, so intermediaries can safely reuse them

• Uniform interface → Resources have URIs; standard HTTP methods and status codes describe operations. In strict REST, the interface is also driven by hypermedia (HATEOAS), though most APIs don’t fully implement this.

• Layered system → Clients can’t tell if they talk to the origin server, a gateway, or a proxy in the middle

• Code-on-demand (optional) → Servers may send executable code (like JS) to extend client behavior

These rules sound theoretical, but they explain why REST scales, why it’s easy to debug, and why you can throw a CDN or API gateway in front without rewriting your app.

The REST constraints, de-jargoned

Here’s how the core constraints translate into day-to-day API design:

Think of REST as the web’s “grammar” for talking about resources. Once you learn it for one API, others feel familiar because they follow the same structure.

How a REST API actually works

A REST API is basically: resources + URIs + HTTP semantics.

• Resources → Domain nouns like users, orders, products, employees

• URIs → Stable paths: /products, /products/42, /customers/123/orders

• Methods → Map closely to CRUD:

  • GET → Read

  • POST → Create

  • PUT / PATCH → Update

  • DELETE → Remove

Add to that:

• Representations → Usually JSON documents that describe the resource state

• Status codes → Shared language for outcomes (200 OK, 201 Created, 404 Not Found, 500 Internal Server Error)

• Caching → GET endpoints designed to be safely cached at browser, proxy, or CDN layers

For example: in an e-commerce API, GET /products lists items, GET /products/123 shows one product, POST /orders creates an order, and GET /customers/123/orders lists a customer’s history. Internally, the server can switch databases or split into microservices, but as long as those URIs and contracts stay stable, clients don’t care.

Why REST became the default

REST won not because it’s perfect, but because the trade-offs line up well with how most applications work.

• Simple to learn → You can poke at a REST API with a browser or curl because it’s just HTTP

• Interoperable → Any language, framework, or device that can speak HTTP can be a client

• Scalable → Stateless servers are easy to scale horizontally behind a load balancer

• Fast enough → Built-in HTTP caching plus lightweight JSON is plenty for most CRUD-style workloads

• Decoupled → Backend teams can refactor internals or swap infrastructure without breaking clients

• Great tooling → OpenAPI, Postman, language SDK generators, gateways, and observability stacks all “speak REST”

If your domain is resource-centric (products, posts, users, tickets) and most operations are CRUD-ish, REST is usually the path of least resistance.

Where REST starts to creak

REST’s weaknesses show up once your data needs get more complex or more demanding than “list + detail”:

• Over-fetching → Clients get big objects when they only needed a few fields

• Under-fetching → Clients chain multiple calls (user, then orders, then recommendations) to render one screen

• Chattiness → Many small HTTP requests stack up latency, especially on mobile networks

• Loose contracts → JSON shapes are enforced by convention and docs, not by the protocol itself

• Versioning pain → Breaking changes often mean /v2 endpoints or awkward backwards compatibility

• Real-time gaps → REST is request–response; you need WebSockets or SSE for live updates

These aren’t fatal flaws, but they’re signals that you might need something in addition to REST.

REST vs GraphQL vs gRPC: choosing wisely

You don’t have to pick a single style for everything. Use them where they fit best:

A good rule of thumb:

• Start with REST → because it’s simple, well-understood, and easy to expose externally

• Add GraphQL → when specific clients juggle too many REST calls or need tailored payloads

• Use gRPC → inside your backend when performance, streaming, or strong typing is critical

When not to use REST

Avoid pure REST as the main interface when:

• Your main goal is to query complex data, not manage individual resources.

• Your services call each other thousands of times per second and every millisecond and byte counts

• Your clients need live streams of updates, not occasional snapshots

In these cases, REST can still play a supporting role (for management, control, or external access), but it shouldn’t carry the whole workload.

Recap

REST is more than JSON over HTTP. It’s a set of constraints that, when followed, give you APIs that are simple, scalable, cacheable, and widely compatible.

Understand those principles and you can do three important things: design cleaner APIs, recognize when a “REST API” isn’t really RESTful, and know when to reach for GraphQL or gRPC instead.

That’s the difference between copying patterns and making deliberate architectural choices.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Build a Multi Agent RAG system with A2A Protocol and LangChain

Presented by Oracle

Most multi-agent systems hit a bottleneck when one agent gets overloaded. The usual fix is scaling the entire pipeline, which adds cost without solving the real problem. With the A2A protocol, agents can discover each other, declare capabilities, and dynamically share workload so you can scale only the bottleneck.

This guide walks you through building a scalable multi-agent RAG system.

Start building

How Single Sign-On Actually Works

You’ve used SSO dozens of times without thinking about it.

You click “Sign in with Google,” land on a familiar login page, and seconds later you’re inside an app that never asked for your password.

It felt instant and frictionless.

But behind that smooth experience is a precise sequence of redirects, tokens, and cryptographic checks played out in an instant.

SSO isn’t magic. It’s a trust protocol; and understanding it changes how you design authentication, debug login failures, and make decisions about identity infrastructure.

The core idea: Centralize identity, decentralize access

At its heart, SSO separates authentication from application logic.

Instead of every app verifying usernames and passwords, one central service does it. That service is called the Identity Provider (IdP). The applications you actually use are called Service Providers (SPs).

The key shift is this: SPs don’t authenticate you. They verify that the IdP authenticated you.

That distinction matters because it defines who owns credentials, who signs tokens, and who enforces trust.

Why browsers make this necessary

Browsers enforce the same-origin policy. That means one website cannot read another website’s cookies.

So if app1.com and app2.com are separate domains, they cannot directly share login sessions.

SSO solves this by introducing a trusted third party: the IdP lives on its own domain (e.g. login.company.com). Each application redirects you there when authentication is required. The IdP becomes the only place where credentials are checked.

Instead of apps sharing cookies, they share trust.

The step-by-step flow

Here’s what happens when you access an SSO-protected app for the first time:

1. User requests a protected app

You visit serviceprovider.com.

The SP checks for a valid session cookie, finds none, and knows it needs to authenticate you.

2. SP redirects to the IdP

Instead of showing a login form, the SP issues a HTTP redirect to the IdP.

That redirect includes:

• A request identifying the SP

• A return URL

• Possibly a signed authentication request

Your browser now talks to the IdP.

3. IdP authenticates the user

If you’re not already logged in at the IdP, it presents a login page.

You enter credentials. The IdP verifies them. It may also enforce:

• Multi-factor authentication

• Device checks

• Risk-based policies

If successful, the IdP creates its own session cookie for you.

4. IdP issues a token

The IdP generates an authentication token. This token contains:

• User identity

• Issuer (the IdP)

• Audience (the SP)

• Expiration timestamp

• Digital signature

The browser is redirected back to the SP with this token attached.

5. SP validates the token

The SP verifies:

• Signature → Was it signed by the trusted IdP?

• Expiry → Is it still valid?

• Audience → Was it intended for this app?

If everything checks out, the SP creates its own session and logs the user in.

At no point did the SP see the password.

Where “single” sign-on actually happens

The magic appears when you visit a second application.

You go to anotherservice.com.

It redirects you to the same IdP.

But now the IdP sees its session cookie. You’re already authenticated.

So it immediately generates a new token for the second app without asking for credentials again.

From your perspective, no login screen appears.

Behind the scenes, the full protocol still runs. It just skips the credential step.

That’s single sign-on.

What makes it secure

The authentication token is the core artifact. Its security rests on three properties:

• Digital signature → The IdP signs the token with its private key. The SP verifies the signature with the IdP’s public key, proving the token wasn’t forged or altered in transit.

• Bounded lifetime → Tokens include an expiry timestamp. A stolen token becomes useless after minutes, limiting the blast radius of interception.

• Audience restriction → Tokens name the intended SP. A token issued for app1.com is rejected by app2.com, preventing token reuse across services.

The SP never sees the user’s password. It only sees the signed assertion.

When SSO becomes a risk

SSO centralizes authentication. That’s powerful, but it concentrates risk.

• If the IdP goes down → every SP login fails

• If the IdP is compromised → every integrated app is exposed

SSO reduces password sprawl but increases dependency on identity infrastructure.

It works best when:

• The app is used by teams or organizations, not just individuals

• Customers expect centralized admin control (onboarding, offboarding, permissions)

• The app handles sensitive data or privileged actions (finance, customer data, infra, admin tooling)

• You’re selling into mid-market or enterprise, where SSO is often a requirement

• Users are part of a broader ecosystem (Google Workspace, Entra, Okta) and need seamless access across tools

• The product is part of a multi-app suite, and reducing login friction improves workflow

It may be overkill for:

• Consumer apps with individual users and no org-level administration

• Early-stage products where few (or no) customers are asking for SSO yet

• Low-risk applications that don’t handle sensitive data or critical operations

• Small internal tools used by a fixed group, especially behind VPN or other trust boundaries

• Apps where identity is not central to the workflow and login happens infrequently

Wrapping up

SSO removes the password from your app.

It doesn’t remove the responsibility.

Token validation, session expiry, single logout, and IdP availability all stay on your plate; and they matter more, not less, because authentication is now shared infrastructure.

The protocol handles the authentication. Everything that follows is still yours to design.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Reproducible Dev Environments That Work Everywhere

Presented by Flox

Flox lets teams define development environments once and run them consistently across local environments, CI pipelines, and production systems. By capturing dependencies, tools, and environment variables in a single manifest, teams can eliminate environment drift and ship software more reliably.

Learn More

Hashing vs Encryption vs Tokenization: How to Pick the Right Tool

Most security vulnerabilities aren’t caused by weak algorithms.

They come from using the right tool in the wrong situation.

Encrypting what should be hashed, or hashing what should be tokenized, doesn’t trigger an alarm; it just quietly erodes your defenses while everything appears to be working fine.

Hashing, encryption, and tokenization all transform sensitive data, but they answer completely different questions. Knowing which question you’re asking is the whole game.

The core question: what are you trying to protect?

Before picking a tool, answer one question:

Do you need to recover the original value?

• If no → hashing is likely the right fit.

• If yes, and the goal is confidentiality → use encryption.

• If yes, but the goal is limiting where sensitive data spreads → use tokenization.

Each technique maps cleanly to a different security objective:

• Hashing → Integrity and verification

• Encryption → Confidentiality with controlled access

• Tokenization → Exposure reduction across systems

They overlap in underlying mechanisms. They do not overlap in purpose.

Hashing

A cryptographic hash function takes input of any size and produces a fixed-length output called a digest. Change a single character, and the output changes completely.

It is intentionally one-way: you cannot feasibly recover the original input from the digest.

Hashing is built for integrity:

• Verifying file downloads

• Detecting message tampering

• Supporting digital signatures

• Storing passwords (with specialized constructions)

But hashing has a subtlety most developers miss: not all hashing is the same.

• Unkeyed hash (e.g. SHA-256) → Confirm that data hasn’t changed, but only through a trusted channel. An attacker who controls both the file and its published digest can swap both.

• Keyed hash (e.g. HMAC) → Use a shared secret to authenticate data against an active attacker. Required when you need integrity and authenticity.

• Password hashing (e.g. Argon2, scrypt, bcrypt) → A specialized case. Deliberately slow and memory-hard to make brute-force attacks expensive after a database breach. Never use SHA-256 for passwords.

The most common failure here is using a fast, unkeyed hash as a stand-in for authentication.

It isn’t.

What hashing cannot do:

• It cannot provide confidentiality.

• It cannot restore data.

• It does not automatically provide authentication unless keyed.

If you need the original value later, hashing is the wrong tool.

Encryption

Encryption transforms plaintext into ciphertext using a key. Only someone with the right key can reverse it.

Unlike hashing, encryption is reversible by design.

This makes it the right tool for confidentiality:

• Data in transit (TLS)

• Data at rest (disk/database encryption)

• Secure messaging

• Secure storage

Modern encryption comes in two main shapes:

• Symmetric encryption (AES) → The same key encrypts and decrypts. Fast and designed for bulk data.

• Asymmetric encryption (RSA, ECC) → Public key encrypts, private key decrypts. Slower, used primarily for key exchange and signatures, not bulk data.

Real systems almost always use hybrid encryption: symmetric encryption for the data itself, asymmetric encryption to securely exchange the symmetric key. TLS works exactly this way.

There’s one critical nuance: confidentiality is not the same as integrity.

Today, encryption without integrity is considered incomplete. Authenticated Encryption with Associated Data (AEAD), such as AES-GCM, protects both:

• Confidentiality → Attackers can’t read it.

• Integrity/authenticity → Attackers can’t modify it undetected .

Older designs that are encrypted without authentication can be manipulated by an attacker who never sees the plaintext. This is why AEAD is now strongly preferred.

What encryption optimizes for:

• Controlled access to original data

• Strong confidentiality guarantees

• Regulatory recognition (GDPR, HIPAA references encryption explicitly)

But encryption introduces a heavy dependency:

Key management.

Key lifecycle, rotation, storage (often HSM-backed), and cryptoperiod policies become first-class concerns.

Encryption protects content. It does not reduce how widely that content spreads.

Tokenization

Tokenization replaces a sensitive value with a surrogate “token”. The token carries no exploitable meaning on its own.

For example, in payment systems a PAN (Primary Account Number) becomes a token and downstream systems store the token, not the card number.

The difference from encryption is architectural:

• Encryption protects data.

• Tokenization limits exposure of data across systems.

Encryption is the answer to “how do we hide this data?”. Tokenization is the answer to “how do we stop this data from spreading?”.

There are three main implementations:

• Vault-based tokenization → A random token is generated and stored alongside the original value in a protected database (the vault). De-tokenization means querying the vault. Simple to reason about, but the vault becomes your highest-value attack target.

• Vaultless (cryptographic) tokenization → The token is generated using format-preserving encryption. No vault needed, but key management becomes critical.

• Irreversible tokenization → Tokens are never designed to be reversed. Used when you only need to validate or reference a record, not recover the original value.

Tokenization earns its place in regulated environments because it limits where sensitive data lives.

Side-by-side comparison

Hashing, encryption, and tokenization all transform data, but they protect it in very different ways.

When not to use each

Knowing when not to reach for a tool is as important as knowing when to use it.

Don’t use hashing when:

• You need to retrieve the original value. A hash is a one-way door.

• You’re protecting a password with a fast algorithm like SHA-256. Use bcrypt, scrypt, or Argon2 instead, because speed is the enemy of password security.

• You’re treating an unkeyed hash as a MAC. It isn’t, and extension attacks will prove it.

Don’t use encryption when:

• You never need to read the original value back. Hashing is simpler and removes the key-management burden.

• You’re trying to limit data spread across services. Encrypting a PAN and sending the ciphertext everywhere just moves the problem; tokenization removes it.

Don’t use tokenization when:

• Your system is small and doesn’t have a serious compliance or data-minimization requirement. Running a vault adds availability dependencies and operational overhead that may not be justified.

• You need the original value to be queryable for range scans or analytics joins. Tokenization scatters values in ways that break ordering and aggregation.

Recap

Hashing, encryption, and tokenization aren’t variations of the same idea; they’re answers to different questions.

Hash when you need to verify. Encrypt when you need to retrieve. Tokenize when you need to contain.

Get the question wrong and the technique doesn’t fail loudly; it just quietly protects the wrong thing.

The algorithm is rarely the problem. The choice usually is.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

The Aiven Free Tier Competition

Presented by Aiven

The best way to learn is by doing. Build with Aiven’s free tier services like Kafka and PostgreSQL. No credit cards, no expiring trials, just real, managed instances to help you get your project off the ground. Build, share, and win $1,000!

Start building

Synchronous vs Asynchronous Communication

Why does placing an order feel instant in one product and slow in another?

The difference is often invisible to users but fundamental to the system.

It comes down to how much of the checkout flow runs synchronously, waiting for every dependency to confirm success, versus how much runs asynchronously, allowing non-critical work to complete after the order is accepted.

The only difference that matters: do you wait?

Imagine clicking “Purchase”.

Behind that button, the system fans out to payments, inventory, shipping, email, analytics, and fraud checks.

In a synchronous design, the checkout flow waits.

The order service calls each dependency and blocks until every one responds. If payment is slow or the email service hiccups, the user stares at a spinner. Nothing moves forward until the last response comes back.

In an asynchronous design, the order is accepted first.

The system records the order, responds to the user, and emits messages for the rest of the work. Payment confirmation, receipts, analytics, and fulfillment happen independently, often through a queue or event bus, without holding the checkout flow hostage.

That’s the real distinction.

Synchronous communication means sending a request and waiting for a reply before continuing.

Asynchronous communication means sending a message and moving on, trusting the system to finish the work later.

Synchronous communication: wait for the response

Synchronous calls feel clean because the control flow stays linear:

• Inline response → you get a result (or error) right away, in the same execution path.

• Known outcome on return → the caller knows whether it succeeded as soon as the call finishes.

• User-facing fit → logins, reads, validations, and short operations match the “user is waiting” model.

But synchronous has temporal coupling (dependency on being up at the same time). If the downstream service slows down or fails, the caller waits, times out, or fails too; often turning one slow component into a platform-wide incident.

It also stacks latency. A single call can be fast, but a chain of calls is serial: A waits for B, then B waits for C, and the user waits for all of it. Under load, threads pile up waiting and throughput collapses.

When not to use synchronous communication

Asynchronous communication: send and continue

Asynchronous systems usually introduce a message broker (a system that stores and routes messages) and communicate via queues or topics. Producers publish; consumers process when they can.

This changes the system’s structure:

• Decoupling & resilience → messages can wait when a consumer is slow or offline, reducing cascading failures.

• Throughput & scalability → producers don’t block, and you can scale consumers horizontally to chew through backlog.

• Fan-out by default → one “order placed” event can trigger email, analytics, fraud checks, and inventory updates without the producer calling each service directly.

The trade is eventual consistency: the system acknowledges the request now, but the final outcome happens later. How long it takes depends on how many messages are waiting and how fast consumers process them.

Async also raises the engineering bar:

• More moving parts → brokers, persistence, acknowledgements, retries.

• Idempotency required → consumers must safely handle duplicates (idempotent means “running it twice produces the same effect as once”).

• Harder failure handling → errors may surface later, sometimes via a dead-letter queue (messages that repeatedly fail and need inspection).

When not to use asynchronous communication

How to choose in practice

Most real systems don’t pick one style.

They place synchronous and asynchronous communication deliberately.

Use synchronous communication when the system cannot move forward without a definitive answer.

This includes login checks, payment authorization, inventory validation, and critical reads. In these cases, waiting is the point because correctness matters more than throughput. Failures should surface immediately and stop the flow.

Use asynchronous communication when the work does not need to block the user.

Long-running tasks, bursty workloads, and fan-out behavior belong here. Queuing the work keeps the system responsive and prevents one slow dependency from stalling everything else.

This is how systems gain autonomy and absorb partial failure.

Most mature architectures follow a simple pattern:

• Sync at the edge → confirm the user’s action quickly and clearly.

• Async behind the edge → process side effects like notifications, analytics, and downstream updates without holding the request open.

That split keeps the user experience fast while letting the system scale and recover under load.

One important caveat: “async code” is not the same as asynchronous communication.

If a service must wait for a response before it can proceed, the interaction is still synchronous at the architecture level; even if it uses promises, callbacks, or non-blocking I/O.

Wrapping up

Go back to the moment the user clicks “Purchase.”

Whether that action feels instant or slow is a design choice.

Payment authorization, inventory checks, and order creation run synchronously because the system needs a definitive answer.

Waiting there is intentional. That’s the critical path.

Everything else (emails, analytics, fraud checks, fulfillment) doesn’t need to block the user. Those steps run asynchronously so the order can complete even when downstream systems are slow or unavailable.

That’s the real trade-off.

Synchronous communication gives you immediate certainty but shared failure. Asynchronous communication gives you resilience and throughput by letting work finish later.

Well-designed systems use both.

If a user flow feels slow, the fix isn’t always faster processing.

Often, it’s deciding what truly needs an answer now; and what can wait.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Why Your AI Agent Has Amnesia

Presented by Oracle

Many AI agents struggle with memory. They lose context, forget prior steps, and produce inconsistent outputs across sessions. This guide explores why it happens and how to fix it using agent memory patterns. A great read!

Read the article

How Databases Keep Passwords Safe

“Hash it and you’re done” isn’t a strategy. It’s how breaches turn into headlines.

Every system is one misconfiguration, forgotten backup, or zero-day exploit away from a database leak.

When that happens, the only thing standing between you and a wave of account takeovers is how those passwords were stored.

Password security isn’t about keeping data secret; it’s about making stolen data useless.

That’s why modern storage relies on one-way hashing, unique salts, and slow, memory-hard algorithms that make every brute-force attempt painfully expensive.

Real security doesn’t assume safety, it assumes compromise and plans for it.

How secure storage works

When a database breach happens, it’s already too late to protect the data itself.

What still matters is how much that data can be used against you.

If your system stores plaintext passwords, attackers can log in immediately. Even “hashed” passwords aren’t safe if you used fast, general-purpose algorithms like MD5 or SHA-1.

GPUs can brute-force at billions of guesses per second.

To prevent that, password storage must slow attackers down.

Modern password storage relies on layered defenses. Each step (hashing, salting, stretching, and sometimes peppering) closes a different gap that attackers exploit.

• Hashing → Converts a password into a one-way digest that can’t be reversed. The server never stores the password itself, only its hash.

  • Use password-specific algorithms like bcrypt, scrypt, or Argon2id, not general-purpose hashes like SHA-256. Fast hashes are designed for data integrity, not for resisting brute force. A proper password KDF (Key Derivation Function) intentionally slows verification to make offline guessing expensive.

• Salting → Adds a random, per-user value to each password before hashing.

  • Salts ensure that identical passwords never produce the same hash and stop rainbow table lookups. They don’t need to be secret, but they must be unique and unpredictable (16 bytes or more).

• Key stretching → Increases the cost of each hash operation through extra iterations or memory hardness.

  • This throttles brute-force attacks: where a fast hash might take microseconds, a stretched one takes hundreds of milliseconds. On your servers, that’s manageable. For an attacker, multiplied by millions of guesses, it’s economically pointless.

• Peppering (optional) → Adds a server-side secret, stored outside the database (like in an HSM or key vault).

  • Even if the database leaks, the attacker can’t verify guesses without the pepper. It strengthens security but complicates rotation; use it only if your infrastructure can protect and rotate secrets safely.

The goal isn’t to make passwords uncrackable, but to make cracking so slow and individualized that attackers move on long before they succeed.

How passwords are stored and verified

When a user creates an account or logs in later, the system never handles their password in plain text.

Both flows (signup and login) rely on the same hashing process but apply it at different points.

1. Signup → Creating and storing the hash

When a user signs up or resets their password, the system never stores the password itself. Instead, it transforms it into a secure representation:

1. Generate a salt → Create a random, unique salt for this user (usually 16 bytes or more).

2. Hash the password → Run the password and salt through a password hashing function such as Argon2id, bcrypt, or scrypt. The function also takes cost parameters (iterations, memory, or parallelism) that control how slow the process is.

3. Store the result → Save the resulting hash, along with the salt and parameters, in the database.

   

The original password is discarded immediately. Only the salted hash and configuration is stored.

2. Login → Verifying the password

When the user logs in later, the system must check the password without knowing the original one:

1. Retrieve stored values → Fetch the user’s stored hash, salt, and parameters.

2. Re-hash the input → Run the entered password through the same function using the retrieved salt and parameters.

3. Compare securely → Use a constant-time comparison to check whether the new hash matches the stored one. Unlike normal string checks that exit early on mismatches, constant-time comparisons take the same amount of time no matter how similar the inputs are. This prevents timing attacks, where an attacker measures tiny response-time differences to guess parts of the correct hash one byte at a time.

4. Rehash if needed → If parameters are outdated (for example, fewer iterations, lower memory, or an outdated algorithm), re-hash and update the stored entry after a successful login.

5. Authorize access → If hashes match, issue a session token or JWT for the user.

   

At no point does the system store, transmit, or recover the plaintext password. The entire process proves identity by reproducing the same one-way result, not by revealing the secret itself.

What not to do

Most password breaches don’t happen because teams ignored security, they happen because they trusted weak defaults or outdated practices.

Here’s what to avoid if you want your password storage to hold up under real attacks.

• Store plaintext passwords → The worst-case scenario. A single database dump means instant access to every account. There’s no recovery from this.

• Use general-purpose hashes (MD5, SHA-1, SHA-256) → These functions are designed for speed, not security. They are trivial to brute-force at scale with modern GPUs and ASICs.

• Skip or reuse salts → Without a unique, random salt per password, identical passwords produce identical hashes, enabling rainbow table attacks and cross-user correlation.

• Set low cost factors → bcrypt cost=8 or PBKDF2 with a few thousand iterations might have been fine a decade ago. Today, it’s too cheap to crack. Always tune cost to current hardware performance.

• Roll out your own algorithm → Security through creativity is not security. Established KDFs like Argon2id, bcrypt, or scrypt have been battle-tested and peer-reviewed; your custom hashing function has not.

• Forget edge behaviors → Logging plaintext passwords during debugging or including them in error traces can silently undo all your protections.

• Expose hints or partial passwords → Anything that helps an attacker narrow the search space reduces the work of cracking. Even “helpful” password hints expose predictable details.

Recap

Good password storage doesn’t depend on secrecy; it depends on math, cost, and discipline.

Hashing, salting, stretching, and (optionally) peppering turn every password into a unique, expensive problem for attackers to solve.

The goal isn’t to make breaches impossible, but to make them irrelevant by ensuring stolen data is useless.

Use proven KDFs like Argon2id or bcrypt, tune them regularly, and avoid shortcuts that trade speed for safety.

When done right, a database leak becomes an inconvenience; not a disaster.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Build AI Apps with MongoDB

Presented by MongoDB

Looking to stay ahead of the curve on AI? MongoDB AI Learning Hub has the technical training pathways and tools you need to level up your AI app-building game. Explore practical guides, tutorials, and quick starts for all skill levels, from foundational concepts like understanding AI tool stacks to advanced implementations using RAG, MongoDB Vector Search, and LLM optimization.

Check it out (it's free)

CAP Theorem Clearly Explained

“Make it consistent, highly available, and fault-tolerant” sounds like a reasonable requirement.

It isn’t.

In any distributed system, a network failure forces a choice that no amount of engineering can avoid: either your nodes agree on the truth and some requests fail, or every request gets an answer and some of those answers are wrong.

Both are valid designs, and CAP Theorem is the lens that makes that tradeoff explicit so you choose deliberately instead of by accident.

What is CAP Theorem?

CAP Theorem states that a distributed data store cannot simultaneously guarantee all three of the following properties at the same time:

• Consistency (C) → Every read returns the most recent write, or an error; never stale data.

• Availability (A) → Every request receives a response, even if some nodes are out of date.

• Partition Tolerance (P) → The system keeps operating despite network failures that split nodes into isolated groups.

The common summary is “pick two of three.”

It’s catchy, but misleading.

CAP is not about permanently giving something up. It’s about what happens when the network splits.

A clearer way to look at it: when a partition occurs, you must choose between consistency (all nodes agree on the same data) and availability (every request receives a response).

During normal operation, when nodes can communicate reliably, you can have both.

The tradeoff only appears the moment the network breaks.

CP vs AP: two different failure philosophies

Systems that lean toward Consistency + Partition Tolerance (CP) will refuse or delay operations when they can’t guarantee a single, up-to-date view of the data. They’d rather return an error than return something wrong.

Systems that lean toward Availability + Partition Tolerance (AP) will keep serving requests even when nodes disagree, accepting that some responses may be stale. They’d rather serve something than serve nothing.

How a partition actually plays out

A partition occurs when there’s a network split and nodes can’t reliably talk to each other.

Imagine a five-node cluster. A network failure divides it into two groups: three nodes on one side, two on the other.

CP behavior

CP systems preserve a single source of truth by refusing to make progress without enough agreement.

• Quorums/consensus → Writes (and sometimes reads) require enough nodes to agree.

• Leader-based replication → Only one leader can accept writes; the other side of a split becomes read-only or errors out.

• User experience → Some users see failures/timeouts, but you avoid split-brain state.

The three-node side has a majority, so it continues processing requests.

The two-node side does not, so it stops accepting writes. It may become read-only or fully unavailable.

When the network heals, the minority side syncs from the majority. There is no data conflict because only one side was allowed to move forward.

The result is clean, but some users saw errors or timeouts during the split.

AP behavior

An AP system keeps accepting operations on each side of the partition and repairs the mismatch later.

• Eventual consistency → Replicas converge after communication returns.

• Conflict resolution → You need a resolution plan: last-write-wins, merge, keep multiple versions, or domain-specific reconciliation.

• User experience → The system stays up, but different users may see different answers temporarily.

Each side believes it has the latest state. When the network reconnects, the system detects divergent updates; for example, two users modifying the same record on different sides.

Now reconciliation begins.

The database applies a conflict strategy such as last-write-wins. The service stayed available, but some data required conflict resolution.

Neither outcome is a bug. Both are deliberate design decisions baked into the system you chose.

Common mistakes that cause outages

Most confusion around CAP doesn’t come from the theorem itself.

It comes from how we simplify it.

Here are the mistakes that cause the most architectural regret.

• Treating CAP as a permanent label → A system is not “CP” or “AP” as a fixed identity. During normal operation, well-designed systems provide both consistency and availability because the network is healthy.

• Assuming partition tolerance is optional → You cannot avoid partitions by trusting your infrastructure. You can reduce their frequency with better networking, tighter time synchronization, or private backbone links, but you cannot eliminate its possibility entirely.

• Confusing CAP consistency with ACID consistency → These are different guarantees. ACID consistency means a transaction leaves the database in a valid state according to defined rules. CAP consistency means every read returns the most recent write across nodes.

• Forgetting to design recovery logic → Choosing availability during a partition is only half the decision. “Eventually consistent” requires a reconciliation strategy. You must define how conflicts resolve.

A decision checklist you can actually use

Start by asking one question: what hurts your product more, returning incorrect data, or returning no data at all?

• Returning incorrect data hurts more → Choose CP.

  • Common use cases: payments, balances, inventory counts, permissions, leader election

• Returning no data at all hurts more → Choose AP.

  • Common use cases: feeds, analytics views, caches, activity timelines, search indexes

Most real systems don’t pick one globally; they pick per component.

A common pattern is to have a small CP core for writes that must be correct (orders, balances, permissions), and AP layers that serve and cache data quickly across regions.

The core enforces truth; the edge keeps the system fast.

The worst outcome is not choosing consciously. Systems that ignore CAP don’t escape its constraints; they just encounter them as surprises.

Wrapping up

Every distributed system has already answered the CAP question, either deliberately or by default.

The database you picked, the replication strategy you configured, the consistency level you left unchanged: all of it adds up to a position on the spectrum.

Understanding CAP doesn’t give you new options. It tells you which option you’re already on, and whether it’s the right one.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Sonar Summit: A Global Conversation on Building Better Software in the AI Era

Presented by Sonar

The question isn’t whether to use AI. It’s how to adopt it without quietly increasing risk and degrading code quality. That shift is exactly why conversations about software quality in the AI era matter right now. This is exactly the kind of conversation happening at Sonar Summit.

The virtual event is on today. It’s free to watch / attend.

Check it out (for free) here

Event-Driven Architecture Clearly Explained

Most systems start as a neat chain of calls: service A calls B, B calls C, C returns.

Add one new downstream requirement, and suddenly you’re editing code three layers deep, redeploying half your stack, and praying the chain holds.

This is the hidden cost of direct coupling, every new feature tightens the knot. Event-Driven Architecture (EDA) cuts it.

Instead of services calling each other directly, they broadcast events: facts about things that happened. Any service that cares can react. The one that emitted the event never needs to know who listened.

What Event-Driven Architecture actually is

Event-Driven Architecture (EDA) is a design style where components communicate by producing and reacting to events rather than calling each other directly.

An event is a message that describes something that happened: “Order Placed,” “Payment Received,” “Temperature Exceeded Threshold.”

It’s a fact, not a command.

Every EDA system has three main roles:

• Producers → Detect a change and publish an event to the broker. They don’t know or care who receives it.

• Broker → The central hub (Kafka, RabbitMQ, AWS EventBridge) that receives events from producers and routes copies to every interested subscriber. It can also buffer events during traffic spikes, ensuring consumers aren’t overwhelmed.

• Consumers → Subscribe to specific event types and react asynchronously. One consumer’s slowdown doesn’t affect others or the producer.

  

The flow looks like this:

• A customer places an order

• The Order Service publishes an “Order Placed” event

• The broker routes it

• The Inventory Service decrements stock

• The Notification Service sends a confirmation email

• The Shipping Service prepares a label

  

The Inventory Service, Notification Service, and Shipping Service all work in parallel, without the Order Service knowing any of them exist.

This is the core power: decoupling. Teams deploy independently. Services evolve independently. A new feature means adding a new consumer, not modifying existing ones.

Why teams choose it

Without EDA, coordination becomes the tax you pay for every new capability.

EDA earns its place the moment you feel how much friction it removes.

• Loose coupling → Producers and consumers share only an event contract, so each side evolves independently without needing to know the other’s internals.

• Async fan-out → One event triggers many parallel workflows instantly, so adding a new downstream step costs nothing in latency.

• Elastic buffering → The broker absorbs traffic spikes so producers keep emitting while consumers scale at their own pace.

• Plug-in extensibility → New features are new consumers, not modifications to existing services.

• Fault isolation → A failed consumer doesn’t cascade; events queue until the service recovers.

The tradeoffs you need to know

EDA doesn’t eliminate complexity, it relocates it.

The architecture gets cleaner upstream; the responsibilities shift to event design, broker operations, and consumer reliability.

• Debugging complexity → Failures don’t follow a clean call path. You see a missing outcome, not the cause. Tracing and centralized logging become essential.

• Delivery semantics → Most brokers guarantee at-least-once delivery. Duplicates happen. Consumers must be idempotent or you risk double charges and inconsistent state.

• Eventual consistency → Services update asynchronously. Temporary disagreement is normal, but unsafe for flows needing immediate correctness.

• Broker dependency → The broker is critical infrastructure. If it fails, events stop flowing; often silently.

When to use it (and when not to)

EDA shines when your system needs to do many things at once in response to a single trigger, or when teams need to move independently without stepping on each other’s toes.

Use EDA when:

• One action should trigger multiple independent downstream reactions.

• Traffic is bursty or unpredictable and you need the broker to absorb spikes.

• You’re working in a microservices architecture where teams deploy independently.

• You need to integrate heterogeneous systems without tight API contracts.

• Requirements change frequently and you want to add behavior by subscribing, not by modifying.

Avoid EDA when:

• You need strict transactional atomicity; e.g. debit one account and credit another in one operation. Events introduce eventual consistency where ACID is required.

• You need guaranteed ordering across multiple consumers. Global ordering in distributed brokers is hard and expensive.

• You need immediate confirmation. If the workflow requires “did this succeed right now?”, request-response fits better.

• The system is small and simple. Introducing a broker, event schemas, and consumer infrastructure into a three-service app adds overhead without proportional benefit.

Best practices that make the difference

Getting the architecture right is only half the work. These practices determine whether it stays maintainable as it grows.

Wrapping up

The coupling that slows teams down rarely announces itself.

It builds quietly: one extra call, one more dependency, one more coordinated deploy.

EDA breaks that pattern by making components responsible only for what they know: something happened.

What the rest of the system does with that is no longer your problem. That’s not just a cleaner architecture. It enforces clearer ownership.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Sonar Summit: A Global Conversation on Building Better Software in the AI Era

Presented by Sonar

The question isn’t whether to use AI. It’s how to adopt it without quietly increasing risk and degrading code quality. That shift is exactly why conversations about software quality in the AI era matter right now.

This is exactly the kind of conversation happening at Sonar Summit. It’s free and open to the community. If you want to stay sharp in the current landscape, this is a valuable space to learn how others are approaching it.

The lineup includes well-known leaders in the engineering space.

Save your seat here (for free)

What Protocols Actually Do When You Load a Page

You don’t “just” load a web page.

Behind every click, a small crowd of network protocols is involved, each doing one very specific job so the whole thing doesn’t fall apart.

Each protocol knows its role.

And the moment you understand how they fit together, debugging shifts from frantic guesswork to methodical investigation.

1. Joining the Network → Physical & Data Link Layers

Before your device can reach the wider internet, it has to join a local network. That requires both a physical connection and a way to talk to nearby devices.

Ethernet → Wired physical & data link layers

Ethernet defines how bits travel over copper or fiber and how frames identify sender and receiver using MAC addresses. It’s stable, fast, and predictable; ideal in data centers or offices where performance matters more than mobility.

Wi-Fi → Wireless physical & data link layers

Wi-Fi carries the same type of frames, but over radio instead of wires. The convenience costs you some consistency, because radio waves are subject to interference, congestion, and range limitations.

ARP → Data link layer address resolution

You can’t talk to another device using its IP address alone. The data link layer wants a MAC address. ARP fills that gap by asking, “Who has this IP?” Local devices respond with the matching MAC. Without ARP, communication would stall on your very first hop.

DHCP → Application layer using network-layer concepts

Once you join a network, DHCP assigns your IP address, subnet mask, gateway, DNS servers, and lease duration. It removes the burden of manual configuration and ensures no two devices collide on the same IP.

When something breaks here:

• If you see “No IP address” or 169.254.x.x self-assigned IPs, the DHCP server didn’t respond or the broadcasts didn’t reach it.

• If you can’t ping your gateway, the wireless signal may be weak, the cable faulty, or ARP resolution may be incorrect.

• If the connection keeps dropping, look for Wi-Fi interference, bad cables, or a failing switch port.

At this stage, failures tend to feel blunt: no internet, unstable connection, or outright inability to join the network.

2. Figuring out where to go → Network & application layers

You’re now connected locally, but the website you want isn’t in your living room. Your device needs two things: a destination IP and a way to reach it.

DNS → Application layer name resolution

Humans like words; machines like numbers. DNS translates www.example.com into an IP address the network layer can route. If DNS fails, everything above it collapses. No IP means no connection.

IP → Network layer routing and delivery

IP packets carry your data across networks owned by different companies and countries. Each router along the way simply checks the destination address, consults its routing table, and forwards the packet along the best known path.

ICMP → Network layer diagnostics

When things go wrong, ICMP speaks up. It reports unreachable networks, expired TTLs, and other routing failures. Tools like ping and traceroute rely on ICMP for visibility into the path your packets take.

BGP → Application layer routing control for the global internet

BGP is how the world’s autonomous networks tell each other which IP ranges they can deliver. It’s why traffic sent from Sydney eventually finds its way to servers in London or São Paulo. Without BGP, the internet would be a collection of isolated networks with no map connecting them.

When something breaks here:

• If DNS queries time out or return wrong answers, you’ll see instant failures loading any site; even if physical connectivity is fine.

• If traceroute stalls or jumps unexpectedly, routing loops, misconfigurations, or upstream outages may be dropping packets at specific hops.

• If ping works but loading pages doesn’t, packets may reach the server but responses may take a different broken return path.

• If an entire region suddenly can’t reach a service, a BGP misannouncement upstream often explains the outage.

These issues often masquerade as “the internet is slow,” but the symptoms typically point squarely at name resolution or routing decisions.

3. Moving data reliably or fast → Transport layer

Now that your device knows where to send data, it needs a transport mechanism. Two major protocols define two very different philosophies.

TCP → Transport layer reliability and order

TCP makes networks feel reliable even when they’re not. It handles connection setup, retransmissions, flow control, congestion management, and ordering. If a packet gets lost, TCP repeats it. If packets arrive out of sequence, TCP rearranges them.

Most of the web still uses TCP under the hood: HTTPS, APIs, database connections, and email all depend on TCP’s guarantees.

UDP → Transport layer speed and simplicity

UDP strips out everything except the essentials: ports, length, and checksum. There’s no handshake, no retransmission, and no ordering. If a packet drops, the application decides whether to care.

When you’re in a Zoom call or a multiplayer game, you’d rather skip a single frame than pause the entire session waiting for recovery. UDP enables that.

When something breaks here:

• If connections hang on “SYN sent,” firewalls or rate limits may be blocking TCP handshakes.

• If applications feel slow despite low CPU, excessive TCP retransmissions may indicate packet loss or congestion.

• If video or audio stutters over UDP, the network may be too lossy for real-time traffic; and since UDP won’t retry, the glitch shows up immediately.

• If long-lived TCP sessions drop unpredictably, Network Address Translation (NAT) timeouts or load balancers may be silently killing idle connections.

Transport issues often feel subtle: partial loads, long pauses, “sometimes it works” behavior.

4. Securing and speaking meaningfully → Session/presentation & application layers

The transport layer moves bytes. Higher layers give those bytes structure, meaning, and security.

TLS → Session/presentation layer security

TLS encrypts the connection between two endpoints. It prevents snooping, tampering, and impersonation. Almost every meaningful web interaction now runs through TLS.

HTTP → Application layer communication

HTTP defines verbs (GET, POST), headers, caching rules, content types, and status codes that let browsers and servers communicate clearly. It is the backbone of everything from websites to REST APIs.

HTTPS → Application layer secure web communication

HTTPS is HTTP delivered through a TLS-encrypted channel. It protects requests and responses from interception or tampering and verifies the server’s identity. Modern browsers and APIs use HTTPS by default to ensure secure, trustworthy communication over the internet.

HTTP/3 + QUIC → Application layer over a modern transport

QUIC redesigns transport semantics on top of UDP. It implements its own encryption (TLS integrated), reliability, congestion control, and multiplexing; without inheriting TCP’s head-of-line blocking.

HTTP/3 sits atop QUIC and benefits from:

• Faster connection setup

• Independent streams

• Better behavior in lossy mobile networks

• Seamless migration when you switch Wi-Fi networks

FTP → Application layer file transfer

FTP enables basic file uploads and downloads using separate control and data channels. Because it sends credentials and data in plaintext, it’s now mostly replaced by secure options like FTPS or SFTP.

FTPS → Application layer secure file transfer

FTPS adds TLS encryption to traditional FTP, securing credentials and file contents while keeping the same command and data channel structure.

When something breaks here:

• If browsers warn about certificate errors, the TLS handshake is failing; often due to expired certificates, incorrect system clocks, or mismatched hostnames.

• If an API returns inconsistent data or unexpected codes, the issue is usually HTTP semantics, not the transport.

• If a connection falls back from HTTP/3 to HTTP/2, middleboxes may be blocking QUIC traffic or UDP entirely.

• If only certain pages break, expect HTTP caching, header size issues, or content negotiation problems rather than lower-layer failures.

Session and application issues tend to be precise: specific URLs fail, certain clients break, or behavior differs between environments.

5. Behind-the-scenes coordination → Application layer protocols

Some application-layer protocols don’t fetch webpages; they orchestrate backend behavior.

gRPC → High-performance service-to-service communication

gRPC lets microservices communicate as if they were calling native functions. It uses HTTP/2 or HTTP/3 streams and compact Protocol Buffers to reduce overhead. It’s type-safe, efficient, and ideal for modern distributed systems.

SSH → Secure remote administration

SSH provides secure remote login, file transfer, and tunneling. It’s the protocol behind almost every server deployment, debug session, and emergency patch.

When something breaks here:

• If gRPC calls fail while HTTP still works, client/server versions may be out of sync or message definitions incompatible.

• If SSH hangs on connect, packet filters or intrusion prevention systems may be delaying or dropping the handshake.

• If tunnels drop frequently, idle timeouts or NAT expiration are likely silently killing the session.

Issues at this layer often look like internal tooling failures rather than customer-facing outages.

Final thought

Networking feels complicated until you see it as a series of cooperating layers.

Each one has a narrow responsibility. Each one hands a problem to the next. And once you learn to read those boundaries, the internet becomes less magical and more mechanical; in the best possible way.

Because now, the next time someone says “The website is down,” you won’t guess. You’ll walk the layers, protocol by protocol, until the real culprit reveals itself.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Spec Driven Development with Rovo Dev

Presented by Atlassian

Specs only work if they stay connected to execution. Rovo Dev enables spec-driven development by grounding AI directly in Jira tickets, acceptance criteria, and Confluence docs. Instead of re-explaining requirements or copying context between tools, developers can generate, validate, and review code against the original spec. The result is less drift, fewer misunderstandings, and faster delivery without sacrificing correctness.

Read the article

Forward Proxy vs Reverse Proxy

Many developers confuse forward proxies with reverse proxies.

Both sit between clients and servers, both relay requests, and both can cache or filter traffic. But their purposes could not be more different.

The key difference is simple:

A forward proxy acts on behalf of the client.

A reverse proxy acts on behalf of the server.

Understanding this distinction is crucial when designing secure, scalable, and efficient systems. Let’s break it down.

Forward proxy

A forward proxy sits between clients and the open internet. The client configures its device or app to route traffic through the proxy, which then makes requests on the client’s behalf.

It’s like using a VPN service. Websites see the VPN’s IP, not yours.

What it does

• Hides identity → The server only sees the proxy’s IP, not the client’s.

• Enforces rules → Organizations can block or allow certain websites.

• Caches → Frequently accessed content is stored and reused.

• Bypasses restrictions → Users can access geo-blocked or censored sites.

• Centralizes access → All outbound traffic can be monitored in one place.

When it shines

• Filtering traffic → A school blocking access to certain websites.

• Anonymity and privacy → A journalist hiding their location.

• Reducing bandwidth costs → A corporate network caching OS updates.

Tradeoffs

• Limited protection → It doesn’t stop malware.

• Possible bottlenecks → All traffic passes through one point.

• Trust risk → The proxy can log or alter data.

In short, forward proxies are for controlling and anonymizing client traffic.

Reverse proxy

A reverse proxy flips the direction. It sits in front of servers and accepts inbound traffic from clients. To the outside world, the proxy looks like the server itself.

A reverse proxy works like a load balancer. Clients hit a single endpoint, but requests are distributed across multiple backend servers.

What it does

• Load balances → Distributes traffic across multiple servers.

• Shields servers → Hides backend IP addresses from attackers.

• Handles TLS → Manages encryption/decryption centrally.

• Caches and compresses → Reduces bandwidth and improves speed.

When it shines

• Handling traffic surges → An e-commerce site managing Black Friday traffic.

• Ensuring high availability → A SaaS provider maintaining uptime.

• Centralizing TLS termination → A company offloading encryption from its backend servers.

Tradeoffs

• Single point of failure → If the proxy goes down and isn’t replicated, the entire service becomes unreachable.

• Latency overhead → Every request takes one extra network hop through the proxy.

• Operational complexity → Proxies require careful setup, tuning, and monitoring to avoid becoming bottlenecks or attack targets.

In short, reverse proxies are for scaling, securing, and managing server-side infrastructure.

Recap

Choosing between a forward proxy and a reverse proxy comes down to where your challenge lies. If your concern is outbound traffic; privacy, control, or bypassing restrictions. Use a forward proxy.

If it’s inbound traffic; scaling servers, shielding infrastructure, or centralizing SSL. Use a reverse proxy.

The right choice depends not on the technology itself but on which side of the network you need to strengthen.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Refactor with Context, Not Guesswork

Presented by Atlassian

Rovo Dev is Atlassian’s context-aware AI agent for the entire SDLC. One of the use cases where it excels is refactoring. It helps teams plan changes, generate consistent multi-file updates, and review pull requests against Jira acceptance criteria and Confluence context. Learn how Atlassian engineers used Rovo Dev and AI-driven workflows to safely refactor a large monorepo.

Read the article

Breaking Down Domain-Driven Design

Early systems succeed because everyone shares the same mental model.

Late-stage systems fail because that model fractures.

Teams grow, terminology drifts, and the code becomes a poor translation of the business. At that point, speed drops even if your infrastructure is flawless.

Domain-Driven Design targets that exact moment of failure.

The real problem DDD solves

In complex products, the hardest part is not the codebase. It’s agreeing on what the business is actually doing.

Domain-Driven Design (DDD) forces the domain into the foreground.

You model the business concepts directly, so the code reads like the business operates.

That only works if you draw boundaries and protect them. Otherwise you get one giant model that tries to mean everything.

Five core patterns that make DDD work

1) Ubiquitous Language

Most teams think they share vocabulary.

They usually don’t.

In meetings, everyone nods when someone says “order,” “account,” or “encounter.” In code, those same words quietly drift into something else.

Ubiquitous language is a shared vocabulary that developers and domain experts use everywhere: meetings, tickets, docs, and code.

You stop translating “business terms” into “engineering terms.”

Pros:

• Shared meaning → Reduces back-and-forth and avoids “same word, different idea.”

• Readable code → New engineers learn by reading domain names, not glue code.

• Better decisions → Clear terms expose missing rules and edge cases.

Cons:

• Upfront effort → Workshops and alignment feel slow early.

• Requires discipline → Sloppy naming breaks the whole benefit.

2) Bounded Contexts

Shared language breaks down at scale.

Not because people are careless, but because the business is not one thing.

Bounded Contexts accepts that reality.

A bounded context is a clear boundary where a particular model and language apply. Outside that boundary, the same word can legitimately mean something else.

In an e-commerce system, “Product” in Inventory can mean descriptions and pricing, while “Product” in Order Management can mean a purchasable line item.

Bounded contexts prevent your system from becoming one giant argument about what a “product” really is.

Pros:

• Clear boundaries → Prevents a single model from becoming a “big ball of mud.”

• Team ownership → Lets teams evolve independently with fewer collisions.

• Safer change → Local refactors stay local because meanings don’t leak.

Cons:

• Integration work → You must design how contexts talk and what they share.

• Boundary mistakes → Wrong splits create friction and duplicated logic.

3) Model identity and value (Entities + Value Objects)

Inside a bounded context, you model the domain with entities and value objects.

An entity is defined by identity and lifecycle; like a Customer that stays the same “person” even if their address changes.

A value object is defined only by its values and is often immutable; like Money or Address.

Pros:

• Clear intent in code → “Identity” and “attributes” stop being mixed together.

• Fewer accidental bugs → Value objects reduce side effects because they are treated as pure descriptive values.

• Better tests → You can test domain behavior directly, because it lives where the domain lives.

Cons:

• More types upfront → You will write more domain code early, because clarity has a cost.

• Requires discipline → If you slip into “anemic models” (data-only objects), you lose most of the value.

4) Enforce consistency with Aggregates

An aggregate is a cluster of domain objects treated as one consistency boundary, controlled by an aggregate root.

Outside code talks to the root, not the internal parts, so business rules stay centralized and enforceable.

Pros:

• Fewer consistency bugs → Invariants live in one place, not duplicated across services and controllers.

• Clear transaction boundaries → Updates don’t cross aggregate boundaries, which simplifies correctness.

• Better maintainability → Code reads like “business operations,” because that’s what the aggregate exposes.

Cons:

• You must choose boundaries carefully → Oversized aggregates become bottlenecks; undersized ones leak invariants.

• Cross-aggregate workflows need orchestration → Some workflows become event-driven or application-layer-led by necessity.

5) Decouple workflows with Domain Events + keep persistence out with Repositories

A domain event captures something meaningful that happened in the domain, usually named in past tense like OrderPlaced or PaymentCompleted.

Inside one context it models a business fact; when published outward it often becomes an integration event used to coordinate other contexts.

That’s how you let Inventory react to OrderPlaced without letting Inventory reach into Order’s internal model.

A repository is a domain-facing interface for loading and saving aggregates (e.g. OrderRepository.save(order)), hiding database details from the domain layer.

Pros:

• Lower coupling → Events let systems react without turning everything into direct dependencies.

• Cleaner domain model → Repositories keep persistence concerns out of business logic.

• Easier evolution → You can change databases or infrastructure with less impact on the core model.

Cons:

• Event-driven complexity → Failures can surface later, and you have to think carefully about cross-context consistency.

• More upfront work → Modeling + workshops can feel slow early on, because you’re buying long-term clarity.

When to choose DDD

DDD pays off most when understanding the domain is the hard part, not just the technical build.

Use these decision checks:

• Domain complexity is high → You have lots of rules, exceptions, and nuanced workflows, so modeling reduces chaos.

• The domain is core to business value → You invest because correctness and flexibility create advantage.

• Many teams touch the system → Context boundaries help teams move independently because contracts stay explicit.

• The system will live for years → A strong model acts like documentation because it preserves business knowledge in code.

When NOT to use DDD

DDD struggles when the problem is simple.

If your app is mostly CRUD, it adds layers and coordination you don’t need.

If you cannot get sustained access to domain experts, DDD becomes guesswork. And guesswork hardens into code faster than you think.

And if the project is short-lived or low-impact, the modeling cost rarely returns value.

Final thoughts

You don’t adopt DDD by adding patterns.

You adopt it by removing ambiguity.

Start with language, because words drive everything else. Split contexts before you share a model “for convenience.” Keep invariants inside aggregates, not scattered across services. Use events when the business thinks in milestones and outcomes.

If understanding breaks before performance does, DDD is the right tool.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

AI Gets Better When Work is Specialized

Presented by Atlassian

Planning, coding, testing, and reviewing are different kinds of work. When one AI tries to do everything, quality drops. Rovo Dev addresses this with focused subagents that operate inside the terminal, reasoning over Jira, Confluence, and repo history via Atlassian’s Teamwork Graph. Each agent owns a narrow responsibility, enabling workflows that scale without losing context or control.

Learn more here

Bloom Filters Clearly Explained

You don’t notice it, but your system wastes more time proving what isn’t there than what is.

Every cache miss, every 404, every “not found” database query costs real CPU, I/O, and bandwidth; all to confirm a negative.

That’s where Bloom filters quietly shine.

They don’t store data; they store possibility. Instead of asking “is this in the set?” and paying full price every time, you ask a probabilistic gatekeeper first.

It answers instantly: definitely not or maybe, go check.

The trick is in accepting a sliver of uncertainty to save massive compute.

When “no” becomes the most expensive answer

Every large system eventually asks the same question: does this exist?

A URL, a user ID, a cache key, a file chunk; all get checked millions of times per second.

That check feels cheap, until you multiply it by scale.

Take a web crawler. It must avoid re-fetching URLs it’s already seen. The obvious solution is to keep every URL in a hash set. Simple, until “every” means billions. Even with compact encoding, that’s terabytes of memory just to remember what you’ve touched. Skip the memory, and you waste bandwidth re-downloading duplicates instead.

At scale, “no” dominates.

Most URLs are new. Most cache lookups miss. Most database keys aren’t there. Yet every query costs the same as a hit: full network calls, disk reads, and CPU time just to confirm nothing exists.

What you really need isn’t a full record of the past, just a cheap way to say “definitely not.”

You don’t care which URLs you’ve seen, only whether something could have been seen before.

Bloom filters help turn expensive “no” answers into near-free ones. What you trade up is a small, controllable chance of being wrong for a massive gain in speed and space.

How it works

A Bloom filter answers one question fast: could this exist?

It does it without storing the item, just a few bits of evidence that it might have been seen before.

Here’s how it works:

1. Start with an empty bit array.
   Picture a long row of bits, all set to 0. This is your blank canvas.

2. Hash the input several times.
   Each item (a URL or user ID) is fed into a number of different hash functions.
   Each hash produces an index which is a position in the bit array.

3. Set those bits to 1.
   For every hash output, that bit becomes 1. You never store the item itself, just the pattern of bits it activates.

4. To query, repeat the hashes.
   When you check if something exists, you hash it again across all hash functions and see which bits it maps to.

   • If any of those bits are 0 → the item definitely doesn’t exist.

   • If all are 1 → it might exist.

A Bloom filter never gives false negatives: if you inserted it, it will always return “maybe.”

But it can give false positives.

The more items you insert, the more bits flip to 1, and the higher the chance that unrelated items collide.

The trick is that you can tune this probability by the number of hash functions you use.

Each hash function sets one more bit in the array. Too few, and the filter doesn’t mark enough spots; it forgets what it’s seen and returns more false positives. Too many, and it flips too many bits; the array fills up faster, and collisions rise, again increasing false positives.

There’s a sweet spot where the filter stays balanced: enough hashes to remember patterns, but not so many that everything looks like a match.

That balance is what makes Bloom filters powerful. You decide how much uncertainty you can afford, and trade a bit of it for massive savings in efficiency.

Where they’re used

You’ve probably relied on Bloom filters without realizing it. They hide inside the systems that need to skip pointless work fast.

• Databases → Engines like Cassandra, LevelDB, and Bigtable use Bloom filters to skip disk reads for keys that don’t exist. This cuts random I/O and keeps “not found” queries cheap.

• Web crawlers → Track which URLs have been seen before. Instead of storing billions of strings, a Bloom filter flags duplicates in memory and saves massive bandwidth.

• Browsers and security tools → Chrome Safe Browsing and spam filters use them to pre-check URLs or sender lists locally, calling a remote service only when there’s a “maybe.”

• Distributed caches → Systems like CDN edge caches use them to decide if an object might exist on another node before sending a request.

• Stream processors → Platforms like Kafka Streams and Flink use them to drop duplicate events before aggregation or storage.

When not to use a Bloom filter

Bloom filters are powerful, but they’re not for every job.

They work best when most lookups miss and precision doesn’t matter. When the cost of being wrong outweighs the savings, the same trick that speeds you up can quietly hurt you.

Here’s when a Bloom filter isn’t worth that cost:

• You need exact answers → Bloom filters always allow false positives. If a wrong “maybe” breaks correctness (like in billing, access control, or authentication) use an exact set or index instead.

• You need frequent deletions → Standard Bloom filters can’t remove items. Once a bit is set, it stays set. Counting or Cuckoo filters solve this, but with more memory and complexity.

• Your dataset is tiny → If you’re only tracking thousands of items, the math doesn’t pay off. A normal hash set fits in memory and stays simpler to maintain.

• Your data grows unpredictably → Bloom filters must be sized in advance. Exceed that capacity and false positives spike fast. Use scalable variants if growth is unbounded.

Putting it into practice

Bloom filters make sense when the goal is speed, not certainty.

They shine when most lookups miss, when wasted work costs more than a small chance of being wrong, and when you can live with “maybe” as an answer.

Bloom filters remind us that at scale, certainty is overrated.

They don’t try to know everything; just enough to skip what doesn’t matter.

When most of your system’s time goes to proving nothing exists, a fast “maybe” is all you need.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Spec Driven Development with Rovo Dev

Presented by Atlassian

Specs only work if they stay connected to execution. Rovo Dev enables spec-driven development by grounding AI directly in Jira tickets, acceptance criteria, and Confluence docs. Instead of re-explaining requirements or copying context between tools, developers can generate, validate, and review code against the original spec. The result is less drift, fewer misunderstandings, and faster delivery without sacrificing correctness.

Explore now

Database Types Clearly Explained

Choosing a database feels like a schema problem. It usually isn’t.

It’s a question problem.

If your system needs updates to either fully succeed or fully fail, but your database doesn’t guarantee that, you’ll constantly deal with subtle bugs. If you need to search by meaning but your database only supports exact matches, you’ll end up adding another system later.

One rule cuts through the noise: pick the database that’s built for the hardest question you ask most often.

“All-or-nothing updates” → Relational and Distributed SQL

Some systems live or die on one requirement: multiple changes must succeed together, or not at all.

That’s what ACID transactions provide (Atomic, Consistent, Isolated, Durable).

Relational databases give you that reliability plus rich SQL joins and aggregations. The trade-off is that traditional relational systems often scale up more naturally than they scale out, and schema changes can be slower because the structure is fixed.

Distributed SQL keeps the same promise (SQL + ACID) but spreads data across nodes with consensus protocols so the database still behaves like one logical system. You gain horizontal scale and often multi-region replication, but you pay with network latency and operational complexity, because coordination across nodes is real work.

When not to use this: avoid relational databases for workloads dominated by key-based lookups or large append-only streams, because the extra guarantees add unnecessary overhead.

“Give me the whole thing as one object” → Document databases

Sometimes the core need is simple: store data in the same shape you read it.

When the read path is “load a user profile” or “load a product,” document databases are built for that.

A document database stores each record as a self-contained document (often JSON) with no fixed schema, so different documents can evolve independently. It’s a strong fit for nested data because embedding reduces the need for joins.

For example, a product catalog where one product has 5 attributes and another has 50 is naturally “document-shaped,” because forcing it into a fixed table often creates sparse columns and migrations.

When not to use this: avoid document databases when your core queries are “join across many collections by relationship,” because you’ll rebuild joins in code and lose the simplicity you came for.

“Get me the value for this key, fast” → Key-Value and In-Memory

When your workload is dominated by direct lookups, the winning option is usually a database that does almost nothing else.

That’s the idea behind key-value stores.

A key-value store is basically a distributed dictionary: you write a value using a key, and you read it back using that same key; all without schemas, joins, or complex queries.

That simplicity makes partitioning and throughput easier, which is why they show up in sessions, caching layers, and feature flags.

In-memory databases push the same idea further by keeping working data in RAM for extremely low latency. The trade-off is that memory is expensive and volatility is real unless you enable persistence; and even then, durability often trades off with performance.

When not to use this: a cache shouldn’t be your source of truth unless you’ve deliberately designed for persistence and consistency.

“Append forever, query by partitions” → Wide-Column and Time-Series

Some workloads aren’t about “updating records.”

They’re about ingesting a firehose of data and then asking time- or key-based slices like “give me everything for this device” or “show the last 10 minutes.”

That’s where wide-column and time-series databases fit best.

Wide-column stores (column-family stores) distribute huge tables across clusters and let each row have a sparse, flexible set of columns. They’re designed for very high throughput when you access data by a primary key (and often range scans on that key), but they require careful up-front modeling because you don’t get relational joins for free.

Time-series databases (TSDBs) specialize even further: time is the primary axis. They use append-only designs, time partitioning, compression, and features like retention policies and downsampling.

This is why they’re a natural fit for metrics, telemetry, and monitoring queries like “last 5 minutes” or “hourly average over a month.”

When not to use this: avoid wide-column and time-series databases for workloads that need frequent updates, complex joins, or multi-row transactions.

“How are these things connected?” → Graph databases

Graph databases make relationships first-class: nodes represent entities, edges represent relationships, and the database is optimized for traversing across those relationships.

If your queries look like “friends of friends,” “how are these things connected,” or “what paths exist between them,” graph databases can answer them quickly, while recursive joins can become slow and complex.

When not to use this: if you rarely traverse relationships, a graph database is overhead; use a simpler store that matches your access pattern.

“Find text like a human would” → Search and Vector databases

Search engines answer: “Which documents contain these terms, and which are most relevant?” using inverted indexes and relevance scoring.

They’re often eventually consistent (documents become searchable after a short delay), and they are not designed to be your transactional source of truth.

Vector databases answer a different question: “Which items are closest in meaning?”

They store embeddings (high-dimensional vectors) and use approximate nearest neighbor (ANN) indexes to avoid brute-force scans.

The speed comes from approximation, so you trade perfect accuracy for low latency.

When not to use this: search and vector databases aren’t a good fit when you need strong transactions, immediate consistency, or your single source of truth.

Final recap: match the database to the question

Every database type in this article exists for a reason.

None of them is “better” in general. Each one is optimized for a specific kind of question, and struggles when pushed outside that use case.

The fastest way to choose is to ignore features and focus on this instead:

What is the hardest question my system asks, every day, under load?

If you answer that honestly, the right database type usually becomes obvious; and you avoid months of work compensating for a bad fit in application code.

Many production systems don’t choose one database. They choose a primary database for correctness and secondary databases for access patterns:

• SQL for transactions

• Search for retrieval

• Cache for speed

• Vector for semantic discovery

That’s not over-engineering. It’s acknowledging that no single database answers every hard question well.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Coder: Secure Environments for Developers and Their Agents

Presented by Coder

Local development works because humans are trusted by default. AI agents inherit that same environment, with access to secrets, configs, and systems. Once agents do real work, teams need governed environments with clear boundaries. Coder brings developers and coding agents together on your infrastructure, open source and deployable anywhere, without compromising security or performance. It’s open source (113k stars) with a completely free Community Edition anyone can run.

Try it now

Top 5 Database Caching Strategies You Should Know

Caching is a core performance optimization technique.

Done well, caching reduces latency, protects your database, and improves throughput. Done poorly, it introduces stale reads, hidden consistency bugs, and memory waste that only shows up in production.

The problem is not whether to cache.
It’s how.

Different caching strategies exist because systems have different access patterns, consistency needs, and failure tolerances.

Let’s dive into five of the most widely used caching strategies.

1) Cache-Aside

In a cache-aside strategy, the application explicitly manages reads and writes to the cache.

It queries the cache first, and on a cache miss, it fetches data from the database, returns it, and updates the cache.

The cache only stores data after it’s been read by the app.

Pros:

• Full control → The application decides what to cache and when.

• Demand-driven → You only cache data that someone asked for.

• Flexible → Works well when access patterns are unpredictable.

Cons:

• Consistency is your responsibility → You must handle invalidation correctly.

• More application logic → Every read path needs cache awareness.

Cache-aside shines in systems where usage patterns are unpredictable or data freshness varies. However, beware that sloppy cache management can lead to stale reads or wasted memory.

2) Write-Through

Write-through caching updates the cache and the database at the same time.

Every write goes through the cache first, which then synchronously persists the change to the database. Reads always hit a warm, up-to-date cache.

Pros:

• Strong consistency → Cache and database never diverge.

• Simple reads → No stale data surprises.

• Predictable behavior → Fewer edge cases to reason about.

Cons:

• Higher write latency → Every write incurs the cost of writing to two systems (cache and database).

• Cache pollution → Cache can be flooded with infrequently read data.

Best used when data integrity and consistency are critical. Think e-commerce transactions or financial apps where stale reads aren’t an option.

3) Write-Behind (Write-Back)

Write-behind (or write-back) caching updates the cache first and the database later in the background. “Dirty” data is flushed based on a schedule or specific triggers like time or batch size.

Pros:

• Fast writes → The application doesn’t wait on the database.

• Reduced database pressure → Writes are smoothed and batched.

• High throughput → Ideal for write-heavy workloads.

Cons:

• Data loss risk → Risk of data loss if the cache node crashes before the database syncs.

• Harder debugging → Failures surface later, not at write time.

Write-behind is ideal for apps where slight delays in database updates are tolerable, like activity feeds, logs, or ephemeral data.

4) Read-Through

A read-through cache acts as a middle layer.

The application queries the cache directly, and on a miss, the cache fetches the data from the database, stores it in the cache, and returns it to the application.

Pros:

• Cleaner application code → No explicit cache logic on reads.

• Centralized behavior → Caching rules live in one place.

• Efficient for repeated reads → Hot data stays warm automatically.

Cons:

• Cold start penalty → Initial cache misses still hit the database, causing a slight delay.

• Staleness risk → Requires careful TTLs or invalidation rules.

This is a solid choice for read-heavy applications with stable data, like content libraries or product catalogues.

5) Write-Around

Write-around caching skips the cache during writes, sending data straight to the database.

Data enters the cache only when it’s read later. This means new or updated data isn’t cached until requested.

Pros:

• Lean cache usage → Avoids caching data that may never be read.

• Reduced churn → Write bursts don’t evict hot read data.

• Simple write path → No cache coordination during writes.

Cons:

• Guaranteed cache misses after writes → First reads after a write always miss the cache, causing read latency.

• Uneven read latency → Inconsistent performance if reads for new data spike unexpectedly.

Best for apps with frequent writes and infrequent reads, like analytics logs or activity events.

Choosing the right strategy

Each caching strategy comes with its own trade-offs. To decide which one fits your system, consider:

• Read vs write ratio → Is your workload read-heavy, write-heavy, or balanced?

• Consistency requirements → Can your system tolerate stale data, or is strict consistency a must?

• Operational complexity → How much logic are you willing to manage in the application vs the cache layer?

Understanding these trade-offs is key to making caching an enabler, not a liability.

Final thoughts

Caching is not just a performance optimization.

It is part of your system’s correctness model.

The right strategy reduces load, smooths traffic, and keeps latency predictable. The wrong one quietly accumulates risk until a cache eviction, deployment, or incident exposes it.

Match your caching strategy to your access patterns, failure tolerance, and consistency needs; and your system will scale calmly instead of nervously.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

Microsoft Foundry: The AI App and Agent Factory

Presented by Microsoft

Most teams can spin up Al demos. Far fewer can run agents reliably in production.

Microsoft Foundry is a unified Azure platform for building, optimizing, and governing Al agents at scale. It brings the widest selection of models on any cloud, data, multi-agent workflows, observability, and security controls together in one interoperable system, so teams can move faster without sacrificing openness, control, or security.

Learn more

Microservices Clearly Explained

Microservices are not “small monoliths.”

They change how software is built, deployed, and owned; for better and for worse.

Teams adopt them to move faster and scale more precisely, then discover they’ve also signed up for distributed complexity.

Knowing where that trade-off helps, and where it hurts, is what determines success.

What microservices really are

A microservices architecture structures an application as a collection of small, independent services.

Each service owns a single business capability, runs in its own process, and communicates with others over a network; usually HTTP APIs or asynchronous messaging.

The key distinction is not size. It’s ownership and independence.

• Each service maps to a real domain like payments, orders, or inventory

• One service can deploy independently

• Each service manages its own database or storage

Instead of one large codebase and database, you have many smaller ones that cooperate.

Updating a payment service no longer risks breaking user profiles. Scaling search traffic no longer forces you to scale checkout.

How microservices work in practice

In a microservices system, a single user action involves multiple service interactions.

Placing an order may trigger a chain of calls: inventory checks, payment authorization, order creation, notifications. These calls happen over HTTP or via asynchronous messages rather than in-memory function calls.

To make this all work, most systems introduce supporting infrastructure:

• An API gateway routes requests and hides internal structure

• Service discovery helps services find each other at runtime

• Message brokers decouple producers from consumers

• Centralized logging and tracing reconstruct request paths

The result feels less like a single application and more like a small distributed system inside your product. Every service is simple on its own. The complexity lives in the interactions.

That’s the trade: you break one big problem into many small ones, and then you have to make them cooperate reliably.

Benefits

Microservices solve real problems when systems and teams reach a certain scale.

• Independent scaling → Hot paths can scale without dragging the entire system along, which avoids over-provisioning and wasted resources.

• Faster deployment → Teams ship changes independently, reducing release coordination and lowering the risk of unrelated regressions.

• Fault isolation → A failure in one service degrades part of the product instead of taking everything down.

• Team autonomy → Small teams own services end-to-end, which reduces coordination overhead and increases delivery speed.

• Technology flexibility → Each service can use the language, database, or runtime that fits its workload best.

These advantages come from the same source: strong boundaries.

The architecture makes it hard for teams to accidentally couple themselves together.

Tradeoffs

Those same boundaries introduce new costs that don’t exist in a monolith.

• Distributed complexity → Network calls replace function calls, bringing latency, retries, and partial failures.

• Operational overhead → CI/CD, observability, and deployment automation become mandatory rather than optional.

• Testing difficulty → End-to-end behavior spans multiple services, making failures harder to reproduce and debug.

• Data consistency challenges → Transactions across services require eventual consistency and compensating actions.

• Duplication risk → Shared logic is harder to centralize, leading to repeated implementations across services.

Microservices don’t remove complexity. They move it from code structure into system behavior and operations.

When microservices make sense

Microservices are a solution to specific problems. Use them when those problems are real and persistent.

They fit best when you need:

• Uneven scaling → Some features receive far more traffic than others

• Frequent independent releases → Deployment speed matters more than simplicity

• Clear domain boundaries → Business logic splits cleanly into services

• Large or growing teams → Ownership needs to scale with people

• Strict resilience requirements → Partial failure is preferable to total outage

In contrast, small teams, early-stage products, or stable problem spaces often benefit more from a modular monolith. You can enforce boundaries without paying the distributed tax upfront.

When not to use microservices

Microservices fail most often when they are used too early.

Avoid microservices when:

• The product is early and still changing shape → Boundaries will shift, and splitting too soon locks in bad decisions.

• The team is small → Coordination costs outweigh deployment benefits.

• Operational maturity is low → Without strong automation and observability, failures become invisible and painful.

• The problem is simple → A modular monolith is easier to build, test, and reason about.

Starting with microservices too early often slows teams down. Sometimes it’s best to build a well-structured monolith first, then extracting services only when pain points become obvious.

Wrapping up

Most successful microservice systems started as monoliths.
Not because teams were cautious; but because they were deliberate.

A well-modularized monolith teaches you where the real seams are. Microservices then become a response to proven pressure, not an architectural guess.

Split when the pain is undeniable and the boundary is obvious.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, clear, and visual system design breakdowns straight to your inbox:

From Similarity to Relevance in Vector Search

Presented by MongoDB

MongoDB Atlas Vector Search now supports lexical prefilters, letting teams apply analyzed text filters (like fuzzy search and phrase matching) or geo constraints before vector similarity runs. The result is higher relevance, lower latency, and more reliable RAG and search at scale.

Learn more here

gRPC: What It Is and Why Teams Use It

Most teams reach for REST by habit.

It works, it’s everywhere, and every tool knows how to talk HTTP+JSON.

But once you have dozens of microservices calling each other thousands of times per request, REST can quietly become the bottleneck.

That’s where gRPC shows up with a very different model: “call a function on another service as if it were local,” and make it fast.

What gRPC (and RPC) really is

Before gRPC, there’s RPC.

Remote Procedure Call (RPC) is a model where you call a function that runs on another machine, but it feels local to the caller.

You call a method, pass arguments, and get a result back. The network exists, but it’s intentionally hidden so developers can think in terms of functions instead of sockets and packets.

gRPC is a modern, open-source RPC framework released by Google in 2015. It takes the RPC idea and standardizes how services define methods, exchange data, and communicate efficiently over the network.

Two building blocks drive almost everything you feel in practice:

• HTTP/2 → The transport protocol. It keeps a long-lived connection and supports multiplexed streams.

• Protocol Buffers (Protobuf) → The data format. It’s a compact binary serialization with a schema (a defined shape).

Together, these turn “call a service” into a strongly typed operation instead of a loosely structured document exchange.

How gRPC actually works

gRPC starts with a service definition. You describe your API in a .proto file by defining services (methods) and messages (request and response shapes).

This file is the contract. Both client and server are built from it.

From that contract, gRPC generates code:

• Client stubs → Methods you call like local functions.

• Server interfaces → Methods you implement with your business logic.

When a client calls a gRPC method, the flow is straightforward:

1. Serialize → The request is encoded into Protobuf’s binary format.

2. Send → The message travels over an existing HTTP/2 stream.

3. Dispatch → The server routes it to the correct method.

4. Execute → Your code runs.

5. Respond → The result is serialized and streamed back.

Because gRPC uses HTTP/2, many calls share one connection and run in parallel. A slow response doesn’t block faster ones behind it, which keeps tail latency under control.

Streaming uses the same mechanism:

• Server-streaming → One request, many responses over time.

• Client-streaming → Many requests, one final response.

• Bidirectional streaming → Both sides send messages independently.

Backpressure is built in. If one side slows down, gRPC slows the stream instead of piling up memory or threads.

The core idea is simple: define a strict contract, generate code from it, and move typed messages efficiently over a shared connection.

Where gRPC pays off

gRPC is a strong fit when you control both ends and you care about efficiency.

• High call volume microservices → Lower per-call overhead adds up fast at scale.

• Latency-sensitive graphs → Multiplexing + smaller payloads reduces tail latency pressure.

• Polyglot stacks → One .proto contract generates stubs across languages, reducing “JSON drift.”

• Service mesh environments → gRPC routes cleanly through modern proxies and is common in mesh control-plane protocols.

Tradeoffs you feel immediately

gRPC’s downsides are predictable, and they usually show up early on.

• Browser calls are not native → You often need gRPC-Web or a REST/JSON gateway for front-end use.

• Debugging is less “curl-friendly” → Binary payloads require tooling like grpcurl or GUI clients with schema access.

• Contracts tighten coupling → Clients must update generated code as schemas evolve, so versioning discipline matters.

• Infra must support HTTP/2 well → Some proxies and firewalls need explicit support or configuration.

How to Decide: Is gRPC a Fit for Your System?

Use gRPC when most of the following are true:

• You control both client and server → Internal microservices inside the same organization.

• You’re performance-sensitive → Many small calls per request, or very high QPS between services.

• You’re polyglot → Multiple languages across teams and services.

• You need streaming → Real-time updates, telemetry, chat, or continuous feeds.

• You want strict contracts → You care about compile-time guarantees and explicit schemas.

Stick to REST (or layer a REST gateway in front of gRPC) when:

• You expose public APIs to unknown clients.

• You want easy browser and curl-based experimentation.

Your main pain is clarity and discoverability, not raw latency or throughput.

In practice, most teams don’t choose one or the other; they split the responsibility.

• gRPC inside your network → Service-to-service, behind an API gateway or service mesh.

• REST/JSON at the edge → For browsers, partners, and mobile apps that prefer HTTP+JSON.

Recap

gRPC is not “REST but faster.”

It’s a different model: remote procedure calls over HTTP/2, using Protobuf contracts, with first-class support for streaming and strong typing.

That makes it excellent for internal microservices, high-performance backends, and real-time systems; as long as you’re willing to invest in schemas, tooling, and a slightly steeper learning curve.

If you’re hitting the limits of REST inside your system (too many chatty JSON calls, tricky real-time updates, or a messy polyglot codebase) gRPC is worth a serious look.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, visual, and simple-to-understand system design articles straight to your inbox:

Refactor with Context, Not Guesswork

Presented by Atlassian

Rovo Dev is Atlassian’s context-aware AI agent for the entire SDLC. One of the use cases where it excels is refactoring. It helps teams plan changes, generate consistent multi-file updates, and review pull requests against Jira acceptance criteria and Confluence context. Learn how Atlassian engineers used Rovo Dev and AI-driven workflows to safely refactor a large monorepo.

Read the article

API Gateway vs Load Balancer vs Reverse Proxy

API gateways, load balancers, and reverse proxies are often grouped together as the same kind of component.

They all sit at the edge, forward requests, and are introduced when systems need to scale.

That surface-level similarity hides an important truth: each one exists to solve a different problem, and using the wrong component quietly creates new failure modes. Many production issues start here; not with bugs, but with blurred responsibilities at the edge.

Knowing how to categorize each component is what keeps your edge simple instead of fragile.

The One Decision Each Component Makes

If you remember nothing else, remember the verb each one optimizes for:

• Load balancer → distribute traffic across identical copies of the same service.

• Reverse proxy → forward + optimize traffic to one or more backends, often with caching, TLS termination, and routing rules.

• API gateway → govern API traffic across many services: auth, rate limits, routing, transformations, and sometimes aggregation.

In practice, it’s like standing at a building entrance:

• A load balancer points you to the least busy elevator (same destination, different car).

• A reverse proxy sends you to the right department and might hand you a cached brochure instead of interrupting staff.

• An API gateway does that and checks your badge, enforces rules, and sometimes translates what you asked for into what the building understands.

What Each One is Good at (and Why)

API Gateway: “One door” for many APIs

An API gateway is a single entry point for clients that need to call multiple backend services. It commonly acts like a reverse proxy plus API management features.

Typical jobs it handles well:

• Authentication + authorization → Enforce consistent access rules once, not in every service.

• Rate limiting + quotas → Stop one client from bringing down your fleet.

• Request/response transformation → Convert formats, add/remove headers, normalize shapes.

• API composition → Fan out to multiple services and aggregate a single response.

• Protocol translation → Bridge HTTP/JSON clients to gRPC (or other) services.

Tradeoffs to plan for:

• Added hop + processing → Auth checks and transforms add latency (often small, but real).

• Operational criticality → If it fails, clients can lose their only entry point unless you build redundancy.

• Config complexity → Lots of power means lots of ways to misconfigure routing/policies.

Load Balancer: “Spread work so nothing overloads”

A load balancer distributes incoming traffic across multiple servers so no single instance becomes the bottleneck. It usually performs health checks and avoids unhealthy instances, which is how it improves uptime.

There are two important categories:

• Layer 4 (L4) → Routes using IP/port; fast and protocol-agnostic, but content-blind.

• Layer 7 (L7) → Understands HTTP and can route by path/headers/cookies.

What it’s great at:

• High availability → Prevents traffic from being sent to unhealthy nodes.

• Horizontal scaling → Add/remove instances and keep one stable client-facing endpoint.

• Basic edge duties → TLS termination, sticky sessions (when you’re stuck with state).

Common gotchas:

• “One LB instance” thinking → A load balancer can also be a single point of failure unless made redundant.

• Expecting API policy features → It’s not designed for per-endpoint auth, transforms, or analytics.

Reverse Proxy: “Front the servers, hide them, and speed things up”

A reverse proxy sits in front of backend servers, forwards requests on their behalf, and makes the backend topology invisible to clients. It typically operates at the application layer (L7) and can make routing decisions based on request content.

Where they add the most value:

• TLS termination → Handle certificates centrally so app servers don’t have to do the encryption work.

• Caching + compression → Serve common responses fast, reduce backend load and bandwidth.

• Security shielding → Hide backend IPs and filter obvious bad traffic before it lands.

• Routing + rewriting → Map clean external paths to messy internal routes.

• “Good enough” load balancing → Many can round-robin across a small pool of backends.

Its limits are mostly scope:

• It’s not full API management → You won’t get rich per-consumer quotas, developer portal concepts, or deep API analytics by default.

• It’s still a choke point → Needs redundancy and careful config like anything in the hot path.

Quick Comparison: Same Edge, Different Responsibilities

When these components get confused, it’s usually because they’re compared by features. A clearer comparison looks at what responsibility each one owns. The table below strips things back to first principles so you can see where overlap is intentional; and where it’s a smell.

Seen this way, the overlap makes sense.

All three sit at the edge, but each one draws the boundary in a different place.

Choosing the Right Tool (and Avoiding the Wrong One)

Most edge mistakes come from reaching for a tool because it’s available, not because it matches the problem.

These components overlap in capability, but they should not overlap in responsibility.

Start by being clear about your context and requirements:

• Many replicas of the same service → use a load balancer, because the core problem is distributing work safely and evenly.

• One domain with multiple backends and performance concerns → use a reverse proxy, because it can route, cache, and protect without API-level policy overhead.

• Many services behind a single public API → use an API gateway, because clients need consistent rules, simplified access, and sometimes response aggregation.

These tools are not mutually exclusive.

Larger systems often layer them deliberately: a gateway for API policy, a load balancer for instance-level distribution, and reverse proxies where caching or TLS termination reduce pressure downstream.

The key is intent.

Each component should earn its place by owning a single, clear responsibility. If it can’t explain why it’s there, it’s probably doing too much; or shouldn’t be there at all.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, visual, and simple-to-understand system design articles straight to your inbox:

Accelerate Agents to Production With Amazon Bedrock AgentCore

Presented by AWS

Accelerate agents to production with composable services that work with any framework, any model. AgentCore is an agentic platform for building and running production-grade AI agents, securely at scale, with no infrastructure to manage. Now with complete control over agent actions through policies, and continuous evaluations of real-world agent behavior to boost quality.

Check it out

Load Balancing Algorithms Clearly Explained

Load balancing is not about spreading traffic evenly. It’s about spreading work safely.

Those only mean the same thing when requests cost about the same and servers behave about the same; which is almost never true in production.

Once you add long-lived connections, uneven instance sizes, caches, or sticky sessions, an “even” distribution can still overload a single backend.

The algorithm you choose quietly decides whether your system stays smooth or starts to buckle.

What a Load Balancer Actually Optimizes

Every algorithm answers one question: “Which backend is best according to this metric?”

But the signal it picks isn’t always the one you assume.

Algorithms typically optimize for things like:

• Request count → “Spread requests evenly across all servers”

• Concurrency → “Send new requests to the server handling the least work right now”

• Latency → “Choose the server that’s responding the fastest”

• Locality → “Keep a user on the same server for state or cache reasons”

• Stability → “Reduce how much traffic moves around when servers scale up or down”

This is where misunderstandings start. “Even traffic” doesn’t mean “even load”.

Static vs Dynamic Algorithms

Most algorithms fall into two buckets:

• Static algorithms → Use a fixed pattern (like rotation or weights) and ignore live load.

• Dynamic algorithms → Adapt based on runtime signals (like active connections or response time).

Round robin (static)

Round robin walks the server list in order, handing out requests one by one.

It’s “fair by count” when servers are similar and requests cost about the same.

When that assumption breaks, it still deals the cards evenly; even if one player keeps getting all the high cards.

• Optimizes for → Even request count

• Works when → Requests have similar cost and servers are similar

• Fails when → Some requests are much heavier than others

• Failure signs → One server overloads even though distribution looks “fair”

Weighted round robin (static)

Weighted round robin keeps the same predictable cycle, but higher-weight servers show up more often (weight = a configured capacity hint).

This works well for mixed hardware or controlled rollouts (like a 90/10 canary), but it still ignores real-time slowness.

• Optimizes for → Even request count scaled by weights

• Works when → Weights reflect real capacity and stay up to date

• Fails when → Capacity changes but weights don’t

• Failure signs → Overloaded “high-weight” nodes or skewed rollout splits

Least connections (dynamic)

Least connections sends new work to the server with the fewest active connections (in-flight requests).

It shines when sessions vary in length (downloads, WebSockets, streaming), because a server stuck on long sessions stops attracting more.

It can still misbehave with uneven hardware, because “5 connections” can mean “fine” on one host and “dying” on another.

• Optimizes for → Lowest active concurrency

• Works when → Connection count correlates with work

• Fails when → Long-lived sessions pin concurrency on one backend

• Failure signs → WebSockets/streaming endpoints cause uneven load

Weighted least connections (dynamic)

Weighted least connections scales “how loaded” a server looks by its capacity (roughly: connections ÷ weight).

You keep the benefits of load awareness, while acknowledging that bigger servers should carry more concurrent work.

It’s a strong default when you have mixed instance sizes.

• Optimizes for → Spreading concurrent load proportional to capacity (connections ÷ weight).

• Works when → Weights match real capacity and connections roughly track load

• Fails when → Requests vary wildly, weights are wrong, or the bottleneck isn’t connections (CPU/DB/latency).

• Failure signs → Hot instances, rising tail latency, and distribution shifts a lot when you tweak weights/scaling.

Least response time (dynamic)

Least response time favors the backend that has responded fastest recently (often using rolling averages). This can improve user latency and avoid slow-but-“low connections” hosts. It can also oscillate: traffic floods the “fast” host, it slows, traffic shifts, and you get waves.

• Optimizes for → Lower observed latency

• Works when → Measurements are stable and you smooth adjustments

• Fails when → Noise causes traffic to swing between servers

• Failure signs → Oscillation: the “fast” host becomes slow after being flooded

Resource-based balancing (dynamic)

Resource-based routing uses CPU/memory/bandwidth signals (usually from agents) to send traffic to the host with the most headroom. It can handle cases where connection counts lie (background jobs, noisy neighbors). It’s also the most operationally complex: agents, freshness, scoring, and stability are your responsibility.

• Optimizes for: Keeping instances out of saturation by routing to where there’s real headroom

• Works when: Your signals are accurate and comparable, update fast enough, and you smooth changes

• Fails when: Metrics are laggy/noisy, you’re measuring the wrong bottleneck

• Failure signs: Traffic “flaps” between hosts, sawtooth utilization, hot spots despite balancing, rising tail latency, and bursts of errors during metric spikes.

Power-of-Two Choices

In big systems you don’t have one load balancer.

You have many: ingress pods, proxies, threads.

That’s why “always pick the global least-loaded host” can backfire: everyone sees the same winner and stampedes it.

Power of Two Choices (P2C) fixes this by sampling two servers at random and choosing the better one (often fewer in-flight requests). You get most of least-connections’ benefit with way less coordination, and less herd behavior because each balancer sees different pairs.

• Random sampling → Low overhead, no full pool scan

• Local comparison → Still favors less-loaded servers

• Distributed stability → Fewer “everyone picks the same winner” moments

• Fails when → Load isn’t reflected by the metric you compare (e.g., CPU/queue/DB is the bottleneck), or the pool is so small that random picks don’t give you real choice.

• Failure signs → Uneven hot spots, tail latency spikes, and “random” oscillation where different instances take turns getting slammed.

Session Affinity

Some services need session affinity (also called stickiness): repeated requests from the same “user” must land on the same backend because state lives in memory or local cache.

Source IP hash (affinity)

Source IP hash picks a backend by hashing the client’s IP. It’s simple and stateless, and it works even when you can’t use cookies. But it can create severe skew if many users share an IP (corporate NAT, mobile carriers) or if one client generates far more traffic than others.

• Optimizes for → Session affinity without extra state

• Works when → Client IPs are diverse and roughly represent users

• Fails when → Many users share IPs (NAT, carrier networks, proxies)

• Failure signs → One backend becomes a hot spot

Consistent hashing (affinity with stable scaling)

Simple IP-hash can reshuffle lots of clients when servers come and go. Consistent hashing cuts that churn: keys live in a stable hash space, so only a small slice moves when the pool changes (e.g. ring hash/virtual nodes, Maglev tables).

It’s great for caches and sharded state because locality survives scaling.

The catch: it won’t automatically relieve a hot server; hot keys keep landing on the same host unless you step in.

• Optimizes for → Stable mapping as servers come and go

• Works when → You want cache locality or sharded state with low churn

• Fails when → A few keys dominate traffic

• Failure signs → One shard stays under pressure because one key stays hot forever

How to Recognize When Your System is Fighting the Balancer

Most balancing problems show up as repeating symptoms. Treat them like fingerprints.

• One instance is always hotter → The algorithm equalizes counts, not work

• Latency spikes come in waves → The routing reacts too quickly to “fastest” signals

• A subset of users is consistently slow → Stickiness pins them to a weaker backend

• Canary looks right in aggregate but bad in practice → Weights or affinity skew who lands where

• Cache hit rate drops after scaling → Key-to-server mapping churns more than expected

• Incidents worsen after retries kick in → Retries amplify imbalance because they create extra traffic

These symptoms don’t prove the algorithm is wrong. They prove its assumptions need checking.

How to Pick an Algorithm for Your Context

Most teams don’t “implement” load balancing; they pick a config, accept a default, or choose a platform.

The trick is matching the algorithm to what your workload actually does:

• Need affinity? → Use consistent hashing or sticky routing. Keeping state local cuts cache misses and avoids constant warm-ups.

• Requests vary in duration? → Use least-connections or P2C. Concurrency tracks contention better than raw request counts.

• Mixed instance sizes? → Add weights or use weighted least-connections. Heterogeneous fleets need proportional distribution.

• Many balancers in play? → Prefer P2C-style strategies. They reduce herd behavior without global coordination.

• Chasing user-perceived latency? → Use latency-aware routing carefully. It can cut tail latency, but it can also oscillate if you react too fast.

Wrapping Up

Load balancing is choosing where work piles up.

“Even traffic” is only safe when requests and servers are boringly uniform (they aren’t).

The right algorithm is the one whose signal matches your bottleneck, and whose failure mode you can live with.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, visual, and simple-to-understand system design articles straight to your inbox:

A Model-Driven Approach to Building and Deploying Agents

Presented by Amazon

Strands Agents empowers you to create intelligent agentic systems that leverage reasoning models to make dynamic decisions and adapt to unexpected situations. You can now build agents in TypeScript, run agents on edge devices, steer agents through modular prompting, and verify agent performance with evaluations.

Start Building

What a CI/CD Pipeline Really Looks Like

Shipping code shouldn’t feel like defusing a bomb, yet many teams treat deploys that way.

Merges slow down, nerves go up, and one bad release can freeze progress for days.

CI/CD exists to make releases predictable and dull; something you do without thinking, not something you plan your week around.

When you build a healthy pipeline, “deploying” stops being an event and becomes an ordinary part of writing software.

Continuous Integration vs Continuous Delivery vs Continuous Deployment

Before we dive in, let’s nail the terminology first:

You can think of it as three layers:

• Continuous Integration → Every commit builds and runs automated tests so the main branch stays healthy.

• Continuous Delivery → Every passing build is automatically packaged and sent to a staging or test environment.

• Continuous Deployment → Every passing build flows into production without a manual button press, relying on strong tests and monitoring.

The pipeline is the same; the only difference is where you keep a human in the loop.

How a CI/CD Pipeline Actually Works

A typical pipeline is just a scripted checklist the computer runs for you:

1. Version control trigger
   A push to the main branch or a pull request hits GitHub/GitLab. That event triggers your CI server.

2. Build and quick tests
   The pipeline checks out the code and runs your build and fast-feedback test suite:

   • Unit tests → Verify small pieces of logic behave correctly on their own

   • Smoke tests → Small, fast set of checks to confirm the system is “basically working.”

   • Static analysis → Catch code issues early through linting, type checks, and security scans

3. If anything fails, the pipeline stops and flags the commit.

4. Artifact and registry
   A successful build produces an artifact (JAR, Docker image, npm package) and stores it in a registry so the same thing gets deployed everywhere.

5. Staging deployment
   Delivery kicks in:

   • Apply infrastructure-as-code → Spin up or update a staging environment

   • Run slower tests → Integration tests, performance checks, and end-to-end user flows

6. Promotion to production

   • With Continuous Delivery → A human reviews changes and clicks “deploy”

   • With Continuous Deployment → The pipeline auto-deploys when staging is green

7. Mature setups add safe rollout patterns

   • Blue-green → Run old and new versions side by side, then flip traffic

   • Canary → Send a small percentage of traffic to the new version, watch metrics, then ramp up if healthy

8. Monitoring and rollback
   After deployment, dashboards and alerts watch error rates, latency, and key business metrics. If something looks off, you either:

   • Roll back → Redeploy the previous artifact

   • Roll forward → Ship a quick fix through the same pipeline

The result: deployments are just another pipeline run, not a manual ritual.

The Benefits of CI/CD

CI/CD is not just “faster releases.” It changes how your team works.

• Faster feedback → Within minutes, you see if a change broke something which makes fixes cheaper and less stressful.

• Fewer production surprises → Every change passes the same set of tests and staging checks before it reaches users.

• Safer, smaller changes → You ship small increments instead of huge batches, so debugging is easier when something goes wrong.

• Less manual glue work → No more copying artifacts around, clicking through release runbooks, or guessing which commit is in prod.

• More confidence to refactor → A strong pipeline acts as a safety net, so you can clean things up instead of living with code rot.

Over time, teams that lean into CI/CD deploy more often, recover from incidents faster, and spend more time writing features instead of wrestling with release mechanics.

The Hidden Costs and Trade-Offs

You do pay for this automation. The costs are real, just usually front-loaded.

• Pipeline setup overhead → You need to wire together CI tools, artifact storage, deployment scripts, and infrastructure-as-code because the pipeline itself becomes critical infrastructure.

• Testing discipline → CI/CD is only as good as your tests. Flaky or shallow tests give you a false sense of safety and quickly erode trust in the pipeline.

• Cultural change → Dev, QA, and Ops must share ownership of quality and deployment because CI/CD breaks old “throw it over the wall” habits.

• Toolchain complexity → Different teams with different tools can accrete a mess of half-integrated pipelines that are hard to reason about.

When not to go all-in

There are cases where a full, multi-stage, fully automated pipeline is overkill:

• Tiny prototype → You’re validating an idea with one developer and rare deployments.

• Extremely stable legacy system → It ships once a year and barely changes.

• Heavy external certifications → For example, embedded medical devices where each release needs formal sign-off.

In those situations, basic CI (build + unit tests on every commit) is still worth it because it keeps even rare releases safer, but you can delay the fancy rollout machinery.

How to Adopt CI/CD Without Burning Out the Team

You do not need to jump straight to “deploy to prod on every commit.” A sane path looks like this:

• Start with CI → Automate build + unit tests on each push because this gives immediate value and surfaces integration issues early.

• Add staging delivery → Automatically deploy the CI-built artifacts to a staging environment because this ensures every release candidate is tested the same way.

• Gradually automate production → Introduce blue-green, canary, and manual approvals inside the pipeline before flipping to full continuous deployment.

The key mindset is to treat the pipeline as a product.

You invest in it, maintain it, and keep it fast and reliable because it becomes the backbone of how your team ships.

If “release day” still feels scary, that’s a signal your CI/CD story isn’t finished yet.

👋 If you liked this post → Like + Restack + Share to help others learn system design.

Subscribe to get high-signal, visual, and simple-to-understand system design articles straight to your inbox: