LUC #78: CAP Theorem Explained In Simple Terms

Plus, Kafka vs Bufstream, how does SQL execution order work, and why is it so important?, and HTTP/2 v HTTP/3 — what's the difference?

This week’s issue brings you:

READ TIME: 5 MINUTES

Thanks to our partners who keep this newsletter free to the reader.

Amazon Q Developer can handle your biggest projects. Write, test and review code with ease with an AI coding assistant by your side. Amazon Q Developer is built to take on ambitious projects, taking you from idea to production code.

Its “/dev” agent helps you implement new features across multiple files.

Its @workspace chat means you can ask questions and get answers about your entire project context.

And “/doc” helps you generate documentation as you go.

CAP Theorem Simplified

Do I keep the system available even though the data is incorrect?

Or do I wait for the data to become consistent throughout the system, even if it means the system is unavailable in the meantime?

This is a classic conundrum faced in distributed systems.

It’s the core dilemma that CAP theorem explores.

Let’s dive in.

CAP Theorem explained

The CAP theorem is a fundamental principle in distributed computing that outlines the trade-offs a distributed system must make when dealing with three key properties—consistency, availability, and partition tolerance.

CAP theorem asserts that a distributed system cannot simultaneously provide consistency, availability, and partition tolerance.

Consistency ensures all nodes display the same data simultaneously, which is crucial for systems that need all clients to receive up-to-date and accurate information.

Availability means that every request (read or write) receives a response, even if it’s not the most recent write. The system remains operational and responsive at all times.

Partition tolerance refers to the system’s ability to continue operating despite message losses or failure within the system.

Given that network partitions are inevitable, systems must choose between consistency and availability.

It’s important to note that CAP theorem assumes ideal conditions of 100% availability and 100% consistency.

In the real world, it’s not so black and white.

The real world is complex, dynamic, and messy, with varying degrees of consistency and availability.

While CAP theorem underscores a crucial aspect of system design—balancing trade-offs—the simplistic model can be misleading. It’s best to think of it as a guide or tool rather than a strict rule.

Practical Implications and Trade-offs

The CAP theorem highlights the need for trade-offs in distributed system design. Different systems must prioritize specific aspects based on their requirements.

Consider an online retail store with multiple inventory databases across different locations.

Consistency vs Availability

Consistency

Ensures all customers see the same inventory information.

For example, if a customer in San Francisco sees that there are 5 units of a product available, a customer in Sydney will see the same.

This prevents overselling but if one database becomes unreachable, the system may deny all sales transactions to maintain consistency, affecting availability.

Availability

Ensures customers can always place orders, even if the inventory databases are not perfectly synchronized. This means that if the San Francisco database is temporarily unreachable, customers can still place orders based on the Sydney database.

This improves customer experience but risks inconsistencies, such as two customers purchasing the same product simultaneously, leading to overselling.

The store must decide which aspect is more critical.

If preventing overselling is paramount, consistency should be prioritized.

If ensuring customers can always place orders is more important, availability should take precedence.

Understanding these trade-offs helps us design a system that best meets their operational needs.

Modern Interpretations and Applications

The principles of CAP theorem remain highly applicable today, as cloud computing, big data, and microservices dominate the tech landscape.

Given that modern workloads are highly dynamic, systems in these environments must continually reevaluate the balance between consistency and availability.

Adopting adaptable models that offer the best balance between these components in real-time is generally advisable.

The CAP theorem continues to serve as a guide for building resilient distributed systems capable of managing unanticipated issues.

While it’s a good starting point, it doesn’t provide a complete picture of the trade-offs to consider when designing robust distributed systems. Distributed systems are complex, and consistency and availability are just two qualities to consider when designing a robust system.

Final Thoughts

The CAP theorem, while simple in its formulation, offers profound insights into the design and operation of distributed systems.

It provides a framework that helps us understand the trade-offs involved in creating robust systems.

Since Eric Brewer introduced it in the year 2,000, technologies have evolved immensely. However, the principles of the CAP theorem continue to guide us in making informed trade-offs and building robust systems that meet requirements.

HTTP/2 vs HTTP/3 — What’s the Difference? (Recap)

HTTP/1.1 (1997) brought key improvements like persistent connections, chunked transfers, and better caching—but suffered from sequential request blocking and reliance on multiple TCP connections.

HTTP/2 (2015) introduced multiplexing (multiple requests over one connection), header compression (HPACK), and stream prioritization. However, it still struggled with TCP’s head-of-line (HoL) blocking.

HTTP/3 (2022) is built on QUIC, using UDP instead of TCP. It solves HoL blocking by allowing independent streams, offers faster handshakes, requires TLS 1.3 encryption, and supports connection migration across networks.

Bottom line: HTTP/2 enhanced TCP, but HTTP/3 moved away from it—built on QUIC over UDP, it delivers faster performance, stronger security, and greater resilience by default.

How Does SQL Execution Order Work, and Why is it so Important? (Recap)

A SQL query executes its statements in the following order:

  1. FROM / JOIN

  2. WHERE

  3. GROUP BY

  4. HAVING

  5. SELECT

  6. DISTINCT

  7. ORDER BY

  8. LIMIT / OFFSET

The techniques you implement at each step help speed up the following steps. This is why it's important to know their execution order. To maximize efficiency, focus on optimizing the steps earlier in the query.

Kafka vs Bufstream — What’s the Difference? (Recap)

Kafka is powerful but complex and costly—great for ultra-low latency needs, but heavy on infrastructure, operational burden, and cost.

Bufstream is a cloud-native Kafka alternative that's ~800% cheaper, easier to scale, and simpler to manage. It uses S3/Iceberg for storage, gRPC for transport, and has built-in schema validation (no external registry needed).

While Bufstream trades off some latency for cost efficiency, it’s ideal for teams looking for scalable, compliant, and hassle-free event streaming without the operational complexity and costs of Kafka.

Where ultra-low latency outweighs 800% cost reduction, reduced complexity and other benefits, Kafka is the better fit.

That wraps up this week’s issue of Level Up Coding’s newsletter!

Join us again next week where we’ll explore and visually distill more important engineering concepts.