CAP Theorem in System Design: Consistency, Availability and Partition Tolerance

The CAP theorem says that when a distributed system suffers a network failure between its nodes, it must choose between staying consistent and staying available. It cannot guarantee both at that moment. This single idea explains why some databases refuse requests during an outage while others keep answering with possibly outdated data.

This tutorial explains each letter of CAP in plain language, walks through a concrete failure scenario, compares CP and AP systems, shows how real databases fit, introduces quorums and the PACELC theorem, and gives you a Java simulation and an interview-ready way to discuss CAP.

1. What Is the CAP Theorem?

The CAP theorem was proposed by computer scientist Eric Brewer in 2000 and formally proved by Seth Gilbert and Nancy Lynch in 2002. It applies to distributed data stores, meaning systems that keep copies of the same data on more than one machine.

It names three desirable properties:

• C – Consistency: every read sees the most recent write.
• A – Availability: every request to a working node gets a non-error response.
• P – Partition tolerance: the system keeps operating even when messages between nodes are lost or delayed.

The popular summary is "pick any two of three", but that phrasing is misleading. In a real distributed system, network problems will happen, so partition tolerance is not optional. The honest version is:

When a network partition happens, a distributed system must choose either consistency or availability. When there is no partition, it can provide both.

2. The Three Properties Explained

Consistency (C)

In CAP, consistency means that all clients see the same, latest data, no matter which node they ask. Once a write succeeds, every later read must return that value or a newer one. The formal name for this strong guarantee is linearizability: the system behaves as if there were only one copy of the data.

Example: you change your profile name from "Ravi" to "Ravi Kumar". With consistency, a friend who loads your profile one second later sees "Ravi Kumar", never the old name.

Availability (A)

Availability means that every request received by a node that has not crashed gets a meaningful response, not an error or a timeout. The response does not have to contain the latest data. It just has to be an answer.

Example: during an outage, a shopping site still shows product pages and accepts items in the cart, even if some numbers are a little out of date.

Partition tolerance (P)

A network partition is a break in communication between groups of nodes. Each group is still running, but they cannot talk to each other. Partition tolerance means the system continues to work, in some defined way, despite this split.

Example: a cable between two data centres is cut. Both data centres are healthy and serving users, but they can no longer replicate data to each other.

3. What Is a Network Partition?

Beginners sometimes think of partitions as rare, exotic events. In practice they happen regularly in large systems, for many ordinary reasons:

• A network switch or router fails or is misconfigured.
• A cable between buildings or regions is damaged.
• A firewall rule change accidentally blocks traffic between services.
• A server is overloaded or paused by a long garbage-collection pause, so other nodes believe it is unreachable.
• Network congestion delays messages so much that they time out.

The tricky part is that a node cannot tell the difference between "the other node crashed" and "the network to the other node is broken". It only knows that it is not getting replies. CAP is about what the system should do in that uncertain situation.

4. Why You Cannot Have All Three

Let us walk through the classic scenario. A bank stores an account balance on two nodes, Node A in Mumbai and Node B in London. Both start with a balance of ₹1,000. Then the network between them fails.

Client 1 withdraws ₹500 through Node A, so Node A's balance becomes ₹500. Node A tries to tell Node B, but the message cannot get through. Now Client 2 asks Node B for the balance. Node B has only two options:

1. Refuse to answer (or wait until the network heals) because it cannot be sure it has the latest value. The system stays consistent but is not available for this request.
2. Answer with ₹1,000, the value it has. The system stays available but is not consistent, because Client 2 sees outdated data.

There is no third option. Node B cannot return ₹500 because it simply does not know about the withdrawal. That is the heart of the CAP theorem.

5. CP Systems: Choosing Consistency

A CP system prefers being correct over being responsive. During a partition, nodes that cannot confirm they have the latest data stop serving the affected reads or writes and return errors or time out.

Most CP systems use a majority (quorum) rule. With five nodes, any group that can reach at least three nodes may continue, while the smaller side becomes read-only or unavailable. Consensus algorithms such as Raft and Paxos work this way.

Choose CP when wrong data is worse than no data:

• Bank balances, payments and money transfers.
• Stock inventory where overselling is costly, such as limited concert tickets.
• Unique usernames or seat reservations, where two people must never get the same item.
• Configuration and coordination data, such as leader election and distributed locks.

Examples of systems usually classed as CP: ZooKeeper, etcd, Consul, HBase, and MongoDB when it uses majority read and write concerns.

6. AP Systems: Choosing Availability

An AP system prefers responding over being perfectly up to date. During a partition, every node keeps accepting reads and writes using its local data. When the network heals, the nodes exchange updates and resolve any conflicts. The data becomes consistent again eventually, which is why this model is called eventual consistency.

Conflicts can be resolved in several ways:

• Last write wins: keep the value with the latest timestamp. Simple, but one update can be silently lost.
• Merging: combine both changes, for example adding items from both versions of a shopping cart.
• CRDTs (Conflict-free Replicated Data Types): special data structures, such as counters and sets, designed to merge automatically without conflicts.
• Application decides: keep both versions and let the application or the user pick.

Choose AP when an answer now is better than a perfect answer later:

• Social media likes, view counts and follower counts.
• Product catalogues, reviews and recommendations.
• Shopping carts, where losing a sale is worse than merging carts later.
• DNS, caches and content delivery.

Examples of systems usually classed as AP: Cassandra, DynamoDB with eventually consistent reads, Riak, CouchDB and DNS.

7. What About CA Systems?

A "CA" system would be consistent and available but not partition tolerant. That only makes sense if partitions cannot happen, which is true for a single machine, such as one PostgreSQL or MySQL server. With only one node, there is no network between copies to break.

The moment you replicate data across machines connected by a network, partitions become possible, and you must decide what happens when one occurs. So for distributed systems, CA is not a real option. When people describe a clustered database as "CA", they usually mean it assumes a very reliable network and will lose either consistency or availability if that assumption breaks.

8. Real Databases and Where They Fit

Treat these labels as typical defaults, not fixed facts. Many databases let you tune the behaviour per query.

9. Tunable Consistency with Quorums

Many AP-style databases let you choose consistency per request using three numbers:

• N: how many replicas store each piece of data.
• W: how many replicas must confirm a write before it counts as successful.
• R: how many replicas must respond to a read.

If R + W > N, the set of replicas you read from always overlaps with the set that accepted the latest write, so at least one replica in every read has the newest value.

This shows that CAP is not a one-time choice for a whole database. You can choose CP-like behaviour for a payment record and AP-like behaviour for a page-view counter, even in the same cluster.

10. Beyond CAP: The PACELC Theorem

CAP only describes what happens during a partition, which is a small fraction of the time. Daniel Abadi proposed PACELC in 2010 to cover normal operation too:

If there is a Partition (P), choose Availability (A) or Consistency (C); Else (E), choose Latency (L) or Consistency (C).

The "else" part matters because keeping replicas perfectly in sync costs time even when the network is healthy: a write must wait for other replicas, often in other regions, to confirm. Systems that skip that wait respond faster but may serve slightly old data.

11. Simulating CP and AP Behaviour in Java

The following program models the bank example with two replicas. It runs the same scenario in CP mode and in AP mode so you can see the different outcomes:

package com.javabytechie.cap;

public class CapDemo {

    enum Mode { CP, AP }

    static class ReplicatedAccount {
        private final Mode mode;
        private int balanceOnNodeA = 1000;
        private int balanceOnNodeB = 1000;
        private boolean partitioned;

        ReplicatedAccount(Mode mode) {
            this.mode = mode;
        }

        void startPartition() {
            partitioned = true;
        }

        void healPartition() {
            partitioned = false;
            if (mode == Mode.AP) {
                balanceOnNodeB = balanceOnNodeA;   // replicas exchange updates and converge
            }
        }

        String writeToNodeA(int newBalance) {
            if (partitioned && mode == Mode.CP) {
                return "WRITE REJECTED: Node A cannot reach Node B, so it refuses the write";
            }
            balanceOnNodeA = newBalance;
            if (!partitioned) {
                balanceOnNodeB = newBalance;        // normal synchronous replication
            }
            return "WRITE OK on Node A: balance = " + newBalance;
        }

        String readFromNodeB() {
            if (partitioned && mode == Mode.CP) {
                return "READ FAILED: Node B cannot confirm it has the latest value";
            }
            return "READ OK from Node B: balance = " + balanceOnNodeB;
        }
    }

    public static void main(String[] args) {
        for (Mode mode : Mode.values()) {
            ReplicatedAccount account = new ReplicatedAccount(mode);
            System.out.println("=== " + mode + " system ===");
            account.startPartition();
            System.out.println(account.writeToNodeA(500));
            System.out.println(account.readFromNodeB());
            account.healPartition();
            System.out.println("-- partition healed --");
            System.out.println(account.readFromNodeB());
            System.out.println();
        }
    }
}

=== CP system ===
WRITE REJECTED: Node A cannot reach Node B, so it refuses the write
READ FAILED: Node B cannot confirm it has the latest value
-- partition healed --
READ OK from Node B: balance = 1000

=== AP system ===
WRITE OK on Node A: balance = 500
READ OK from Node B: balance = 1000
-- partition healed --
READ OK from Node B: balance = 500

The CP system never shows a wrong balance, but it fails requests during the partition, and the client has to retry the withdrawal later. The AP system keeps working, but Client 2 briefly sees ₹1,000 when the true balance is ₹500. After the partition heals, the AP replicas converge on the correct value.

12. How to Choose: CP or AP for Each Feature

Real systems rarely pick one model for everything. Decide per feature by asking: "What is worse for this data: showing something outdated, or showing an error?"

13. Common Misconceptions

• "Pick any two of three." In distributed systems you cannot give up partition tolerance, so the real choice is C or A during a partition.
• "An AP system is never consistent." Without a partition, AP systems are usually consistent within milliseconds. The trade-off only bites during failures or replication delays.
• "C in CAP is the same as C in ACID." It is not. ACID consistency means a transaction keeps data valid according to rules such as constraints. CAP consistency means all replicas show the latest write.
• "Availability in CAP means 99.99% uptime." CAP availability is a strict theoretical property: every live node must answer every request. A CP system can still have excellent real-world uptime.
• "A database is permanently CP or AP." Many databases let you choose consistency per query, so the label depends on how you use them.
• "CAP covers all trade-offs." It ignores latency and performance during normal operation, which is why PACELC exists.

14. How to Discuss CAP in a System Design Interview

1. State the real trade-off: "Partitions are unavoidable in a distributed system, so I need to decide what each feature does during one."
2. Classify the data: separate data where correctness is critical (payments, inventory) from data where freshness can lag (feeds, counters).
3. Pick a behaviour per category: CP with quorum writes or a single leader for critical data; AP with eventual consistency for the rest.
4. Explain conflict handling: for AP data, say how conflicts are resolved, such as last-write-wins, merging or CRDTs.
5. Mention latency: bring in PACELC when discussing multi-region deployments, where strong consistency adds cross-region delay to every write.
6. Name concrete tools: for example, "PostgreSQL with synchronous replication for orders, Cassandra for activity events."

15. Summary

• The CAP theorem says a distributed system cannot guarantee both consistency and availability while a network partition is happening.
• Consistency means every read sees the latest write; availability means every live node answers; partition tolerance means surviving network splits.
• Because partitions are unavoidable, the practical choice is between CP (correct but may reject requests) and AP (always answers but may be stale).
• Quorums (R + W > N) let many databases tune consistency per operation.
• PACELC extends CAP: without partitions, systems still trade latency for consistency.

Use CAP as a thinking tool: for each piece of data, decide whether stale data or an error is the lesser evil. Open the FAQ tab for answers to common questions about the CAP theorem.

Frequently Asked Questions (FAQ)