Fault Tolerance in System Design: Techniques, Examples and Java Code

Fault tolerance is a system's ability to keep working correctly when some of its parts fail. A fault-tolerant system expects servers to crash, networks to drop messages and services to slow down, and is built so these faults do not become outages for users. It does this with redundancy, failure detection and automatic recovery.

This tutorial explains the key ideas in plain terms, shows how they fit together, and implements three of them in Java: timeouts with fallbacks, retries with backoff and failover. It also covers traps many guides skip, such as retry storms, correlated failures and Java's unlimited default timeouts.

1. What Is Fault Tolerance?

On a single server, a crash means downtime. Large systems run on hundreds of machines, so something is always broken: a disk, a network switch, or a service that just shipped a bug. Fault tolerance accepts this and designs for it.

Ideally users notice nothing. At worst they get a slightly reduced experience, such as bestsellers instead of personalised recommendations. This is called graceful degradation. Fault tolerance is closely tied to availability and reliability.

2. Faults, Errors and Failures

These three words describe a chain:

• A fault is the root cause, such as a crashed server or a slow database.
• An error is the wrong internal state it causes, such as a request stuck waiting.
• A failure is what the user sees: an error page or a timeout.

Fault tolerance breaks the chain early, so faults never become failures. Common faults include hardware crashes, network problems, slow dependencies, software bugs and human mistakes such as a bad configuration change. Slowness is the sneakiest: a service that answers in 30 seconds can do more damage than one that is down, because callers keep waiting and pile up.

3. How Fault Tolerance Works

3.1 Redundancy

Keep more than one of everything that matters: several app instances, database replicas, and servers in different availability zones, which are separate data centres within one region. In active-active setups every copy serves traffic; in active-passive setups a standby waits to take over.

3.2 Detection

You cannot recover from a failure you have not noticed. Health checks let a load balancer stop sending traffic to a broken instance, and timeouts stop a caller from waiting forever.

3.3 Recovery

Retries handle brief glitches, failover switches to a healthy copy, and fallbacks return a simpler answer when nothing else works.

3.4 Containment

A circuit breaker stops calling a dependency that keeps failing, giving it time to recover. A bulkhead gives each dependency its own limited pool of threads, so one slow service cannot use them all.

4. Measuring Fault Tolerance

Availability is often measured in "nines": 99.9% allows about 8.8 hours of downtime a year, and 99.99% about 53 minutes. Two numbers drive it: how often things fail, the mean time between failures (MTBF), and how quickly you recover, the mean time to recovery (MTTR).

Redundancy multiplies. If each of two replicas is up 99% of the time and they fail independently, both are down only 0.01% of the time, giving 99.99%. The key word is independently. Two replicas in the same rack, on the same power supply or running the same buggy release fail together, and the maths no longer holds. This is a common reason "redundant" systems still go down.

5. Fault Tolerance in Java: Code Examples

The examples use only the JDK and need Java 17 or later, for records and Random.nextLong(bound).

5.1 Example 1: Timeout with a Fallback

package com.javabytechie.faulttolerance;

import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;

public class TimeoutFallbackDemo {

    // The recommendation service is overloaded and takes 2 seconds to answer
    static List<String> fetchRecommendations(String customerId) {
        try {
            Thread.sleep(2_000);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return List.of("personalised-1", "personalised-2");
    }

    public static void main(String[] args) {
        long start = System.nanoTime();

        List<String> items = CompletableFuture
                .supplyAsync(() -> fetchRecommendations("C-1001"))
                .completeOnTimeout(List.of("bestseller-1", "bestseller-2"), 300, TimeUnit.MILLISECONDS)
                .join();

        long elapsedMs = (System.nanoTime() - start) / 1_000_000;
        System.out.println("Shown to customer: " + items);
        System.out.println("Answered within 500 ms: " + (elapsedMs < 500));
    }
}

Shown to customer: [bestseller-1, bestseller-2]
Answered within 500 ms: true

completeOnTimeout supplies the fallback after 300 ms, so the customer sees bestsellers instead of a two-second wait. Note that it does not cancel the slow call: the background thread stays busy for the full two seconds. Under heavy load, that is why you also need limits such as bulkheads.

5.2 Example 2: Retry with Exponential Backoff and Jitter

package com.javabytechie.faulttolerance;

import java.util.HashMap;
import java.util.Map;
import java.util.Random;
import java.util.function.Supplier;

public class RetryWithBackoffDemo {

    // Times out on the first two calls, then works. It remembers idempotency keys,
    // so a retried request can never charge the customer twice.
    static class FlakyPaymentGateway {
        private final Map<String, String> processed = new HashMap<>();
        private int calls;

        String charge(String idempotencyKey, int amountPaise) {
            calls++;
            if (calls <= 2) {
                throw new IllegalStateException("gateway timeout");
            }
            return processed.computeIfAbsent(idempotencyKey, k -> "PAID " + amountPaise + " paise, ref " + k);
        }
    }

    static <T> T retry(Supplier<T> action, int maxAttempts, long baseDelayMs, Random random) throws InterruptedException {
        for (int attempt = 1; ; attempt++) {
            try {
                return action.get();
            } catch (IllegalStateException e) {
                if (attempt == maxAttempts) {
                    throw e;
                }
                long cap = baseDelayMs * (1L << (attempt - 1));   // 100, 200, 400 ms ...
                long delay = random.nextLong(cap + 1);            // full jitter: anywhere from 0 to cap
                System.out.printf("attempt %d failed (%s), retrying in %d ms%n", attempt, e.getMessage(), delay);
                Thread.sleep(delay);
            }
        }
    }

    public static void main(String[] args) throws InterruptedException {
        FlakyPaymentGateway gateway = new FlakyPaymentGateway();
        String idempotencyKey = "order-5521-payment";   // same key on every attempt

        String result = retry(() -> gateway.charge(idempotencyKey, 49_900), 4, 100, new Random(42));
        System.out.println(result);
    }
}

attempt 1 failed (gateway timeout), retrying in 2 ms
attempt 2 failed (gateway timeout), retrying in 106 ms
PAID 49900 paise, ref order-5521-payment

The maximum wait doubles after each failure (100, 200, 400 ms), and the actual wait is a random value up to that maximum. This randomness, called jitter, stops thousands of clients from retrying at the same moment. Every attempt sends the same idempotency key, so a retry after a lost response cannot charge twice. The seeded Random makes the output repeatable; real code would not seed it. The retry pattern page covers this in depth.

5.3 Example 3: Failover Between Replicas

package com.javabytechie.faulttolerance;

import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class FailoverClient {

    interface Replica {
        String name();
        String fetchStock(String sku);
    }

    record InventoryReplica(String name, boolean up) implements Replica {
        public String fetchStock(String sku) {
            if (!up) {
                throw new IllegalStateException("connection refused");
            }
            return sku + "=42 units";
        }
    }

    private static final int MAX_CONSECUTIVE_FAILURES = 2;

    private final List<Replica> replicas;                       // in order of preference
    private final Map<String, Integer> failures = new HashMap<>();

    FailoverClient(List<Replica> replicas) {
        this.replicas = replicas;
    }

    String fetchStock(String sku) {
        for (Replica replica : replicas) {
            if (failures.getOrDefault(replica.name(), 0) >= MAX_CONSECUTIVE_FAILURES) {
                System.out.println("  skipping " + replica.name() + " (marked unhealthy)");
                continue;
            }
            try {
                String stock = replica.fetchStock(sku);
                failures.put(replica.name(), 0);                // success resets the count
                return stock + " from " + replica.name();
            } catch (IllegalStateException e) {
                failures.merge(replica.name(), 1, Integer::sum);
                System.out.println("  " + replica.name() + " failed: " + e.getMessage());
            }
        }
        throw new IllegalStateException("all replicas are down");
    }

    public static void main(String[] args) {
        FailoverClient client = new FailoverClient(List.of(
                new InventoryReplica("inventory-mumbai", false),   // primary zone is down
                new InventoryReplica("inventory-pune", true)));

        for (int request = 1; request <= 3; request++) {
            System.out.println("request " + request + ":");
            System.out.println("  " + client.fetchStock("SKU-778"));
        }
    }
}

request 1:
  inventory-mumbai failed: connection refused
  SKU-778=42 units from inventory-pune
request 2:
  inventory-mumbai failed: connection refused
  SKU-778=42 units from inventory-pune
request 3:
  skipping inventory-mumbai (marked unhealthy)
  SKU-778=42 units from inventory-pune

The first two requests fail on Mumbai and fail over to Pune. After two failures in a row, Mumbai is marked unhealthy and skipped. A real client would test it again after a while, like a circuit breaker's half-open state, so it can rejoin once fixed.

6. Fault Tolerance vs High Availability vs Disaster Recovery

RTO (recovery time objective) is how long recovery may take, and RPO (recovery point objective) is how much recent data you can afford to lose.

7. Common Mistakes

• No timeouts: Java's HttpClient waits indefinitely unless you set HttpRequest.Builder.timeout(), and HttpURLConnection defaults to 0, meaning no limit.
• Retry storms: retrying at every layer multiplies traffic; three layers with three attempts each turn one request into 27 calls to the last service.
• Retrying non-idempotent writes: without an idempotency key, a retried payment can charge twice.
• Correlated redundancy: replicas in one zone, rack or release fail together.
• Untested failover: recovery code that never runs tends to break exactly when it is needed.
• Split brain: if a primary is only cut off, not dead, two primaries may accept writes; use quorum-based election and fencing.

8. Best Practices

• Set a timeout on every remote call, shorter than the caller's own deadline.
• Retry only temporary errors, with capped exponential backoff, jitter and a retry limit.
• Make writes idempotent with idempotency keys.
• Spread replicas across zones and roll out releases gradually.
• Plan useful fallbacks so the system degrades gracefully.
• Guard risky dependencies with circuit breakers and bulkheads.
• Test failures on purpose with fault injection and game days.
• Monitor errors and latency, and work to lower MTTR.

9. Summary

• Fault tolerance keeps a system working when parts of it fail.
• It stops faults from becoming user-visible failures.
• The core techniques are redundancy, detection, recovery and containment.
• Redundancy only helps if failures are independent.
• Timeouts, retries with backoff and jitter, idempotency keys and failover are the everyday building blocks.
• Untested recovery paths and retry storms are common hidden risks.

Good fault tolerance is less about preventing every fault and more about making sure no single fault becomes an outage. Open the FAQ tab for fault tolerance interview questions.

Fault Tolerance Interview Questions and Answers