Microservices Resilience Patterns in Java: Retry, Circuit Breaker, Timeout and Bulkhead

Microservices resilience patterns are coding techniques that keep a service working, or failing gracefully, when the services it depends on are slow, broken or overloaded. The five core patterns are timeout, retry, circuit breaker, bulkhead and fallback. Together they stop one bad dependency from bringing down your whole system.

In a monolith, a method call either works or throws an exception. In microservices, that call travels over a network to another process. Resilience patterns are how you plan for the network failing. This tutorial explains each pattern, shows how a circuit breaker works internally, and builds three runnable Java 21 examples.

1. Why Microservices Need Resilience

Picture an online grocery store. When a customer opens the checkout page, the Checkout service calls three other services: Stock, Shipping and Recommendations.

Now suppose the Recommendations service becomes slow. Each request to it hangs for 30 seconds. Checkout keeps sending requests, and each one holds a thread while it waits. Within a minute, every Checkout thread is stuck waiting on Recommendations. Checkout can no longer serve anyone, even though Stock and Shipping are perfectly healthy.

This chain reaction is called a cascading failure: one failing service drags down the services that call it, and then the services that call those.

2. The Five Core Microservices Resilience Patterns

2.1 Timeout

A timeout is the maximum time you are willing to wait for a reply. Without one, a thread can wait forever. A timeout turns "hang" into a quick, predictable error that you can handle.

Choose the value from real data. If 99% of calls to Shipping finish in 200 ms, a timeout of 500 ms is reasonable. A timeout of 60 seconds protects nothing. Read more in the timeout pattern tutorial.

2.2 Retry

A retry repeats a failed call, because many network failures are transient, meaning they disappear on their own. A pod restarting or a brief network blip are good examples.

Retrying immediately is risky. If a service is struggling, a flood of instant retries makes things worse. So we use exponential backoff: wait 100 ms, then 200 ms, then 400 ms, doubling each time. We also add jitter, a small random extra delay, so that thousands of clients don't all retry at the same moment. See the retry pattern for details.

2.3 Circuit Breaker

A circuit breaker watches calls to a dependency. When too many fail, it "trips" and stops sending calls for a while. Calls fail instantly instead of waiting on a service that is clearly down. This gives the sick service room to recover and saves your threads. The circuit breaker pattern tutorial covers it in depth.

2.4 Bulkhead

A ship's hull is split into watertight sections called bulkheads. If one section floods, the others stay dry. In software, a bulkhead limits how many concurrent calls can go to one dependency. If Recommendations is slow, it can use up at most, say, 10 slots. The rest of your capacity stays free for Stock and Shipping. See the bulkhead pattern.

2.5 Fallback

A fallback is the answer you return when the real call fails: cached data, a default value, or a simpler result. For example, show "bestsellers" when personalised recommendations are unavailable. For a deeper look, see our guide on the fallback pattern in microservices.

3. How the Circuit Breaker Works Internally

A circuit breaker is a small state machine, an object that is always in exactly one of a fixed set of states and moves between them based on rules.

• CLOSED: Normal operation. Calls go through and failures are counted.
• OPEN: The breaker has tripped. Calls are rejected immediately, and the dependency is not contacted at all.
• HALF_OPEN: The cool-down is over. A few trial calls are allowed through. If they succeed, the breaker closes. If they fail, it opens again and the cool-down restarts.

One detail many tutorials skip: an OPEN breaker has no idea whether the dependency has recovered. It only finds out by letting a trial call through after the wait duration.

4. Code Examples

All three programs use plain Java 21 with no libraries, so you can see what a library like Resilience4j does for you behind the scenes.

4.1 Retry with Exponential Backoff and Jitter

import java.util.Random;
import java.util.function.Supplier;

public class RetryWithBackoffDemo {

    // Simulates a stock service that fails twice, then recovers
    static class FlakyStockService {
        private int calls = 0;

        int availableUnits(String sku) {
            calls++;
            if (calls <= 2) {
                throw new IllegalStateException("503 Service Unavailable");
            }
            return 17;
        }
    }

    static <T> T retry(Supplier<T> action, int maxAttempts, long baseDelayMs, Random random)
            throws InterruptedException {
        for (int attempt = 1; ; attempt++) {
            try {
                T result = action.get();
                System.out.println("Attempt " + attempt + ": success");
                return result;
            } catch (IllegalStateException e) { // retry only errors we know are temporary
                System.out.println("Attempt " + attempt + ": failed (" + e.getMessage() + ")");
                if (attempt == maxAttempts) {
                    throw e; // give up and let the caller decide
                }
                long backoff = baseDelayMs * (1L << (attempt - 1)); // 100, 200, 400 ...
                long jitter = random.nextLong(backoff / 2 + 1);     // spread retries out
                System.out.printf("  backing off %d ms (+%d ms jitter)%n", backoff, jitter);
                Thread.sleep(backoff + jitter);
            }
        }
    }

    public static void main(String[] args) throws InterruptedException {
        var stock = new FlakyStockService();
        var random = new Random(7); // fixed seed so the output is repeatable

        int units = retry(() -> stock.availableUnits("SKU-TEA-500G"), 4, 100, random);
        System.out.println("Units in stock: " + units);
    }
}

Attempt 1: failed (503 Service Unavailable)
  backing off 100 ms (+36 ms jitter)
Attempt 2: failed (503 Service Unavailable)
  backing off 200 ms (+65 ms jitter)
Attempt 3: success
Units in stock: 17

The fixed seed 7 makes the jitter values repeatable for this demo. In production, use an unseeded random source so every client gets different delays. We catch only IllegalStateException, which stands in for "temporary" errors.

4.2 A Circuit Breaker Built from Scratch

This version trips after 3 failures in a row and stays open for 5 seconds. A fake clock keeps the output predictable.

import java.util.List;
import java.util.function.Supplier;

public class CircuitBreakerDemo {

    enum State { CLOSED, OPEN, HALF_OPEN }

    static class SimpleCircuitBreaker {
        private final int failureThreshold;
        private final long openDurationMs;
        private State state = State.CLOSED;
        private int consecutiveFailures = 0;
        private long openedAt = 0;

        SimpleCircuitBreaker(int failureThreshold, long openDurationMs) {
            this.failureThreshold = failureThreshold;
            this.openDurationMs = openDurationMs;
        }

        <T> T call(Supplier<T> action, Supplier<T> fallback, long nowMs) {
            if (state == State.OPEN) {
                if (nowMs - openedAt >= openDurationMs) {
                    state = State.HALF_OPEN; // cool-down over: allow one trial call
                } else {
                    return fallback.get();   // fail fast, dependency is not touched
                }
            }
            try {
                T result = action.get();
                consecutiveFailures = 0;
                state = State.CLOSED;
                return result;
            } catch (RuntimeException e) {
                consecutiveFailures++;
                if (state == State.HALF_OPEN || consecutiveFailures >= failureThreshold) {
                    state = State.OPEN;      // trip (or re-trip) the breaker
                    openedAt = nowMs;
                }
                return fallback.get();
            }
        }

        State state() {
            return state;
        }
    }

    // A shipping-rate service we can switch on and off
    static class ShippingRateService {
        boolean healthy = true;
        int hits = 0;

        String quote(String pinCode) {
            hits++;
            if (!healthy) {
                throw new RuntimeException("connection refused");
            }
            return "Rs 49 express";
        }
    }

    record Step(long timeMs, boolean serviceHealthy) { }

    public static void main(String[] args) {
        var shipping = new ShippingRateService();
        var breaker = new SimpleCircuitBreaker(3, 5_000);
        Supplier<String> fallback = () -> "Rs 79 standard (cached)";

        // A fake clock keeps the demo deterministic
        List<Step> timeline = List.of(
                new Step(0, true), new Step(1_000, false), new Step(2_000, false),
                new Step(3_000, false), new Step(4_000, false), new Step(8_000, false),
                new Step(9_000, true), new Step(13_000, true), new Step(14_000, true));

        for (Step step : timeline) {
            shipping.healthy = step.serviceHealthy();
            String price = breaker.call(() -> shipping.quote("560001"), fallback, step.timeMs());
            System.out.printf("t=%6d ms | %-9s | hits=%d | %s%n",
                    step.timeMs(), breaker.state(), shipping.hits, price);
        }
    }
}

t=     0 ms | CLOSED    | hits=1 | Rs 49 express
t=  1000 ms | CLOSED    | hits=2 | Rs 79 standard (cached)
t=  2000 ms | CLOSED    | hits=3 | Rs 79 standard (cached)
t=  3000 ms | OPEN      | hits=4 | Rs 79 standard (cached)
t=  4000 ms | OPEN      | hits=4 | Rs 79 standard (cached)
t=  8000 ms | OPEN      | hits=5 | Rs 79 standard (cached)
t=  9000 ms | OPEN      | hits=5 | Rs 79 standard (cached)
t= 13000 ms | CLOSED    | hits=6 | Rs 49 express
t= 14000 ms | CLOSED    | hits=7 | Rs 49 express

Read the hits column carefully. It counts how many requests actually reached the Shipping service:

• t=3000: the third failure in a row trips the breaker.
• t=4000: hits stays at 4. The breaker answered with the fallback without calling Shipping.
• t=8000: the 5-second cool-down is over, so one trial call goes through (hits=5). It fails, and the breaker opens again with a fresh timer.
• t=9000: Shipping is healthy again, but the breaker is still OPEN and still serves the fallback. It cannot know about the recovery yet.
• t=13000: the next trial call succeeds and the breaker closes.

4.3 Timeout and Bulkhead Together (with a Hidden Trap)

This client allows at most 2 calls in flight (bulkhead) and waits at most 300 ms per call (timeout). It uses Java 21 virtual threads.

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;

public class TimeoutBulkheadDemo {

    static class RecommendationClient implements AutoCloseable {
        private final Semaphore permits = new Semaphore(2); // bulkhead: max 2 calls in flight
        private final ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

        CompletableFuture<String> recommendationsFor(String userId, long latencyMs) {
            if (!permits.tryAcquire()) { // reject instantly instead of queueing
                return CompletableFuture.completedFuture("bulkhead full -> bestsellers");
            }
            return CompletableFuture
                    .supplyAsync(() -> remoteCall(userId, latencyMs), executor)
                    .orTimeout(300, TimeUnit.MILLISECONDS)          // stop waiting after 300 ms
                    .exceptionally(ex -> "timed out -> bestsellers"); // fallback value
        }

        private String remoteCall(String userId, long latencyMs) {
            try {
                Thread.sleep(latencyMs); // pretend this is a slow HTTP call
                return "personalised picks for " + userId;
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
                return "interrupted";
            } finally {
                permits.release(); // free the slot only when the real work ends
            }
        }

        @Override
        public void close() {
            executor.close(); // waits for running tasks (Java 19+)
        }
    }

    public static void main(String[] args) throws InterruptedException {
        try (var client = new RecommendationClient()) {
            var first = client.recommendationsFor("u1", 1_000);  // slow
            var second = client.recommendationsFor("u2", 1_000); // slow
            var third = client.recommendationsFor("u3", 50);     // no slot left

            System.out.println("u3: " + third.join());
            System.out.println("u1: " + first.join());
            System.out.println("u2: " + second.join());

            // The caller gave up, but both slow calls are still running
            System.out.println("u4: " + client.recommendationsFor("u4", 50).join());

            Thread.sleep(1_000); // let the abandoned calls finish
            System.out.println("u5: " + client.recommendationsFor("u5", 50).join());
        }
    }
}

u3: bulkhead full -> bestsellers
u1: timed out -> bestsellers
u2: timed out -> bestsellers
u4: bulkhead full -> bestsellers
u5: personalised picks for u5

Look at u4. The callers for u1 and u2 have already received their "timed out" fallback, yet u4 is still rejected. Why? orTimeout() only completes the future with a TimeoutException. It does not stop the thread that is still sleeping inside remoteCall(). Even cancel(true) on a CompletableFuture does not interrupt the running task. The two slow calls keep their permits until they really finish, which is why u5 succeeds only after the extra one-second wait.

5. Comparison Table

6. Common Mistakes and Pitfalls

6.1 Retry Storms Across Layers

Suppose the API Gateway retries 3 times, Checkout retries 3 times, and Stock retries 3 times against its database. One user click can become 3 × 3 × 3 = 27 database calls, right when the database is already struggling. Retry at one layer only, usually the one closest to the failing dependency.

6.2 Retrying Non-Idempotent Operations

An operation is idempotent if running it twice has the same effect as running it once. Reading stock is idempotent. Charging a card is not. If a payment call times out, the charge may have succeeded, and a blind retry charges the customer twice. Use an idempotency key so the server can detect duplicates.

6.3 Retrying Errors That Will Never Succeed

A 400 Bad Request or 404 Not Found will fail on every attempt. Retry only on timeouts, connection errors and responses like 503.

6.4 Believing a Timeout Stops the Work

As Example 4.3 showed, a timeout frees the caller, not the callee. The downstream service may still be processing the request.

6.5 Expecting One Shared Breaker State

Each instance of your service has its own circuit breaker in its own memory. With 10 pods, some breakers can be OPEN while others are CLOSED. This is normal, not a bug.

6.6 A Fallback That Calls Another Fragile Service

If your fallback fetches data from a second remote service, you have just moved the problem. Good fallbacks are local and cheap: a cache, a constant, or an empty list.

7. Best Practices

• Always set a timeout on every remote call, including HTTP clients, database drivers and message brokers.
• Order the patterns deliberately. By default, Resilience4j's Spring Boot integration nests them as Retry(CircuitBreaker(RateLimiter(TimeLimiter(Bulkhead(call))))), so each retry attempt passes through the breaker and is counted.
• Use semaphore bulkheads with virtual threads. Virtual threads are cheap, so you shouldn't pool them to limit concurrency. A Semaphore is the recommended way to cap concurrent calls.
• Monitor breaker state. Export state changes and failure rates as metrics. An OPEN breaker should raise an alert, not fail silently.
• Test failures on purpose. Inject latency and errors in staging so you can prove your timeouts and fallbacks work.
• Prefer a library in production. Use Resilience4j or Spring Cloud Circuit Breaker. The hand-written versions above are for learning; real ones need thread safety and sliding-window statistics. To limit calls per second as well, add the rate limiting pattern.

8. Summary

• Microservices resilience patterns stop a single failing dependency from causing a cascading failure.
• Timeout limits waiting, retry handles temporary errors, circuit breaker fails fast when a service is down, bulkhead caps concurrency, and fallback gives users a safe answer.
• Use exponential backoff with jitter for retries, and retry only idempotent operations and temporary errors.
• A circuit breaker moves between CLOSED, OPEN and HALF_OPEN, and only learns about recovery through trial calls.
• A timeout does not stop the remote work; CompletableFuture.orTimeout() does not interrupt the running task.
• Breaker state is local to each service instance.
• In Java 21, pair virtual threads with semaphore-based bulkheads.

Master these microservices resilience patterns and you will design systems that bend under failure instead of breaking.

Microservices Resilience Patterns: Interview Questions

Common interview questions on Microservices Resilience Patterns, from basic to scenario-based, with short answers.

resilience4j.retry.instances.stock:
  maxAttempts: 3
  retryExceptions: [java.io.IOException, java.util.concurrent.TimeoutException]
  ignoreExceptions: [com.shop.InvalidSkuException]