00:00

Help Us Improve

We're sorry this tutorial wasn't helpful. Please tell us what we can do better.

Rate Limiting Pattern in Microservices

Last updated:

In a microservices architecture, many independent services and clients send requests to each other all day long. Keeping that traffic stable and fair is essential. One of the most important patterns for this is rate limiting. It acts as a traffic officer for your APIs, preventing overload and keeping the experience good for every user.

This guide explains how rate limiting works, compares the main algorithms, and shows working Java examples: a token bucket and sliding window in plain Java, a Spring Boot filter, Spring Cloud Gateway with Redis, Resilience4j, and distributed limits shared across many instances.

1. What Is the Rate Limiting Pattern?

In microservices architecture, the Rate Limiting Pattern controls the number of requests a service can handle within a specific time window. It prevents service overload, manages traffic spikes, and maintains system stability and availability. Since microservices often depend on one another, rate limiting helps prevent excessive requests from overwhelming individual services and reduces the risk of cascading failures across the system.

2. Why Rate Limiting Is Important in Microservices

In a monolith, resource use is managed in one place. In a distributed microservices system, the need for rate limiting grows, because many clients and services share the same back ends:

ReasonHow rate limiting helps
Resource protectionStops any single consumer from using up CPU, memory or database connections
Cost controlIn the cloud, usage means money; limits stop runaway clients from creating huge bills
SecurityA first line of defense against denial-of-service and brute-force attacks, such as password guessing
Fair usage and quotasShares capacity fairly between clients, and enables plans such as free and premium tiers
ReliabilitySmooths out spikes, so performance stays consistent for everyone
Protecting downstream servicesKeeps your service from overwhelming slower dependencies or paid third-party APIs

3. Rate Limiting Algorithms

Rate limiting can be implemented using the following algorithms, each of which has different advantages in terms of traffic control, resource utilization, and request handling. You can choose based on the requirements.

AlgorithmHow it worksAllows bursts?Good for
Token bucketA bucket fills with tokens at a steady rate up to a maximum. Each request takes a token; no token means the request is rejectedYes, up to the bucket sizeMost APIs; the most widely used
Leaky bucketRequests enter a queue that “leaks” (is processed) at a constant rate. A full queue rejects new requestsNo; output is perfectly smoothProtecting systems that need an even flow
Fixed window counterCounts requests in fixed windows, such as each calendar minute, and resets at the start of each windowYes, at window edgesSimple limits where small bursts are acceptable
Sliding window logStores the timestamp of every request and counts those within the last windowNoExact limits for low traffic; uses more memory
Sliding window counterCombines the current and previous window counts, weighted by time, to estimate a rolling windowVery smallAccurate limits with little memory

4. Building a Token Bucket in Plain Java

The token bucket is easy to understand and to build. The bucket starts full. Tokens are added back continuously, and each request takes one:

TokenBucket.java 
package com.example.ratelimit;

import java.time.Duration;

public class TokenBucket {

    private final long capacity;          // maximum burst size
    private final double tokensPerNano;   // refill speed
    private double availableTokens;
    private long lastRefillTime;

    public TokenBucket(long capacity, long refillTokens, Duration refillPeriod) {
        this.capacity = capacity;
        this.tokensPerNano = (double) refillTokens / refillPeriod.toNanos();
        this.availableTokens = capacity;
        this.lastRefillTime = System.nanoTime();
    }

    public synchronized boolean tryConsume() {
        refill();
        if (availableTokens >= 1) {
            availableTokens -= 1;
            return true;    // request allowed
        }
        return false;       // bucket empty: reject
    }

    private void refill() {
        long now = System.nanoTime();
        double newTokens = (now - lastRefillTime) * tokensPerNano;
        availableTokens = Math.min(capacity, availableTokens + newTokens);
        lastRefillTime = now;
    }
}
Usage 
// 100 requests per minute, with bursts of up to 100
TokenBucket bucket = new TokenBucket(100, 100, Duration.ofMinutes(1));

if (bucket.tryConsume()) {
    // handle the request
} else {
    // respond with 429 Too Many Requests
}

The bucket size controls how big a burst can be, and the refill rate controls the long-term average. A bucket of 20 refilled with 100 tokens per minute allows short bursts of 20 while still averaging 100 per minute.

5. Building a Sliding Window Counter

The sliding window counter keeps only two numbers per client: the count for the current window and the count for the previous one. It estimates the requests in the last full window like this:

Sliding window formula 
estimated = previousCount x (1 - elapsedInCurrentWindow / windowLength)
          + currentCount

Example: limit 100/minute, 15 seconds into the current minute
  previousCount = 80, currentCount = 30
  estimated = 80 x (1 - 15/60) + 30 = 60 + 30 = 90  -> allowed
SlidingWindowCounter.java 
package com.example.ratelimit;

import java.time.Duration;

public class SlidingWindowCounter {

    private final int limit;
    private final long windowMillis;
    private long currentWindowStart;
    private int currentCount;
    private int previousCount;

    public SlidingWindowCounter(int limit, Duration window) {
        this.limit = limit;
        this.windowMillis = window.toMillis();
        this.currentWindowStart = System.currentTimeMillis();
    }

    public synchronized boolean tryAcquire() {
        long now = System.currentTimeMillis();
        long elapsed = now - currentWindowStart;

        if (elapsed >= windowMillis) {
            // Move forward; if more than one full window passed, the previous count is 0
            previousCount = (elapsed < 2 * windowMillis) ? currentCount : 0;
            currentCount = 0;
            currentWindowStart = now - (elapsed % windowMillis);
            elapsed = now - currentWindowStart;
        }

        double previousWeight = 1.0 - (double) elapsed / windowMillis;
        double estimated = previousCount * previousWeight + currentCount;

        if (estimated < limit) {
            currentCount++;
            return true;
        }
        return false;
    }
}

These plain-Java classes are great for learning and for a single instance. For production, libraries such as Bucket4j, Resilience4j or your API gateway’s built-in limiter add tested implementations, metrics and distributed storage.

6. A Rate Limiting Filter in Spring Boot

A servlet filter can apply the token bucket from section 4 to every incoming request, with one bucket per client:

RateLimitFilter.java 
package com.example.product.ratelimit;

import java.io.IOException;
import java.time.Duration;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;

import com.example.ratelimit.TokenBucket;

import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;

@Component
public class RateLimitFilter extends OncePerRequestFilter {

    private static final int LIMIT_PER_MINUTE = 100;

    private final Map<String, TokenBucket> buckets = new ConcurrentHashMap<>();

    @Override
    protected void doFilterInternal(HttpServletRequest request,
                                    HttpServletResponse response,
                                    FilterChain chain) throws ServletException, IOException {

        String clientKey = resolveClientKey(request);
        TokenBucket bucket = buckets.computeIfAbsent(clientKey,
                key -> new TokenBucket(LIMIT_PER_MINUTE, LIMIT_PER_MINUTE, Duration.ofMinutes(1)));

        if (bucket.tryConsume()) {
            chain.doFilter(request, response);
            return;
        }

        response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());
        response.setHeader("Retry-After", "1");
        response.setHeader("X-RateLimit-Limit", String.valueOf(LIMIT_PER_MINUTE));
        response.setContentType("application/json");
        response.getWriter().write(
                "{\"error\":\"too_many_requests\",\"message\":\"Please slow down.\"}");
    }

    // Prefer an API key; fall back to the caller's IP address
    private String resolveClientKey(HttpServletRequest request) {
        String apiKey = request.getHeader("X-API-Key");
        return (apiKey != null) ? "key:" + apiKey : "ip:" + request.getRemoteAddr();
    }

    @Override
    protected boolean shouldNotFilter(HttpServletRequest request) {
        // Never rate limit health checks
        return request.getRequestURI().startsWith("/actuator/health");
    }
}
Limitations: This map grows with every new client; use a cache with expiry, such as Caffeine, in real code. The limits also live in the memory of one instance, so with three instances a client effectively gets three times the limit. Section 9 solves this with Redis.

7. Rate Limiting at the API Gateway

Enforcing limits at the API gateway gives one consistent place for all services. Spring Cloud Gateway has a built-in, Redis-backed token bucket limiter. Add the reactive Redis starter (spring-boot-starter-data-redis-reactive), then configure a route:

application.yml (gateway) 
spring:
  cloud:
    gateway:
      server:
        webflux:
          routes:
            - id: product-service
              uri: lb://product-service
              predicates:
                - Path=/api/products/**
              filters:
                - StripPrefix=1
                - name: RequestRateLimiter
                  args:
                    redis-rate-limiter.replenishRate: 2      # tokens per second (about 120/minute)
                    redis-rate-limiter.burstCapacity: 20     # maximum short burst
                    redis-rate-limiter.requestedTokens: 1    # tokens per request
                    key-resolver: "#{@apiKeyResolver}"
RateLimitKeyConfig.java 
package com.example.gateway.config;

import org.springframework.cloud.gateway.filter.ratelimit.KeyResolver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import reactor.core.publisher.Mono;

@Configuration
public class RateLimitKeyConfig {

    // One bucket per API key; requests without a key fall back to the IP address
    @Bean
    public KeyResolver apiKeyResolver() {
        return exchange -> {
            String apiKey = exchange.getRequest().getHeaders().getFirst("X-API-Key");
            if (apiKey != null) {
                return Mono.just("key:" + apiKey);
            }
            return Mono.justOrEmpty(exchange.getRequest().getRemoteAddress())
                    .map(address -> "ip:" + address.getAddress().getHostAddress())
                    .defaultIfEmpty("ip:unknown");
        };
    }
}

Because the counters live in Redis, every gateway instance shares the same limits. Rejected requests get 429, and the gateway adds headers such as X-RateLimit-Remaining automatically.

8. Limiting Outgoing Calls with Resilience4j

Rate limiting also protects the services you call. A paid third-party API may allow only 10 requests per second; going over it might cost money or get your account blocked. Resilience4j’s rate limiter caps outgoing calls:

application.yml 
resilience4j:
  ratelimiter:
    instances:
      currency-api:
        limit-for-period: 10          # 10 calls...
        limit-refresh-period: 1s      # ...per second
        timeout-duration: 200ms       # wait up to 200 ms for permission
CurrencyClient.java 
package com.example.order.client;

import java.math.BigDecimal;

import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;

import io.github.resilience4j.ratelimiter.RequestNotPermitted;
import io.github.resilience4j.ratelimiter.annotation.RateLimiter;

@Component
public class CurrencyClient {

    private final RestClient restClient;
    private final ExchangeRateCache cache;

    public CurrencyClient(RestClient.Builder builder, ExchangeRateCache cache) {
        this.restClient = builder.baseUrl("https://api.currency.example").build();
        this.cache = cache;
    }

    @RateLimiter(name = "currency-api", fallbackMethod = "cachedRate")
    public BigDecimal getRate(String from, String to) {
        BigDecimal rate = restClient.get()
                .uri("/rates?from={from}&to={to}", from, to)
                .retrieve()
                .body(BigDecimal.class);
        cache.put(from, to, rate);
        return rate;
    }

    // Used when our own limit is reached
    private BigDecimal cachedRate(String from, String to, RequestNotPermitted ex) {
        return cache.get(from, to);
    }
}

The annotation requires the resilience4j-spring-boot3 dependency and Spring AOP. Note that this limit applies per application instance; with several instances, divide the provider’s limit between them or use a shared limiter.

9. Distributed Rate Limiting with Redis

Microservices usually run as several instances behind a load balancer. If each instance keeps its own counters, a client spread across three instances gets three times the intended limit. The solution is a shared store, most often Redis, that every instance updates.

Here is a simple fixed-window limiter using Redis’s atomic INCR command with Spring Data Redis:

RedisRateLimiter.java 
package com.example.ratelimit;

import java.time.Duration;
import java.time.Instant;

import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;

@Component
public class RedisRateLimiter {

    private static final int LIMIT = 100;
    private static final Duration WINDOW = Duration.ofMinutes(1);

    private final StringRedisTemplate redis;

    public RedisRateLimiter(StringRedisTemplate redis) {
        this.redis = redis;
    }

    public boolean tryAcquire(String clientKey) {
        // One key per client per minute, for example "rate:key:abc123:29852160"
        long window = Instant.now().getEpochSecond() / WINDOW.toSeconds();
        String redisKey = "rate:" + clientKey + ":" + window;

        // INCR is atomic, so concurrent requests on all instances are counted correctly
        Long count = redis.opsForValue().increment(redisKey);

        if (count != null && count == 1) {
            // First request in this window: let Redis delete the key afterwards
            redis.expire(redisKey, WINDOW.plusSeconds(5));
        }
        return count != null && count <= LIMIT;
    }
}
  • All instances share the same counter, so the limit is exact across the cluster.
  • Keys expire automatically, so Redis does not fill up with old windows.
  • For a token bucket or sliding window in Redis, run the logic as a small Lua script, so the read and update happen atomically in one step. Spring Cloud Gateway and Bucket4j already do this for you.
  • Decide what happens if Redis is down: usually allow requests (“fail open”) so a Redis problem does not take down your API, and alert on it.

10. Choosing the Rate Limiting Key

KeyGood forWatch out for
API keyPublic and partner APIs; supports per-plan limitsKeys can be shared or leaked
User ID (from the login token)Fair limits for logged-in usersNot available before login
IP addressAnonymous traffic, login pages, basic abuse protectionMany users can share one IP (offices, mobile networks); behind proxies, read the client IP carefully
EndpointProtecting expensive operations, such as search or reportsCombine with a client key so one client cannot use the whole budget
CombinationFor example user + endpoint, or IP + endpoint for loginMore keys to store

Use different limits for different endpoints: stricter for expensive or sensitive ones (search, login, password reset) and relaxed or none for cheap ones (health checks, static data).

11. Best Practices

  1. Use the API gateway for consistent, central limits across all services, and add service-level limits for special cases.
  2. Choose the right key: API key, user ID, IP address or a combination, based on your security and business needs.
  3. Communicate limits clearly with 429, Retry-After and rate limit headers, and document them.
  4. Differentiate between endpoints: stricter limits for expensive operations such as search, relaxed ones for cheap requests.
  5. Use shared storage, such as Redis, when services run as several instances.
  6. Allow reasonable bursts with a token bucket, so normal users clicking quickly are not punished.
  7. Offer tiers: different limits for free, paid and internal clients.
  8. Never limit health checks or monitoring traffic.
  9. Monitor and adapt: track rejections per client and endpoint, and adjust limits to real traffic and capacity.
  10. Combine with other patterns such as circuit breakers, bulkheads and timeouts for complete protection.

12. Common Mistakes to Avoid

MistakeWhat goes wrongBetter approach
In-memory counters with many instancesClients get several times the intended limitStore counters in Redis or use the gateway’s shared limiter
Limiting only by IP addressWhole offices or mobile networks are blocked togetherPrefer API keys or user IDs where possible
Returning 500 or 503 instead of 429Clients cannot tell overload from a real errorReturn 429 with Retry-After
Clients retrying immediatelyRejected traffic multiplies and makes overload worseRespect Retry-After and use exponential backoff
Limits that are too strictNormal users hit errors during regular useBase limits on measured traffic and allow bursts
One limit for every endpointCheap endpoints are blocked, expensive ones are under-protectedSet limits per endpoint group
Rate limiting health checksLoad balancers think healthy instances are downExclude health and monitoring endpoints
Unbounded per-client mapsMemory grows with every new clientUse caches with expiry or external storage

13. Summary

The rate limiting pattern is a foundation of resilient, secure and fair microservices. It controls how many requests each client may make in a time window, protects services from spikes, bugs and attacks, keeps cloud costs predictable and enables fair, tiered plans. Choose an algorithm that fits your needs, usually a token bucket or sliding window; enforce limits centrally at the API gateway with shared Redis storage; protect outgoing calls with Resilience4j; and always answer with 429 and clear headers so clients can back off politely. With thoughtful limits in place, your services stay available and fast for everyone, even under unexpected load. Continue with the Bulkhead pattern to learn how to limit concurrent calls as well.