Microservices API Gateway Pattern

The front door to your microservices. In modern software architecture, microservices have become a popular way to build scalable and flexible applications. Instead of one large application, you have many small, independent services, each responsible for one business function. But how do clients talk to all of them? The answer is the API Gateway pattern.

This guide explains what an API gateway is, what it does, and when to use it. You will then build a working gateway with Spring Cloud Gateway, step by step: routing, filters, security, rate limiting, fault tolerance and response aggregation.

1. The Problem: Clients Talking to Many Services

Imagine an online shop built from four microservices: users, products, orders and inventory. Without a gateway, the mobile app must know about and call every one of them directly.

An API gateway solves these problems by putting one entry point in front of all services.

2. What Is an API Gateway?

An API gateway is a server that acts as the single entry point for all client requests. It sits between the clients (web apps, mobile apps, partner systems) and your backend services. Clients talk only to the gateway; the gateway forwards each request to the right service and returns the response.

   Web app     Mobile app    Partner
       \           |           /
        \          |          /
         +---------+---------+
                   |
             [ API Gateway ]
                   |
     +---------+---+-----+----------+
     |         |         |          |
   Users   Products   Orders   Inventory

3. How a Request Flows Through the Gateway

Every request passes through the same steps inside the gateway:

1. Receive the request from the client, for example GET /api/orders/42.
2. Authenticate the caller by checking their token.
3. Apply policies such as rate limits.
4. Route: find the matching rule and pick the target service, here the order service.
5. Transform the request if needed, for example add headers or rewrite the path.
6. Forward the request to a healthy instance of the service, with a timeout.
7. Return the response to the client, possibly transformed, and record logs and metrics.

4. Core Responsibilities of an API Gateway

Other common jobs include CORS handling, SSL/TLS termination (handling HTTPS so internal services do not have to), request validation and adding correlation IDs for tracing.

The key idea: these are cross-cutting concerns that every service would otherwise implement separately. Moving them to the gateway keeps each service focused on its business logic.

5. A Concrete Example: The E-Commerce Application

Our example shop has four microservices:

• User service: manages user accounts and profiles.
• Product service: handles product information and details.
• Order service: processes orders and payments.
• Inventory service: tracks product stock levels.

The mobile app needs to show a customer’s order history, including product names and current stock.

5.1 Scenario 1: without an API gateway

1. Call the user service to check the login.
2. Call the order service for the list of orders.
3. Call the product service once for each product in those orders.
4. Call the inventory service for stock levels.
5. Combine all the data inside the app.

Result: many direct calls over a slow mobile network, poor performance and complicated client code that breaks whenever a service changes.

5.2 Scenario 2: with an API gateway

The app makes one request: GET /my/order-history. Behind the scenes:

1. The mobile app sends the request with its authentication token.
2. The gateway validates the token.
3. The gateway calls the order, product and inventory services over the fast internal network, in parallel where possible.
4. It combines the responses into a single JSON object.
5. It returns that one response to the app.

Result: faster screens, simpler client code and a better user experience. The services can change internally without the app noticing.

6. Building a Gateway with Spring Cloud Gateway

Spring Cloud Gateway is the gateway from the Spring family. It is built on Spring Boot, so it is a natural choice for Java teams. A gateway is simply a separate Spring Boot application.

6.1 Dependency

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-gateway-server-webflux</artifactId>
</dependency>

Also import the Spring Cloud BOM (spring-cloud-dependencies) in <dependencyManagement> so that versions match your Spring Boot version.

6.2 Routes in configuration

A route has three parts: an ID, a predicate that decides which requests match, and a URI to forward them to. Filters can change the request or response on the way.

server:
  port: 8080

spring:
  application:
    name: shop-gateway
  cloud:
    gateway:
      server:
        webflux:
          routes:
            - id: product-service
              uri: http://localhost:8081
              predicates:
                - Path=/api/products/**
              filters:
                - StripPrefix=1          # /api/products/7 -> /products/7

            - id: order-service
              uri: http://localhost:8082
              predicates:
                - Path=/api/orders/**
              filters:
                - StripPrefix=1

            - id: user-service
              uri: http://localhost:8083
              predicates:
                - Path=/api/users/**
              filters:
                - StripPrefix=1

Clients now call only port 8080. A request to http://localhost:8080/api/products/7 is forwarded to the product service as /products/7.

7. Defining Routes in Java

The same routes can be written in Java, which is handy when you need logic or want compile-time checking:

package com.example.gateway.config;

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.HttpMethod;

@Configuration
public class GatewayRoutesConfig {

    @Bean
    public RouteLocator shopRoutes(RouteLocatorBuilder builder) {
        return builder.routes()
                // Anyone may read products
                .route("product-service", r -> r
                        .path("/api/products/**")
                        .and().method(HttpMethod.GET)
                        .filters(f -> f
                                .stripPrefix(1)
                                .addResponseHeader("Cache-Control", "max-age=60"))
                        .uri("lb://product-service"))

                // Orders, with a fallback if the service is down
                .route("order-service", r -> r
                        .path("/api/orders/**")
                        .filters(f -> f
                                .stripPrefix(1)
                                .circuitBreaker(cb -> cb
                                        .setName("orders")
                                        .setFallbackUri("forward:/fallback/orders")))
                        .uri("lb://order-service"))

                // Send 10% of user traffic to the new version (canary release)
                .route("user-service-v2", r -> r
                        .path("/api/users/**")
                        .and().weight("users", 10)
                        .filters(f -> f.stripPrefix(1))
                        .uri("lb://user-service-v2"))
                .route("user-service-v1", r -> r
                        .path("/api/users/**")
                        .and().weight("users", 90)
                        .filters(f -> f.stripPrefix(1))
                        .uri("lb://user-service"))
                .build();
    }
}

The weighted routes show a common gateway trick: a canary release. A new version receives a small share of real traffic first, and you increase it once you are confident it works.

8. Filters: Changing Requests and Responses

8.1 Built-in filters

8.2 A custom global filter: correlation IDs

A global filter runs for every route. This one gives each request a correlation ID, so the same request can be followed through the logs of every service:

package com.example.gateway.filter;

import java.util.UUID;

import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.GlobalFilter;
import org.springframework.core.Ordered;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;

import reactor.core.publisher.Mono;

@Component
public class CorrelationIdFilter implements GlobalFilter, Ordered {

    private static final String HEADER = "X-Correlation-Id";

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String incoming = exchange.getRequest().getHeaders().getFirst(HEADER);
        String correlationId = (incoming != null) ? incoming : UUID.randomUUID().toString();

        // Pass the ID on to the backend service...
        ServerHttpRequest request = exchange.getRequest().mutate()
                .header(HEADER, correlationId)
                .build();

        // ...and return it to the client, which helps with support requests
        exchange.getResponse().getHeaders().add(HEADER, correlationId);

        return chain.filter(exchange.mutate().request(request).build());
    }

    @Override
    public int getOrder() {
        return Ordered.HIGHEST_PRECEDENCE;   // run before other filters
    }
}

Spring Cloud Gateway (WebFlux edition) is built on Spring WebFlux, so filters return a Mono. Avoid blocking calls, such as plain JDBC, inside filters, because they would stall the gateway’s small pool of threads.

9. Securing APIs at the Gateway

Checking tokens at the gateway means unauthenticated requests are rejected before they reach any service. Add Spring Security’s resource server support:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
</dependency>

spring:
  security:
    oauth2:
      resourceserver:
        jwt:
          issuer-uri: https://login.example.com/realms/shop

package com.example.gateway.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.HttpMethod;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.web.reactive.EnableWebFluxSecurity;
import org.springframework.security.config.web.server.ServerHttpSecurity;
import org.springframework.security.web.server.SecurityWebFilterChain;

@Configuration
@EnableWebFluxSecurity
public class GatewaySecurityConfig {

    @Bean
    public SecurityWebFilterChain securityWebFilterChain(ServerHttpSecurity http) {
        http
            .authorizeExchange(exchange -> exchange
                // Product browsing is public
                .pathMatchers(HttpMethod.GET, "/api/products/**").permitAll()
                .pathMatchers("/actuator/health").permitAll()
                // Everything else needs a valid token
                .anyExchange().authenticated()
            )
            .oauth2ResourceServer(oauth -> oauth.jwt(Customizer.withDefaults()))
            // Stateless token API: no cookies, so CSRF protection is not needed
            .csrf(ServerHttpSecurity.CsrfSpec::disable);

        return http.build();
    }
}

The gateway forwards the Authorization header to the services, so they can still read the user’s identity and make their own detailed decisions, such as “customers may only see their own orders”.

10. Rate Limiting

Rate limiting protects your services from overload, abuse and runaway clients. Spring Cloud Gateway includes a Redis-based limiter using the token bucket algorithm. Add the reactive Redis starter (spring-boot-starter-data-redis-reactive), then configure the filter on a route:

filters:
  - StripPrefix=1
  - name: RequestRateLimiter
    args:
      redis-rate-limiter.replenishRate: 10     # tokens added per second
      redis-rate-limiter.burstCapacity: 20     # maximum short burst
      key-resolver: "#{@clientIpKeyResolver}"

The key resolver decides what counts as “one client”. Here it is the caller’s IP address:

package com.example.gateway.config;

import org.springframework.cloud.gateway.filter.ratelimit.KeyResolver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import reactor.core.publisher.Mono;

@Configuration
public class RateLimitConfig {

    @Bean
    public KeyResolver clientIpKeyResolver() {
        return exchange -> Mono.justOrEmpty(exchange.getRequest().getRemoteAddress())
                .map(address -> address.getAddress().getHostAddress())
                .defaultIfEmpty("unknown");
    }
}

Each client may make about 10 requests per second, with short bursts up to 20. Extra requests receive 429 Too Many Requests. For logged-in users, limiting by user ID (taken from the token) is usually fairer than limiting by IP address, because many users can share one IP.

11. Timeouts, Circuit Breakers and Fallbacks

The gateway calls every service, so one slow service could tie up the gateway and hurt everyone. Protect it with timeouts and circuit breakers.

spring:
  cloud:
    gateway:
      server:
        webflux:
          httpclient:
            connect-timeout: 2000      # milliseconds
            response-timeout: 5s

The circuit breaker used in section 7 needs the Resilience4j starter, spring-cloud-starter-circuitbreaker-reactor-resilience4j. When the order service fails repeatedly, the circuit “opens” and requests go straight to the fallback instead of waiting for timeouts:

package com.example.gateway.fallback;

import java.util.Map;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class FallbackController {

    @RequestMapping("/fallback/orders")
    public ResponseEntity<Map<String, String>> ordersFallback() {
        return ResponseEntity
                .status(HttpStatus.SERVICE_UNAVAILABLE)
                .body(Map.of("message", "Orders are temporarily unavailable. Please try again in a moment."));
    }
}

Clients get a fast, friendly answer instead of a long wait and a cryptic error. Learn more in the Circuit Breaker Pattern guide.

12. Aggregating Responses from Several Services

Routing forwards one request to one service. For screens that need data from several services, the gateway can call them in parallel and combine the results. This implements scenario 2 from section 5 in a simplified form:

package com.example.gateway.aggregation;

import java.util.List;

import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.security.oauth2.jwt.Jwt;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.reactive.function.client.WebClient;

import reactor.core.publisher.Mono;

@RestController
public class DashboardController {

    private final WebClient webClient;

    public DashboardController(WebClient.Builder builder) {
        this.webClient = builder.build();
    }

    @GetMapping("/my/dashboard")
    public Mono<Dashboard> dashboard(@AuthenticationPrincipal Jwt jwt) {
        String userId = jwt.getSubject();

        Mono<UserProfile> profile = webClient.get()
                .uri("http://localhost:8083/users/{id}", userId)
                .headers(h -> h.setBearerAuth(jwt.getTokenValue()))
                .retrieve()
                .bodyToMono(UserProfile.class);

        Mono<List<OrderSummary>> recentOrders = webClient.get()
                .uri("http://localhost:8082/orders?customerId={id}&limit=5", userId)
                .headers(h -> h.setBearerAuth(jwt.getTokenValue()))
                .retrieve()
                .bodyToFlux(OrderSummary.class)
                .collectList();

        // Both calls run at the same time; the result is ready when both finish
        return Mono.zip(profile, recentOrders, Dashboard::new);
    }

    public record UserProfile(String id, String name, String email) {
    }

    public record OrderSummary(Long id, String status, double total) {
    }

    public record Dashboard(UserProfile profile, List<OrderSummary> recentOrders) {
    }
}

Keep aggregation simple: combining and reshaping data is fine, but real business rules belong in the services. If aggregation logic grows large, move it into a dedicated service or a backend for frontend.

13. Backend for Frontend (BFF)

Different clients often need different APIs. A mobile app wants small responses to save data, while a web dashboard wants rich, detailed ones. One gateway that serves everyone can become cluttered with special cases.

The Backend for Frontend (BFF) pattern solves this by giving each type of client its own gateway:

 Mobile app      Web app      Partners
     |              |             |
 [Mobile BFF]   [Web BFF]   [Partner API]
      \             |             /
       +------------+------------+
                    |
        Users  Products  Orders  Inventory

Many systems combine both: a shared edge gateway handles security and rate limiting, and small BFFs behind it shape the data for each client.

14. API Gateway vs Load Balancer, Reverse Proxy and Service Mesh

These tools work together rather than replace each other. A typical setup is: cloud load balancer → API gateway → services, with an optional service mesh such as Istio or Linkerd handling traffic between services.

15. Popular API Gateway Solutions

How to choose: a Java and Spring team that wants full control often picks Spring Cloud Gateway. Teams fully on one cloud often pick that cloud’s managed gateway to avoid running servers. Organizations that publish APIs to many external partners often need a full API management platform such as Apigee, Kong or Azure API Management.

16. Benefits and Drawbacks

Avoid the single point of failure by running several gateway instances behind a load balancer, across availability zones. Keep the extra latency small by keeping filters light and avoiding blocking calls.

17. Best Practices

1. Keep the gateway thin. Routing and cross-cutting concerns only; business logic belongs in services.
2. Run several instances behind a load balancer so the gateway is never a single point of failure.
3. Set timeouts on every route and use circuit breakers with fallbacks for important services.
4. Validate tokens at the gateway, and still authorize in the services (defense in depth).
5. Apply rate limits per client or per user, and return 429 with a clear message.
6. Add correlation IDs and central logging, so a request can be followed across services.
7. Use service discovery (lb://) instead of hard-coded addresses.
8. Version your APIs, for example /api/v1/..., and use the gateway to route versions.
9. Keep internal services private: only the gateway should be reachable from the internet.
10. Monitor the gateway closely: latency, error rates and rate-limit rejections per route.

18. Common Mistakes to Avoid

19. Summary

The API gateway is the front door of a microservices architecture. It gives clients a single entry point, routes each request to the right service, and handles cross-cutting concerns such as authentication, rate limiting, caching, logging and response aggregation in one place. With Spring Cloud Gateway you can define routes in YAML or Java, add filters, secure APIs with JWT, limit traffic with Redis, and protect the system with timeouts and circuit breakers. Keep the gateway thin and highly available, combine it with backends for frontends when clients differ a lot, and your microservices stay independent while the outside world sees one clean, secure and scalable API. Continue with the Circuit Breaker Pattern to learn more about keeping services resilient.

20. Frequently Asked Questions (FAQ)

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-gateway-server-webmvc</artifactId>
</dependency>

routes:
  - id: orders-v1
    uri: lb://order-service
    predicates:
      - Path=/api/v1/orders/**
  - id: orders-v2
    uri: lb://order-service-v2
    predicates:
      - Path=/api/v2/orders/**

spring:
  cloud:
    gateway:
      server:
        webflux:
          globalcors:
            cors-configurations:
              '[/api/**]':
                allowed-origins: "https://shop.example.com"
                allowed-methods: GET,POST,PUT,DELETE
                allowed-headers: Authorization,Content-Type

filters:
  - name: Retry
    args:
      retries: 3
      methods: GET
      statuses: BAD_GATEWAY,SERVICE_UNAVAILABLE

- id: notifications
  uri: lb:ws://notification-service
  predicates:
    - Path=/ws/notifications/**

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@AutoConfigureWebTestClient
class GatewayRoutesTest {

    @Autowired
    private WebTestClient client;

    @Test
    void productRequestsAreRoutedAndPrefixIsStripped() {
        client.get().uri("/api/products/7")
              .exchange()
              .expectStatus().isOk();
    }

    @Test
    void ordersRequireAToken() {
        client.get().uri("/api/orders/1")
              .exchange()
              .expectStatus().isUnauthorized();
    }
}