00:00

API Gateway vs Load Balancer

Last updated:

An API gateway and a load balancer both receive requests and send them on, so diagrams often show them as interchangeable boxes. They answer different questions. The API gateway decides which service should handle a call and whether the caller is allowed to make it. The load balancer decides which instance of that service should do the work.

This tutorial explains the split, shows how the two cooperate in a microservices system, and gives Spring Cloud Gateway and Kubernetes examples. It builds on Load Balancing and API Gateway vs Reverse Proxy.

1. What Are an API Gateway and a Load Balancer?

An API gateway is the single entry point for API clients. It authenticates callers, applies rate limits and other policies, routes each call to the right service, and can transform or aggregate requests.

A load balancer distributes requests or connections across multiple healthy instances of the same service, using an algorithm and health checks, to provide scalability and availability. It can work at Layer 4 (TCP) or Layer 7 (HTTP).

Put simply, the gateway is a traffic controller for APIs, while the load balancer is a dispatcher for identical workers.

2. How They Fit Together

API gateway compared with load balancer A client calls the API gateway, which authenticates the request and chooses the right service: orders or users. For each service, a load balancer then spreads requests across its instances: orders-1 and orders-2, users-1 and users-2. The gateway decides which service; the load balancer decides which instance. Client API Gatewayauth · rate limitpicks the service ORDERS-SERVICE Load balancerRound Robin orders-1 orders-2 USERS-SERVICE Load balancerRound Robin users-1 users-2
The gateway answers "which service?". The load balancer answers "which instance of it?".

In this design the gateway sends traffic to services, and a load balancer inside each service spreads it across instances. In Spring Cloud the gateway often performs this balancing itself through Spring Cloud LoadBalancer.

3. Step by Step

  1. A client calls POST /api/orders with a JWT.
  2. A cloud load balancer in front of the gateway cluster picks one gateway instance, because the gateway is itself scaled out.
  3. The gateway validates the token, checks the caller's rate limit, and matches the route to orders-service.
  4. Spring Cloud LoadBalancer looks up the healthy instances of orders-service in the registry and picks orders-2 with Round Robin.
  5. The order is created and the response travels back through the gateway.

Notice that load balancing appears twice: once in front of the gateways and once behind them. That is normal in production.

Failure handling

The two layers react to failures differently. When an instance stops passing health checks, the load balancer removes it within seconds and spreads traffic over the remaining instances. When a whole service is slow or down, the gateway is the right place for a circuit breaker: it returns a fallback response or a clear 503 instead of letting requests pile up. Spring Cloud Gateway provides this through its CircuitBreaker filter, which can use Resilience4j.

4. Key Concepts

Different questions

Gateway: who is calling and which service should answer. Load balancer: which healthy instance should answer.

Layer of operation

Gateways work at Layer 7 and understand HTTP, tokens and APIs. Load balancers can be Layer 4 or Layer 7.

Server-side vs client-side balancing

Load balancing can be a separate component, such as AWS ALB or a Kubernetes Service, or a library in the caller, such as Spring Cloud LoadBalancer; see client-side load balancing.

Overlap

Many gateways include balancing, and many Layer 7 balancers offer basic routing and authentication. The difference is the main responsibility, not a strict feature list.

Service mesh

Inside a cluster, a service mesh such as Istio or Linkerd moves load balancing, retries and mTLS into sidecar or node-level proxies. The API gateway then handles north-south traffic from clients, while the mesh handles east-west traffic between services.

5. Practical Implementation

Spring Cloud Gateway with built-in load balancing

The lb:// scheme makes the gateway resolve the service name and balance across its instances.

application.yml
spring:
  cloud:
    gateway:
      server:
        webflux:
          routes:
            - id: orders
              uri: lb://orders-service          # load-balanced across registered instances
              predicates:
                - Path=/api/orders/**
              filters:
                - StripPrefix=1                  # /api/orders/42 -> /orders/42
                - name: RequestRateLimiter
                  args:
                    redis-rate-limiter.replenishRate: 10
                    redis-rate-limiter.burstCapacity: 20
                    key-resolver: "#{@userKeyResolver}"
            - id: users
              uri: lb://users-service
              predicates:
                - Path=/api/users/**
              filters:
                - StripPrefix=1
  security:
    oauth2:
      resourceserver:
        jwt:
          issuer-uri: https://auth.example.com/realms/shop   # validate JWT access tokens
Note: These property names are for Spring Cloud 2025.0 and later, which use the spring-cloud-starter-gateway-server-webflux starter. Older releases use spring.cloud.gateway.routes without server.webflux. The rate limiter also needs the reactive Redis starter, and JWT validation needs the OAuth2 resource server starter.

Kubernetes: gateway behind a cloud load balancer

A LoadBalancer Service spreads traffic across the gateway pods, and each backend Service balances across its own pods.

gateway-k8s.yaml
apiVersion: v1
kind: Service
metadata:
  name: api-gateway
spec:
  type: LoadBalancer          # cloud load balancer -> gateway pods
  selector:
    app: api-gateway
  ports:
    - port: 443
      targetPort: 8443
---
apiVersion: v1
kind: Service
metadata:
  name: orders-service        # balances across all ready orders pods
spec:
  selector:
    app: orders
  ports:
    - port: 8080
      targetPort: 8080

6. Advantages and Disadvantages

API gatewayLoad balancer
Main benefitSecurity and API policy in one placeScaling and high availability
Understands callersYesUsually no
Performance costHigher (token checks, filters)Low, especially at Layer 4
Main riskBottleneck or misplaced business logicSingle point of failure if not redundant

7. API Gateway vs Load Balancer Comparison

AspectAPI gatewayLoad balancer
DecidesWhich service, and whether the call is allowedWhich instance of a service
LayerLayer 7Layer 4 or Layer 7
Authentication and rate limitsCore featuresRare or basic
Health checks and algorithmsOften delegated to a balancerCore features
Request transformationYesNo (Layer 4) or minimal
ExamplesSpring Cloud Gateway, Kong, AWS API GatewayAWS ALB/NLB, HAProxy, Nginx upstream, Kubernetes Service

8. Real-World Use Cases

  • Microservices platform: cloud load balancer, then gateway cluster, then per-service balancing to pods.
  • Monolith on several servers: a load balancer alone is enough; there is only one service to route to.
  • Public partner API: the gateway enforces API keys and quotas; balancers keep each backend highly available.
  • Internal service-to-service calls: usually skip the gateway and use client-side balancing or a service mesh.

Example: a sale-day traffic peak

An online store expects ten times its normal traffic. The team scales the orders service from 4 to 20 pods, and the Kubernetes Service and the gateway's lb:// route start using the new pods automatically. That is the load balancing layer doing its job. At the same time, the gateway enforces per-client rate limits so that one misbehaving partner integration cannot consume the extra capacity meant for shoppers. Each layer handles a different part of the peak.

9. Best Practices

  • Never run a single gateway instance: put the gateway cluster behind a load balancer.
  • Let each component do its job: policy at the gateway, instance selection at the balancer.
  • Use health checks everywhere: gateway instances and service instances both need them.
  • Keep internal calls off the public gateway to avoid extra hops and coupling.
  • Monitor both layers: gateway latency per route, and balancer distribution per instance.
  • Load test both layers: confirm that the gateway, not only the services, can handle peak traffic.

10. Summary

  • The API gateway chooses the service and enforces API policy.
  • The load balancer chooses a healthy instance of a service.
  • Production systems use both, often with balancing before and after the gateway.
  • Spring Cloud Gateway balances across instances with lb:// URIs.
  • A monolith may need only a load balancer; microservices with public APIs usually need both.

Finally, put all three components together in Reverse Proxy vs API Gateway vs Load Balancer.