API Gateway vs Load Balancer
An API gateway and a load balancer both receive requests and send them on, so diagrams often show them as interchangeable boxes. They answer different questions. The API gateway decides which service should handle a call and whether the caller is allowed to make it. The load balancer decides which instance of that service should do the work.
This tutorial explains the split, shows how the two cooperate in a microservices system, and gives Spring Cloud Gateway and Kubernetes examples. It builds on Load Balancing and API Gateway vs Reverse Proxy.
1. What Are an API Gateway and a Load Balancer?
An API gateway is the single entry point for API clients. It authenticates callers, applies rate limits and other policies, routes each call to the right service, and can transform or aggregate requests.
A load balancer distributes requests or connections across multiple healthy instances of the same service, using an algorithm and health checks, to provide scalability and availability. It can work at Layer 4 (TCP) or Layer 7 (HTTP).
Put simply, the gateway is a traffic controller for APIs, while the load balancer is a dispatcher for identical workers.
2. How They Fit Together
In this design the gateway sends traffic to services, and a load balancer inside each service spreads it across instances. In Spring Cloud the gateway often performs this balancing itself through Spring Cloud LoadBalancer.
3. Step by Step
- A client calls
POST /api/orderswith a JWT. - A cloud load balancer in front of the gateway cluster picks one gateway instance, because the gateway is itself scaled out.
- The gateway validates the token, checks the caller's rate limit, and matches the route to
orders-service. - Spring Cloud LoadBalancer looks up the healthy instances of
orders-servicein the registry and picksorders-2with Round Robin. - The order is created and the response travels back through the gateway.
Notice that load balancing appears twice: once in front of the gateways and once behind them. That is normal in production.
Failure handling
The two layers react to failures differently. When an instance stops passing health checks, the load balancer removes it within seconds and spreads traffic over the remaining instances. When a whole service is slow or down, the gateway is the right place for a circuit breaker: it returns a fallback response or a clear 503 instead of letting requests pile up. Spring Cloud Gateway provides this through its CircuitBreaker filter, which can use Resilience4j.
4. Key Concepts
Different questions
Gateway: who is calling and which service should answer. Load balancer: which healthy instance should answer.
Layer of operation
Gateways work at Layer 7 and understand HTTP, tokens and APIs. Load balancers can be Layer 4 or Layer 7.
Server-side vs client-side balancing
Load balancing can be a separate component, such as AWS ALB or a Kubernetes Service, or a library in the caller, such as Spring Cloud LoadBalancer; see client-side load balancing.
Overlap
Many gateways include balancing, and many Layer 7 balancers offer basic routing and authentication. The difference is the main responsibility, not a strict feature list.
Service mesh
Inside a cluster, a service mesh such as Istio or Linkerd moves load balancing, retries and mTLS into sidecar or node-level proxies. The API gateway then handles north-south traffic from clients, while the mesh handles east-west traffic between services.
5. Practical Implementation
Spring Cloud Gateway with built-in load balancing
The lb:// scheme makes the gateway resolve the service name and balance across its instances.
spring:
cloud:
gateway:
server:
webflux:
routes:
- id: orders
uri: lb://orders-service # load-balanced across registered instances
predicates:
- Path=/api/orders/**
filters:
- StripPrefix=1 # /api/orders/42 -> /orders/42
- name: RequestRateLimiter
args:
redis-rate-limiter.replenishRate: 10
redis-rate-limiter.burstCapacity: 20
key-resolver: "#{@userKeyResolver}"
- id: users
uri: lb://users-service
predicates:
- Path=/api/users/**
filters:
- StripPrefix=1
security:
oauth2:
resourceserver:
jwt:
issuer-uri: https://auth.example.com/realms/shop # validate JWT access tokens
spring-cloud-starter-gateway-server-webflux starter. Older releases use spring.cloud.gateway.routes without server.webflux. The rate limiter also needs the reactive Redis starter, and JWT validation needs the OAuth2 resource server starter.Kubernetes: gateway behind a cloud load balancer
A LoadBalancer Service spreads traffic across the gateway pods, and each backend Service balances across its own pods.
apiVersion: v1
kind: Service
metadata:
name: api-gateway
spec:
type: LoadBalancer # cloud load balancer -> gateway pods
selector:
app: api-gateway
ports:
- port: 443
targetPort: 8443
---
apiVersion: v1
kind: Service
metadata:
name: orders-service # balances across all ready orders pods
spec:
selector:
app: orders
ports:
- port: 8080
targetPort: 8080
6. Advantages and Disadvantages
| API gateway | Load balancer | |
|---|---|---|
| Main benefit | Security and API policy in one place | Scaling and high availability |
| Understands callers | Yes | Usually no |
| Performance cost | Higher (token checks, filters) | Low, especially at Layer 4 |
| Main risk | Bottleneck or misplaced business logic | Single point of failure if not redundant |
7. API Gateway vs Load Balancer Comparison
| Aspect | API gateway | Load balancer |
|---|---|---|
| Decides | Which service, and whether the call is allowed | Which instance of a service |
| Layer | Layer 7 | Layer 4 or Layer 7 |
| Authentication and rate limits | Core features | Rare or basic |
| Health checks and algorithms | Often delegated to a balancer | Core features |
| Request transformation | Yes | No (Layer 4) or minimal |
| Examples | Spring Cloud Gateway, Kong, AWS API Gateway | AWS ALB/NLB, HAProxy, Nginx upstream, Kubernetes Service |
8. Real-World Use Cases
- Microservices platform: cloud load balancer, then gateway cluster, then per-service balancing to pods.
- Monolith on several servers: a load balancer alone is enough; there is only one service to route to.
- Public partner API: the gateway enforces API keys and quotas; balancers keep each backend highly available.
- Internal service-to-service calls: usually skip the gateway and use client-side balancing or a service mesh.
Example: a sale-day traffic peak
An online store expects ten times its normal traffic. The team scales the orders service from 4 to 20 pods, and the Kubernetes Service and the gateway's lb:// route start using the new pods automatically. That is the load balancing layer doing its job. At the same time, the gateway enforces per-client rate limits so that one misbehaving partner integration cannot consume the extra capacity meant for shoppers. Each layer handles a different part of the peak.
9. Best Practices
- Never run a single gateway instance: put the gateway cluster behind a load balancer.
- Let each component do its job: policy at the gateway, instance selection at the balancer.
- Use health checks everywhere: gateway instances and service instances both need them.
- Keep internal calls off the public gateway to avoid extra hops and coupling.
- Monitor both layers: gateway latency per route, and balancer distribution per instance.
- Load test both layers: confirm that the gateway, not only the services, can handle peak traffic.
10. Summary
- The API gateway chooses the service and enforces API policy.
- The load balancer chooses a healthy instance of a service.
- Production systems use both, often with balancing before and after the gateway.
- Spring Cloud Gateway balances across instances with
lb://URIs. - A monolith may need only a load balancer; microservices with public APIs usually need both.
Finally, put all three components together in Reverse Proxy vs API Gateway vs Load Balancer.
API Gateway vs Load Balancer: Top Interview Questions
These are the API gateway and load balancer questions most often asked in system design interviews, with short answers you can explain in your own words.
Q1. Which question does an API gateway answer, and which does a load balancer answer?
The API gateway decides which service should handle a call and whether the caller is allowed, applying authentication, rate limits and routing. The load balancer decides which healthy instance of a service handles it, using an algorithm and health checks.
Q2. Do you need a load balancer if you have an API gateway?
Usually yes. The gateway itself runs as several instances and needs a load balancer in front of it, and each backend service needs balancing across its instances. The gateway may perform that second step itself.
Q3. Can a load balancer replace an API gateway?
Only for simple routing. A Layer 7 load balancer can route by path and sometimes validate tokens, but it typically lacks per-consumer rate limits, transformation, aggregation and API analytics.
Q4. Where does load balancing happen in Spring Cloud Gateway?
When a route uses an lb://service-name URI, Spring Cloud LoadBalancer resolves the service through a discovery client and picks an instance for each request, using Round Robin by default.