Reverse Proxy vs API Gateway vs Load Balancer
Reverse proxy, API gateway and load balancer: three terms that appear in nearly every architecture diagram and are often used loosely. All three sit between clients and servers and forward requests, but each exists for a different reason.
This tutorial brings the three together. It defines each by its main responsibility, shows how they stack in a real microservices platform, and gives a decision table you can use in design reviews and interviews.
1. What Are They?
- Reverse proxy: fronts servers. It hides them, terminates TLS, caches, compresses and routes by host or path. See Reverse Proxy.
- Load balancer: spreads traffic across healthy instances of a service using an algorithm and health checks, at Layer 4 or Layer 7. See Load Balancing.
- API gateway: governs API calls. It authenticates consumers, enforces rate limits and quotas, routes to services and transforms requests.
A useful shorthand: the reverse proxy is about protection and delivery, the load balancer about distribution, and the API gateway about API policy.
2. How the Three Layer Together
3. One Request Through All Three
- A mobile app calls
https://api.shop.example/api/payments. - Load balancer: a cloud Layer 4 balancer spreads the TCP connection to one of several ingress nodes.
- Reverse proxy (ingress): terminates TLS, applies basic request limits, and forwards
/api/*to the gateway Service. A request for/static/*would be served here and never reach the gateway. - API gateway: validates the JWT, checks the caller's quota, and routes to
payments-service. - Load balancing again: the gateway, or the Kubernetes Service, picks one healthy payments pod.
- The response returns along the same path.
Every hop should pass on the original client address and a request ID. With X-Forwarded-For, each proxy appends the address it received the request from, so the header becomes a list such as 198.51.100.4, 10.0.0.12. Services should use the entry added by the first trusted proxy, not simply the first value, because a client can send a fake header.
When a layer fails, the request stops there, and the error tells you where. The edge balancer usually causes connection errors or timeouts, the reverse proxy returns 502 or 504 when it cannot reach the next hop, and the gateway returns 401, 429 or 503 for policy and circuit-breaker decisions.
4. Responsibilities Side by Side
| Responsibility | Reverse proxy | Load balancer | API gateway |
|---|---|---|---|
| Hide backend servers | Yes | Yes | Yes |
| TLS termination | Yes | Layer 7 only | Often |
| Caching and compression | Yes | Rarely | Sometimes |
| Algorithms and health checks | Basic | Core | Usually delegated |
| Authentication of API consumers | No | No | Core |
| Per-consumer rate limits | No (per IP at most) | No | Core |
| Request transformation and aggregation | Limited | No | Yes |
Overlap in modern products
Many products now cover more than one role. Envoy works as a reverse proxy and load balancer, and with extensions as an API gateway. The Kubernetes Gateway API describes HTTP, gRPC and TCP routing in one model, and many implementations add gateway features on top. Cloud Layer 7 load balancers terminate TLS, route by path and can validate tokens. When one product does several jobs, still decide and document which job each instance performs, so configuration does not drift.
5. Practical Implementation
A typical Kubernetes setup uses all three: a cloud load balancer created by the ingress controller's Service, an Ingress as the reverse proxy, and Spring Cloud Gateway as the API gateway.
Reverse proxy layer: Kubernetes Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: shop-entry
spec:
ingressClassName: nginx
tls:
- hosts: [api.shop.example]
secretName: shop-tls # TLS terminated here
rules:
- host: api.shop.example
http:
paths:
- path: /static
pathType: Prefix
backend:
service: { name: static-site, port: { number: 80 } }
- path: /api
pathType: Prefix
backend:
service: { name: api-gateway, port: { number: 8080 } }
API gateway layer: Spring Cloud Gateway
spring:
cloud:
gateway:
server:
webflux:
routes:
- id: orders
uri: lb://orders-service # load-balanced across registered instances
predicates:
- Path=/api/orders/**
filters:
- StripPrefix=1 # /api/orders/42 -> /orders/42
- name: RequestRateLimiter
args:
redis-rate-limiter.replenishRate: 10
redis-rate-limiter.burstCapacity: 20
key-resolver: "#{@userKeyResolver}"
- id: users
uri: lb://users-service
predicates:
- Path=/api/users/**
filters:
- StripPrefix=1
security:
oauth2:
resourceserver:
jwt:
issuer-uri: https://auth.example.com/realms/shop # validate JWT access tokens
spring-cloud-starter-gateway-server-webflux starter. Older releases use spring.cloud.gateway.routes without server.webflux. The rate limiter also needs the reactive Redis starter, and JWT validation needs the OAuth2 resource server starter.The load balancing layer needs no extra file: the ingress controller's LoadBalancer Service creates the cloud balancer, and lb:// routes or Kubernetes Services balance across pods.
6. Advantages and Disadvantages of Layering
Advantages
- Each layer stays simple and focused.
- Static content and attacks are stopped before they reach the API layer.
- Each layer scales independently.
Disadvantages
- Every hop adds latency, often around a millisecond within a region.
- More components to configure, secure and monitor.
- Overlapping features can be configured twice and drift apart, such as rate limits in both proxy and gateway.
7. Decision Table
| If you need to... | Use |
|---|---|
| Serve one web app with HTTPS and caching | Reverse proxy |
| Run several identical instances for capacity and failover | Load balancer |
| Balance non-HTTP traffic such as databases or MQTT | Layer 4 load balancer |
| Expose APIs with tokens, quotas and versioning | API gateway |
| Route many microservices behind one domain | Reverse proxy or ingress, plus an API gateway for APIs |
| Run a large public platform | All three, layered |
8. Real-World Use Cases
- Small business site: one Nginx instance as a reverse proxy, which also balances two app servers.
- SaaS product: a cloud Layer 7 balancer with TLS and an API gateway for the public API.
- Large microservices platform: a Layer 4 edge balancer, ingress reverse proxies, an API gateway cluster, and per-service balancing.
- Global consumer app: a CDN in front of everything; see Reverse Proxy vs CDN.
Latency budget
Each layer adds a little time. Inside one region, a well-tuned proxy hop typically costs around a millisecond, while token validation and rate-limit lookups in Redis add a few more. For an API that must answer in 200 ms that overhead is small, but it adds up when requests also pass through a CDN, a service mesh and several internal calls. Measure each hop with distributed tracing before adding another layer.
9. Best Practices
- Assign one owner per concern: decide where TLS ends, where rate limits live and where authentication happens, and document it.
- Avoid unnecessary layers: start with one component that covers your needs, and add layers when requirements appear.
- Make every layer redundant and health-checked.
- Pass client context consistently: keep
X-Forwarded-For, request IDs and trace headers through every hop. - Secure internal hops: use network policies or mTLS between layers so traffic cannot skip the gateway.
Common mistakes
- Terminating and re-establishing TLS at every layer without a reason, which adds latency.
- Configuring rate limits in three places with different values.
- Health checks that test only the proxy, not the service behind it.
- Keep every layer's configuration in version control and review changes to routes, limits and TLS settings like code.
10. Summary
- Reverse proxy: fronts and protects servers (TLS, caching, routing).
- Load balancer: distributes traffic across healthy instances (algorithms, health checks).
- API gateway: governs API calls (authentication, rate limits, transformation).
- One product can play several roles; the distinction is about responsibility.
- Large systems layer them: edge balancer, then reverse proxy or ingress, then API gateway, then service balancing.
Next, move from networking to data storage with ACID vs BASE.
Reverse Proxy vs API Gateway vs Load Balancer: Top Interview Questions
These are the reverse proxy, API gateway and load balancer questions most often asked in system design interviews, with short answers you can explain in your own words.
Q1. Explain the difference between a reverse proxy, a load balancer and an API gateway.
A reverse proxy fronts servers and handles TLS, caching and routing. A load balancer spreads traffic across healthy instances. An API gateway governs API calls with authentication, rate limits and transformation. They overlap, and one product can do several jobs.
Q2. In what order do they usually appear in a request path?
Commonly: edge load balancer, then reverse proxy or ingress, then API gateway, then per-service load balancing to instances. Small systems collapse several of these into one component.
Q3. Can one tool do all three jobs?
Yes. Nginx, Envoy or a cloud Layer 7 balancer can proxy, balance and apply some API policies. Dedicated gateways are chosen when API-specific features such as per-consumer quotas or developer portals are needed.
Q4. Where should rate limiting be implemented in this stack?
Coarse, per-IP limits fit at the edge or reverse proxy to absorb abuse early. Per-consumer and per-plan limits belong in the API gateway, where the caller's identity is known.
Q5. How do you prevent clients from bypassing the gateway?
Expose services only on private networks, use Kubernetes network policies or security groups to allow traffic only from the gateway, and consider mTLS between the gateway and services.