00:00

Reverse Proxy vs API Gateway vs Load Balancer

Last updated:

Reverse proxy, API gateway and load balancer: three terms that appear in nearly every architecture diagram and are often used loosely. All three sit between clients and servers and forward requests, but each exists for a different reason.

This tutorial brings the three together. It defines each by its main responsibility, shows how they stack in a real microservices platform, and gives a decision table you can use in design reviews and interviews.

1. What Are They?

  • Reverse proxy: fronts servers. It hides them, terminates TLS, caches, compresses and routes by host or path. See Reverse Proxy.
  • Load balancer: spreads traffic across healthy instances of a service using an algorithm and health checks, at Layer 4 or Layer 7. See Load Balancing.
  • API gateway: governs API calls. It authenticates consumers, enforces rate limits and quotas, routes to services and transforms requests.

A useful shorthand: the reverse proxy is about protection and delivery, the load balancer about distribution, and the API gateway about API policy.

2. How the Three Layer Together

Reverse proxy, API gateway and load balancer in one architecture Requests from web and mobile clients pass through four stages. First an edge load balancer at Layer 4 spreads connections. Then a reverse proxy or ingress terminates TLS, caches and compresses. Then an API gateway authenticates, rate limits and routes each API call. Finally the request reaches the orders, users or payments service. CLIENTS EDGE ENTRY API LAYER SERVICES Web Mobile Load BalancerLayer 4 ·connections Reverse ProxyTLS · cache ·compression API Gatewayauth · limits ·API routing Orders Users Payments In small systems one product, such as Nginx or a cloud ALB, may play several of these roles.
Each layer has one main job: spread connections, front the platform, govern API calls, then serve the request.

3. One Request Through All Three

  1. A mobile app calls https://api.shop.example/api/payments.
  2. Load balancer: a cloud Layer 4 balancer spreads the TCP connection to one of several ingress nodes.
  3. Reverse proxy (ingress): terminates TLS, applies basic request limits, and forwards /api/* to the gateway Service. A request for /static/* would be served here and never reach the gateway.
  4. API gateway: validates the JWT, checks the caller's quota, and routes to payments-service.
  5. Load balancing again: the gateway, or the Kubernetes Service, picks one healthy payments pod.
  6. The response returns along the same path.

Every hop should pass on the original client address and a request ID. With X-Forwarded-For, each proxy appends the address it received the request from, so the header becomes a list such as 198.51.100.4, 10.0.0.12. Services should use the entry added by the first trusted proxy, not simply the first value, because a client can send a fake header.

When a layer fails, the request stops there, and the error tells you where. The edge balancer usually causes connection errors or timeouts, the reverse proxy returns 502 or 504 when it cannot reach the next hop, and the gateway returns 401, 429 or 503 for policy and circuit-breaker decisions.

4. Responsibilities Side by Side

ResponsibilityReverse proxyLoad balancerAPI gateway
Hide backend serversYesYesYes
TLS terminationYesLayer 7 onlyOften
Caching and compressionYesRarelySometimes
Algorithms and health checksBasicCoreUsually delegated
Authentication of API consumersNoNoCore
Per-consumer rate limitsNo (per IP at most)NoCore
Request transformation and aggregationLimitedNoYes

Overlap in modern products

Many products now cover more than one role. Envoy works as a reverse proxy and load balancer, and with extensions as an API gateway. The Kubernetes Gateway API describes HTTP, gRPC and TCP routing in one model, and many implementations add gateway features on top. Cloud Layer 7 load balancers terminate TLS, route by path and can validate tokens. When one product does several jobs, still decide and document which job each instance performs, so configuration does not drift.

5. Practical Implementation

A typical Kubernetes setup uses all three: a cloud load balancer created by the ingress controller's Service, an Ingress as the reverse proxy, and Spring Cloud Gateway as the API gateway.

Reverse proxy layer: Kubernetes Ingress

ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop-entry
spec:
  ingressClassName: nginx
  tls:
    - hosts: [api.shop.example]
      secretName: shop-tls          # TLS terminated here
  rules:
    - host: api.shop.example
      http:
        paths:
          - path: /static
            pathType: Prefix
            backend:
              service: { name: static-site, port: { number: 80 } }
          - path: /api
            pathType: Prefix
            backend:
              service: { name: api-gateway, port: { number: 8080 } }

API gateway layer: Spring Cloud Gateway

application.yml
spring:
  cloud:
    gateway:
      server:
        webflux:
          routes:
            - id: orders
              uri: lb://orders-service          # load-balanced across registered instances
              predicates:
                - Path=/api/orders/**
              filters:
                - StripPrefix=1                  # /api/orders/42 -> /orders/42
                - name: RequestRateLimiter
                  args:
                    redis-rate-limiter.replenishRate: 10
                    redis-rate-limiter.burstCapacity: 20
                    key-resolver: "#{@userKeyResolver}"
            - id: users
              uri: lb://users-service
              predicates:
                - Path=/api/users/**
              filters:
                - StripPrefix=1
  security:
    oauth2:
      resourceserver:
        jwt:
          issuer-uri: https://auth.example.com/realms/shop   # validate JWT access tokens
Note: These property names are for Spring Cloud 2025.0 and later, which use the spring-cloud-starter-gateway-server-webflux starter. Older releases use spring.cloud.gateway.routes without server.webflux. The rate limiter also needs the reactive Redis starter, and JWT validation needs the OAuth2 resource server starter.

The load balancing layer needs no extra file: the ingress controller's LoadBalancer Service creates the cloud balancer, and lb:// routes or Kubernetes Services balance across pods.

6. Advantages and Disadvantages of Layering

Advantages

  • Each layer stays simple and focused.
  • Static content and attacks are stopped before they reach the API layer.
  • Each layer scales independently.

Disadvantages

  • Every hop adds latency, often around a millisecond within a region.
  • More components to configure, secure and monitor.
  • Overlapping features can be configured twice and drift apart, such as rate limits in both proxy and gateway.

7. Decision Table

If you need to...Use
Serve one web app with HTTPS and cachingReverse proxy
Run several identical instances for capacity and failoverLoad balancer
Balance non-HTTP traffic such as databases or MQTTLayer 4 load balancer
Expose APIs with tokens, quotas and versioningAPI gateway
Route many microservices behind one domainReverse proxy or ingress, plus an API gateway for APIs
Run a large public platformAll three, layered

8. Real-World Use Cases

  • Small business site: one Nginx instance as a reverse proxy, which also balances two app servers.
  • SaaS product: a cloud Layer 7 balancer with TLS and an API gateway for the public API.
  • Large microservices platform: a Layer 4 edge balancer, ingress reverse proxies, an API gateway cluster, and per-service balancing.
  • Global consumer app: a CDN in front of everything; see Reverse Proxy vs CDN.

Latency budget

Each layer adds a little time. Inside one region, a well-tuned proxy hop typically costs around a millisecond, while token validation and rate-limit lookups in Redis add a few more. For an API that must answer in 200 ms that overhead is small, but it adds up when requests also pass through a CDN, a service mesh and several internal calls. Measure each hop with distributed tracing before adding another layer.

9. Best Practices

  • Assign one owner per concern: decide where TLS ends, where rate limits live and where authentication happens, and document it.
  • Avoid unnecessary layers: start with one component that covers your needs, and add layers when requirements appear.
  • Make every layer redundant and health-checked.
  • Pass client context consistently: keep X-Forwarded-For, request IDs and trace headers through every hop.
  • Secure internal hops: use network policies or mTLS between layers so traffic cannot skip the gateway.

Common mistakes

  • Terminating and re-establishing TLS at every layer without a reason, which adds latency.
  • Configuring rate limits in three places with different values.
  • Health checks that test only the proxy, not the service behind it.
  • Keep every layer's configuration in version control and review changes to routes, limits and TLS settings like code.

10. Summary

  • Reverse proxy: fronts and protects servers (TLS, caching, routing).
  • Load balancer: distributes traffic across healthy instances (algorithms, health checks).
  • API gateway: governs API calls (authentication, rate limits, transformation).
  • One product can play several roles; the distinction is about responsibility.
  • Large systems layer them: edge balancer, then reverse proxy or ingress, then API gateway, then service balancing.

Next, move from networking to data storage with ACID vs BASE.