Load Balancing: Algorithms and Strategies

In modern applications, thousands of users may use a system at the same time. If every request goes to a single server, that server soon becomes slow, and if it crashes, the whole application goes down. Load balancing solves this problem by spreading incoming requests across several servers, so that no single server is overloaded and users always reach one that is healthy.

This tutorial explains what load balancing is, how a load balancer works, the most common algorithms, Layer 4 and Layer 7 load balancing, health checks, sticky sessions and the main types of load balancer. It also shows practical examples with Nginx and Spring Boot.

1. What Is Load Balancing?

Load balancing is the process of distributing network traffic across multiple servers to improve performance, reliability and availability. The component that does this is called a load balancer. It sits between the users and a group of servers, often called a server pool or backend pool, and decides which server should handle each request.

To the user, the whole pool looks like one application with one address. Behind that address, the load balancer quietly shares the work and sends traffic only to servers that are working correctly.

2. Why Load Balancing Is Important

• Better performance: work is shared, so each server stays fast and response times stay low, even at busy times.
• No overloaded servers: a sudden spike in traffic is spread out instead of crushing one machine.
• High availability: if one server crashes, the load balancer sends traffic to the others, and users may not notice anything.
• Easy scaling: you can add servers when traffic grows and remove them when it drops, without changing the address users connect to. This is called horizontal scaling.
• Zero-downtime updates: servers can be updated one at a time while the others keep serving traffic.
• A single place for common tasks: many load balancers also handle HTTPS, compression and basic security checks.

3. How a Load Balancer Works

Every request goes through the same simple steps:

1. Receive: the user's request arrives at the load balancer's public address.
2. Check health: the load balancer already knows which servers are healthy, because it checks them regularly.
3. Choose: it uses a load balancing algorithm to pick one healthy server.
4. Forward: it sends the request to that server.
5. Return: the server's response travels back to the user, usually through the load balancer.

The choice in step 3 is where the algorithms below come in. Some ignore the current state of the servers (static algorithms), while others look at how busy each server is right now (dynamic algorithms).

4. Common Load Balancing Algorithms

4.1 Round Robin

Round Robin is the simplest algorithm. Requests are sent to the servers one after another in a fixed circular order. With three servers, A, B and C:

• Request 1 → Server A
• Request 2 → Server B
• Request 3 → Server C
• Request 4 → Server A again, and the cycle repeats.

It works well when all servers have the same capacity and requests take roughly the same time. It does not notice if one server is struggling.

4.2 Weighted Round Robin

Weighted Round Robin gives each server a weight based on its capacity. A server with a higher weight receives proportionally more requests.

• Server A, weight 3 (a large machine)
• Server B, weight 1 (a small machine)

Out of every four requests, Server A receives three and Server B receives one. This is useful when your servers are not all the same size, or when you want to send a small share of traffic to a new version for testing.

4.3 Least Connections

Least Connections sends each new request to the server with the fewest active connections at that moment. If Server A has 10 active connections, Server B has 5 and Server C has 8, the next request goes to Server B.

This is a good choice when requests take very different amounts of time, such as a mix of quick page loads and long file uploads, because busy servers naturally receive fewer new requests.

4.4 Least Response Time

Least Response Time combines two measurements: how many connections a server has and how quickly it has been answering. The request goes to the server that is likely to respond fastest. It adapts well to servers that slow down temporarily, but the load balancer has to keep measuring response times.

4.5 IP Hash

IP Hash turns the client's IP address into a number using a hash function and uses that number to pick a server. Because the same IP always produces the same result, a given client keeps reaching the same server, as long as the server list does not change.

It is useful when a server keeps some information about the user in memory, but it can spread traffic unevenly, for example when many users share one IP address behind a company network.

4.6 Consistent Hashing

Simple hashing has a problem: if a server is added or removed, almost every client may be moved to a different server. Consistent hashing places servers and keys on an imaginary ring, so adding or removing one server only moves a small share of the keys. It is widely used for distributed caches and databases, where moving keys is expensive.

4.7 Random and "Power of Two Choices"

The Random algorithm simply picks a server at random, which evens out over many requests. A clever improvement is the power of two choices: pick two servers at random and send the request to the less busy of the two. It is cheap to compute yet spreads load almost as well as checking every server.

5. Comparing the Algorithms

If you are not sure where to start, Round Robin for identical, stateless servers and Least Connections for uneven workloads cover most applications.

6. Layer 4 vs Layer 7 Load Balancing

Load balancers are often described by the layer of the network model they work at:

Many large systems use both: a Layer 4 load balancer at the front for raw speed, passing traffic to Layer 7 load balancers that make the detailed routing decisions.

7. Load Balancing Strategies

7.1 Server-Side Load Balancing

In server-side load balancing, a dedicated load balancer sits between clients and servers. Clients only know the load balancer's address; the load balancer chooses the server. This is the most common setup for public websites and APIs, because clients stay simple and the server list can change freely.

7.2 Client-Side Load Balancing

In client-side load balancing, the calling application gets a list of available server instances, usually from a service registry, and picks one itself. There is no extra network hop through a separate load balancer, which makes it popular for calls between microservices inside a private network.

7.3 DNS-Based Load Balancing

DNS-based load balancing returns different IP addresses for the same domain name, so different users connect to different servers. It is simple and needs no extra hardware, but DNS answers are cached by browsers and internet providers, so traffic can keep going to a failed server until the cache expires.

7.4 Global Server Load Balancing (GSLB)

Global server load balancing spreads traffic across data centres or cloud regions around the world. Users are sent to the nearest healthy region, which reduces delay, and if a whole region fails, its traffic moves to another. It is usually built on smart DNS or on anycast networking, where many locations share the same IP address.

8. Health Checks and Failover

A load balancer must never send users to a broken server, so it keeps checking the health of every server in the pool.

• Active health checks: the load balancer regularly calls a health endpoint, such as /health, every few seconds. If a server fails several checks in a row, it is removed from the pool; when it passes again, it is added back.
• Passive health checks: the load balancer watches real traffic. If requests to a server keep failing or timing out, it stops using that server for a while.
• Connection draining: before a server is removed for maintenance, the load balancer stops sending it new requests but lets current requests finish, so no user is cut off.

A good health endpoint checks that the application can really do its job, for example that it can reach its database, but stays fast and cheap so the checks themselves do not add load.

9. Session Persistence (Sticky Sessions)

Some applications store user data, such as a shopping cart, in the memory of one server. If the next request goes to another server, that data is missing. Sticky sessions fix this by sending all of a user's requests to the same server, usually by setting a cookie or by using IP Hash.

Sticky sessions are easy to switch on, but they have drawbacks: load can become uneven, and if that server fails, its users lose their session. The better long-term solution is to make servers stateless and keep session data in a shared store, such as Redis or a database, so any server can handle any request. Read more in Stateful vs Stateless Architecture.

10. Hardware, Software and Cloud Load Balancers

In container platforms, load balancing is often built in. For example, a Kubernetes Service spreads traffic across the pods behind it, and an Ingress controller adds Layer 7 routing.

11. Example: Load Balancing with Nginx

Nginx is a popular open-source web server that also works as a load balancer. This configuration spreads requests across three application servers using Least Connections, gives the larger server a higher weight, temporarily stops using a server that keeps failing, and keeps a spare server for emergencies:

http {
    upstream shop_backend {
        least_conn;                                      # send to the least busy server

        server 10.0.0.11:8080 weight=2;                  # bigger machine, gets more traffic
        server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
        server 10.0.0.13:8080 max_fails=3 fail_timeout=30s;
        server 10.0.0.20:8080 backup;                    # used only if the others are down
    }

    server {
        listen 80;
        server_name shop.example.com;

        location / {
            proxy_pass http://shop_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}

Without least_conn, Nginx uses Round Robin by default. Other options include ip_hash to keep a client on the same server and hash $request_uri consistent for consistent hashing. The max_fails and fail_timeout settings are a form of passive health checking.

12. Example: Client-Side Load Balancing in Spring Boot

In a Spring Boot microservice, Spring Cloud LoadBalancer provides client-side load balancing. Add the spring-cloud-starter-loadbalancer dependency (or a discovery client such as Eureka, which includes it) and mark an HTTP client builder with @LoadBalanced:

import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestClient;

@Configuration
public class HttpClientConfig {

    @Bean
    @LoadBalanced                       // resolve service names to real instances
    public RestClient.Builder restClientBuilder() {
        return RestClient.builder();
    }
}

import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;

@Component
public class InventoryClient {

    private final RestClient restClient;

    public InventoryClient(RestClient.Builder builder) {
        // "inventory-service" is a service name, not a host:
        // each call goes to one of its running instances
        this.restClient = builder.baseUrl("http://inventory-service").build();
    }

    public int stockFor(String sku) {
        return restClient.get()
                .uri("/api/stock/{sku}", sku)
                .retrieve()
                .body(Integer.class);
    }
}

Each call is sent to a different instance of inventory-service. By default, Spring Cloud LoadBalancer uses Round Robin, and a Random strategy is also available. The list of instances usually comes from a service registry; see Service Discovery for how that works.

13. Avoiding a Single Point of Failure

A load balancer protects your servers, but what protects the load balancer? If you run only one and it fails, the whole application is unreachable, so the load balancer itself must be redundant:

• Active-passive pair: two load balancers share a virtual (floating) IP address. The standby watches the active one and takes over the address within seconds if it fails. Tools such as keepalived implement this.
• Active-active: several load balancers handle traffic at the same time, with DNS or anycast spreading users between them.
• Managed cloud load balancers: these are built from many machines across several data centres, so redundancy is handled for you.

14. A Simple Real-Life Example

Think of a busy supermarket with several checkout counters. If every customer queued at one counter, the line would be very long and the other cashiers would sit idle. Instead, a staff member at the front directs each customer to a free counter:

• The staff member is the load balancer.
• The checkout counters are the servers.
• Sending customers to the counter with the shortest queue is the Least Connections algorithm.
• Closing a counter whose till has broken, and sending its customers elsewhere, is a health check and failover.
• Opening extra counters at busy times is horizontal scaling.

Everyone is served faster, and one broken till does not stop the shop. Load balancing does the same for servers.

15. Best Practices

• Keep servers stateless and store sessions in a shared store, so any server can handle any request.
• Configure meaningful health checks that confirm the application really works, and keep them fast.
• Choose the algorithm to match the workload: Round Robin for similar requests, Least Connections for uneven ones, hashing when data lives on specific servers.
• Run at least two load balancers, or use a managed cloud load balancer, to avoid a single point of failure.
• Use connection draining during deployments so users are never cut off mid-request.
• Terminate HTTPS at the load balancer when it suits your security needs, so certificates are managed in one place.
• Set sensible timeouts so slow servers cannot hold connections forever.
• Monitor request rates, error rates, response times and the health of each server, and alert when the pool is shrinking.
• Combine with auto-scaling so servers are added automatically when traffic rises.

16. Common Mistakes to Avoid

17. Summary

• Load balancing spreads traffic across several servers to improve performance, availability and scalability.
• A load balancer receives each request, picks a healthy server using an algorithm and forwards the request.
• Common algorithms include Round Robin, Weighted Round Robin, Least Connections, Least Response Time, IP Hash, Consistent Hashing and Power of Two Choices.
• Layer 4 load balancers route by IP and port; Layer 7 load balancers route by HTTP content such as paths and headers.
• Strategies include server-side, client-side, DNS-based and global server load balancing.
• Health checks and connection draining keep users away from broken or restarting servers.
• Stateless servers with shared session storage work better than sticky sessions in the long run.
• The load balancer itself must be redundant to avoid a single point of failure.

By choosing the right algorithm and strategy, and pairing load balancing with health checks and auto-scaling, applications can handle heavy traffic smoothly and give every user a fast, reliable experience. Open the FAQ tab for answers to common questions about load balancing.

Frequently Asked Questions (FAQ)