Load Balancing with Nginx

Load Balancing with Nginx

Load balancing distributes incoming requests across multiple backend servers. It provides horizontal scaling (add more servers to handle more traffic) and high availability (if one server goes down, others continue serving traffic). Nginx's upstream module supports several load balancing algorithms and passive health checks in the open-source version. For active health checks, Nginx Plus (commercial) or a separate solution like HAProxy is needed.

Load balancing algorithms

Basic load balancer configuration

# Three backend app servers, round-robin:
sudo nano /etc/nginx/sites-available/loadbalancer.conf

/etc/nginx/sites-available/loadbalancer.conf

upstream app_servers {
    # Round-robin (default) — each request goes to the next server
    server 192.168.1.11:8080;
    server 192.168.1.12:8080;
    server 192.168.1.13:8080;

    # Keepalive connections to backends (reduces connection overhead):
    keepalive 32;
}

server {
    listen 80;
    server_name app.example.com;

    location / {
        proxy_pass http://app_servers;
        proxy_http_version 1.1;
        proxy_set_header Connection "";   # Required for keepalive
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

# For least connections (better for long-running requests):
# upstream app_servers {
#     least_conn;
#     server 192.168.1.11:8080;
#     server 192.168.1.12:8080;
# }

# Weighted round-robin (if server 1 is more powerful):
# upstream app_servers {
#     server 192.168.1.11:8080 weight=3;
#     server 192.168.1.12:8080 weight=1;
# }

sudo nginx -t && sudo systemctl reload nginx

Health checks and failover

# Nginx open-source uses passive health checks:
# If a server returns errors, it's temporarily removed from rotation
upstream app_servers {
    server 192.168.1.11:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.12:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.13:8080 backup;    # Only used if all others fail
}

# max_fails=3        → mark server down after 3 failed requests
# fail_timeout=30s   → consider server down for 30s, then retry

# Monitor which servers are active:
sudo nginx -s reload
# Check /var/log/nginx/error.log for "no live upstreams" errors

Session persistence

# Problem: if user logs in on server1, next request may go to server2 which
# doesn't have that session. Solutions:

# Option 1: IP hash (same IP always goes to same server):
# upstream app_servers {
#     ip_hash;
#     server 192.168.1.11:8080;
#     server 192.168.1.12:8080;
# }

# Option 2: Store sessions in Redis/database (better — stateless backends):
# Both servers share the same Redis session store
# This is the recommended approach for new applications

# Option 3: Sticky cookie (Nginx Plus feature only)
# open-source Nginx does not support sticky cookies natively

Conclusion

Round-robin is the default and works well for stateless applications. Use least_conn when request processing time varies significantly (some requests take 10x longer than others). Configure max_fails and fail_timeout on every upstream server so Nginx automatically routes around failed backends. The cleanest architecture for session handling is stateless backends with shared session storage (Redis) — it works with any load balancing algorithm and makes adding/removing servers straightforward.