Cloud Parts

Load balancers

Spreads incoming traffic across healthy servers.

4 technologies on this page ↓
Live model
What you’re watchingRequests are handed out in turn across three servers. When server 2 fails its health check, it’s skipped until it recovers.

How it works

A load balancer gives a service one stable address and forwards each request or connection to one of several backends. It runs health checks and stops sending traffic to failed instances, terminates TLS, and lets you add or remove servers without clients noticing. It picks a backend with an algorithm such as round robin, least connections or hashing. Layer 4 balancers forward TCP and UDP connections; layer 7 balancers understand HTTP and can route by host name or path.

Think of it as

The host at a busy restaurant, seating each new party with a waiter who’s free and skipping the section that’s closed.

Reach for it when

  • Running more than one copy of any service
  • Zero-downtime deploys: drain old instances, add new ones
  • Terminating TLS in one place

Think twice when

  • A single instance with no redundancy; it adds cost and little else
  • Cross-region failover on its own; that needs DNS or a global balancer

A real example

Three API servers sit behind an Application Load Balancer. When server 2 fails its /health check twice in a row, the balancer stops routing to it, and users only ever see the other two answer.

More on load balancers

Narrower ideas that build on this one.

Technologies

Specific products, and what each does differently.

What each provider calls it

AWS
Elastic Load Balancing (ALB, NLB)
Google Cloud
Cloud Load Balancing
Azure
Load Balancer, Application Gateway
Open source & others
NGINX, HAProxy, Envoy

See it in a real stack

Architectures that use load balancers

Web app on KubernetesA load balancer spreads requests over app pods on Kubernetes, which check Redis before going to PostgreSQL.AWS Application Load BalancerKubernetesRedisPostgreSQL

In code · nginx

upstream api {
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
    server 10.0.1.13:8080;
}
server {
    listen 443 ssl;
    location / { proxy_pass http://api; }   # round robin by default
}

Often confused with

API gatewaysCDNsAutoscalingDNSService discovery & meshRemote procedure callsLoad-balancing algorithmsLoad balancer, API gateway or CDN?
Where it came from1996–2026

When early websites outgrew one machine, the first trick was round-robin DNS: publish several server addresses under one name and rotate their order. It spread load, but DNS had no idea whether a server was alive. If one failed, the DNS kept handing out its address, and caches meant clients kept trying it.

The fix was a box in front of the servers. Cisco’s LocalDirector was conceived in late 1995, when Netscape’s webmaster described customers whose round-robin DNS left some servers swamped and others idle. It reached its first customers in 1996. Such appliances gave a site one virtual address and mapped each connection to one of the servers behind it. Software caught up: the Linux Virtual Server project started in 1998, Willy Tarreau began writing HAProxy in 2000, and nginx’s first public release followed in 2004.

Large operators then moved load balancing back onto ordinary machines. Google has run Maglev, a load balancer built from commodity Linux servers, since 2008. In 2009 AWS made load balancing a managed service with Elastic Load Balancing, and its layer-7 Application Load Balancer arrived in 2016.

Why it’s built this way

One virtual address in front of many servers
Clients only ever see the balancer’s address, so servers can be added, removed or patched behind it without anyone updating DNS or client configuration.
Health checks decide where traffic goes
Round-robin DNS kept sending users to dead servers. A balancer that probes each backend stops using a failed one after a few missed checks.
Layer 4 for speed, layer 7 for smarts
Forwarding TCP connections without reading them is fast and works for any protocol. Reading HTTP costs more CPU but allows routing by host or path, TLS termination and retries.
Software on commodity servers instead of appliances
Google built Maglev this way because it needs no special rack deployment and capacity grows by adding servers. Cloud load balancers work on the same principle.

Timeline

  1. 1996

    Cisco LocalDirector ships, an early dedicated load-balancing appliance

  2. 1998

    Wensong Zhang starts the Linux Virtual Server project

  3. 2000

    Willy Tarreau starts writing HAProxy

  4. 2008

    Google’s Maglev software load balancer starts serving production traffic

  5. 2009

    AWS launches Elastic Load Balancing as a managed service

  6. 2016

    AWS adds the layer-7 Application Load Balancer; Google publishes the Maglev paper

  7. 2026

    Kubernetes retires the community ingress-nginx controller in March, pointing users to Gateway API

Sources11

Improve this page on GitHub