What is Load Balancer?
Updated: 3 days ago
A load balancer is a network or application component that distributes incoming connections or requests across multiple available servers or services. It helps avoid concentrating all traffic on one backend, can improve capacity and resilience, and can direct new requests away from unhealthy servers when suitable health checks are configured. It does not automatically make a slow application fast or guarantee uninterrupted service.
Load balancing can operate at the transport layer, commonly using TCP or UDP information, or at the application layer, where an HTTP-aware proxy can make decisions based on hostnames, paths, headers and other request details. The distinction matters for encryption, routing, observability and performance.
The infographic illustrates how client requests are distributed among backend servers and how an unhealthy backend can be removed from the active pool.
How does a load balancer work?
A client connects to a service address, often a virtual IP address or DNS name. The load balancer accepts or forwards the connection, selects an eligible backend using its configured algorithm, and passes the traffic to that backend. Depending on its mode, it may terminate the client connection and create a separate upstream connection, or forward packets without terminating the application protocol.
The client resolves the service name and connects to the published endpoint.
The load balancer evaluates eligible healthy backends and routing rules.
A backend is selected according to the configured policy.
Traffic is forwarded or proxied to the chosen server.
Responses return through the applicable network path; monitoring influences subsequent selections.
The actual return path depends on architecture. Direct server return, full proxy and network address translation designs do not behave identically.
Layer 4 vs Layer 7 load balancing
Layer 4 load balancing operates mainly using transport information such as source and destination addresses and ports. It is suitable for TCP and UDP services and may have less application awareness. Layer 7 HTTP load balancing can inspect requests after appropriate TLS handling, route by hostname or path and apply application policies.
An HTTP load balancer is commonly a reverse proxy, but the terms are not identical. A reverse proxy can serve a single backend, cache responses or perform security checks without distributing traffic among multiple servers.
Load balancing algorithms
Round robin: distributes successive requests across eligible backends in rotation.
Weighted round robin: sends a larger share to backends assigned higher weights.
Least connections: prefers a backend with fewer active connections, often accounting for weights.
Hash based routing: uses a chosen key, such as client IP or a request attribute, to influence backend selection.
Least response time: uses observed latency and other metrics where the product supports it.
Random or power of two choices: samples backends and selects according to a configured policy.
Algorithms do not guarantee equal CPU utilisation. A request may take milliseconds or minutes, and different servers may have different processing capacities.
Health checks and failover
Health checks can be passive, using real connection failures and response errors, or active, sending dedicated probes to backend endpoints. A backend that fails the configured threshold can be removed from new traffic until recovery criteria are met.
A successful TCP connection is not proof that the whole application works. A useful application readiness check should verify the dependencies needed to serve real requests without making the probe itself unnecessarily expensive. Failover can still cause brief errors or lost in-flight requests.
For example, NGINX open-source HTTP upstreams support passive failure detection, while periodic active upstream health checks are documented as a commercial NGINX Plus feature. Behaviour differs by product and version.
Does a load balancer improve speed?
It can improve response times when a single backend is saturated and multiple backends have spare capacity. It cannot eliminate a shared database bottleneck, slow application code, storage contention, external API delays or the client's broadband latency.
The load balancer itself also consumes processing capacity and can become a bottleneck. Additional proxy hops, TLS termination and inspection may introduce overhead. The goal is better overall service behaviour, not automatically lower latency for every individual request.
Load balancer vs router, firewall and CDN
A router forwards packets according to network routes. A firewall enforces traffic policies. A load balancer chooses among multiple eligible service backends. Some appliances combine these functions, but they solve distinct problems.
A content delivery network can cache content closer to users and route traffic across geographically distributed points of presence. Global traffic steering and DNS load balancing can complement local load balancers, but DNS caching means changes do not necessarily take effect instantly.
Session persistence and sticky sessions
Some applications keep session data on one server. Session persistence can attempt to direct a client back to the same backend using cookies, address hashing or other identifiers. This may simplify legacy applications but creates operational trade-offs when a server fails or traffic patterns are uneven.
A shared session store or stateless authentication architecture often reduces the need for sticky sessions. Client IP hashing can be unreliable as an individual-user identifier when many users share CGNAT or corporate egress addresses.
TLS termination and client IP addresses
An HTTPS load balancer may terminate TLS, inspect HTTP requests and establish another encrypted or unencrypted connection to the backend according to configuration. End-to-end security depends on certificate validation and the upstream transport, not merely the presence of HTTPS at the public endpoint.
Proxies commonly pass the original client address using headers such as X-Forwarded-For or standardised Forwarded, but applications must trust only authorised proxies. Untrusted client-supplied headers should not be treated as verified identity.
High availability and avoiding a single point of failure
A single load balancer can itself become a single point of failure. Production designs may use redundant load balancers, managed regional services, anycast, floating addresses or external failover mechanisms.
Multiple backends are useful only when shared dependencies and failure domains are considered. Two servers relying on one database or power supply do not provide complete fault tolerance.
Load balancers in cloud and home labs
Cloud providers offer managed load balancing products with differing transport, HTTP, regional and health-check capabilities. Self-managed deployments commonly use NGINX, HAProxy, Envoy or dedicated network appliances.
In a home lab, a reverse proxy can route services by hostname and optionally distribute requests across multiple instances. Internal-only services may not need public internet exposure. When remote access is required, prefer secure VPN or appropriately secured ingress designs.
What UK broadband changes and what it does not
A load balancer hosted in a UK data centre does not automatically improve the physical FTTP, FTTC, cable or mobile connection into a user's premises. The end-to-end experience still depends on access-network latency, packet loss, local WiFi or Ethernet, router capacity, the ISP path and the remote service.
Hosting multiple application servers behind a load balancer can improve the service's ability to handle concurrent users. It does not combine several ordinary home broadband connections into one faster connection unless a separate multi-WAN or bonding design explicitly supports that behaviour.
Techie Broadband calculation: requests per second
Consider three identical application servers, each capable of sustainably processing 400 requests per second for a particular workload. Their theoretical combined backend capacity is 3 × 400 = 1,200 requests per second, assuming requests distribute effectively and no shared dependency becomes the bottleneck.
At 900 requests per second, an even distribution would place approximately 300 requests per second on each backend, or 75% of its stated capacity. If one backend fails, the remaining theoretical capacity is 800 requests per second. The same 900 request-per-second demand would exceed that capacity by 100 requests per second, or 12.5%.
This is a theoretical model, not a benchmark. Real throughput depends on request cost, connection reuse, caching, database limits, network bandwidth, TLS processing and the load balancer's own capacity.
What happens when one backend fails?
Suppose a pool has three healthy servers and one becomes unavailable. After detection, the load balancer may stop assigning new requests to that server and send eligible new requests to the remaining two. Existing connections or in-flight requests can still fail, and detection is not instantaneous.
Capacity planning should therefore account for the loss of one backend or a larger failure domain. If the remaining capacity cannot support peak demand, failover alone does not prevent overload.
Common load balancing problems
502 or 503 responses: the proxy cannot reach a usable backend or has no eligible targets.
504 timeouts: an upstream response exceeded a configured timeout, possibly due to application or network delays.
Uneven traffic: weights, persistence, connection duration or health status affect distribution.
Redirect loops: proxy headers or TLS termination assumptions are wrong.
Missing client IP: forwarded headers or proxy protocol are not configured consistently.
Healthy probe but broken application: the probe does not cover the failing dependency.
Slow requests: a shared database, backend saturation or network path may dominate.
How to troubleshoot a load balancer
Check the client error and timestamp, then identify the affected hostname or service.
Inspect load balancer logs, upstream status and health-check results.
Verify backend reachability, DNS resolution, ports and firewall policies.
Test each backend directly where safe, bypassing the load balancer.
Compare request counts, connection counts, response times and error rates per backend.
Check TLS termination, certificate trust, proxy headers and session persistence.
Review recent configuration or deployment changes and resource saturation.
Repeat tests during realistic traffic while monitoring the complete request path.
Load balancer FAQs
Is a load balancer the same as a reverse proxy?
No. A reverse proxy receives client requests and forwards them to one or more upstream services. A load balancer distributes connections or requests among multiple eligible backends. Many HTTP load balancers are implemented as reverse proxies, but a reverse proxy can also serve just one backend or provide caching and authentication without balancing traffic.
Does a load balancer make broadband faster?
No. It does not increase the physical download or upload rate of your broadband connection. It may improve the responsiveness of a remote website when the site's servers are overloaded and there is spare capacity elsewhere. Your access network, WiFi, router, ISP path and the application's own dependencies can still limit performance.
What happens if the load balancer itself fails?
If there is only one load balancer and no alternate route, the service may become unreachable even if all application servers remain healthy. Production systems commonly use redundant instances, managed services or failover mechanisms. Those designs also need testing because shared DNS, networking or control-plane dependencies can still cause outages.
What is the difference between Layer 4 and Layer 7 load balancing?
Layer 4 generally uses transport-level connection information such as TCP or UDP addresses and ports to select backends. Layer 7 understands an application protocol such as HTTP and can route by path, hostname, header or cookie when the traffic is available for inspection. The appropriate layer depends on the service, security requirements, protocol and performance constraints.
Do load balancers automatically detect failed servers?
Some do, when appropriate health checking is enabled and configured. Detection can rely on active probes, passive failures or both, and the available features depend on the product. A poorly designed health check may report a server as healthy when its application is broken, or remove a healthy server unnecessarily. Detection and failover also take time.
Can a load balancer handle millions of users?
Potentially, as part of a suitably engineered distributed system, but there is no universal user limit. Capacity depends on request rate, payload sizes, connection counts, TLS work, hardware or managed-service quotas, application servers and shared dependencies. The number of registered users alone does not indicate how much load a service must handle.
Key takeaway
A load balancer distributes traffic among eligible backends and can improve scalability and resilience when correctly configured. Its effectiveness depends on algorithms, health checks, session handling, redundancy and the capacity of the complete application path.
To explore the related technical concepts used in this Load Balancer guide, visit our broadband terminology guide.
