load-balancing

Name: load-balancing
Author: proyecto26

45installs1forks

How to install load-balancing?

proyecto26/system-design-skills/load-balancing

$npx -y skills add proyecto26/system-design-skills --skill load-balancing

Installs into the current project.

Summary

This skill should be used when the user adds a "load balancer", asks about "L4 vs L7" (transport vs application layer), picks a balancing algorithm ("round robin", "least connections", "weighted", "IP hash"), configures "health checks", needs "sticky sessions" / "session affinity

Files · 1

View on GitHub

SKILL.md

1	# Load Balancing
2
3	Spread incoming requests across a pool of identical backends so no single server
4	is the bottleneck or the single point of failure. Get it wrong and the balancer
5	becomes the SPOF it was meant to remove, routes traffic to dead servers, or
6	amplifies an outage by hammering a backend that is already on its knees.
7
8	## When to reach for this
9	Reach for a balancer when more than one backend serves the same role and traffic
10	must be split across them; when a single entry point is a SPOF to eliminate; to
11	add or remove servers without clients noticing (the enabler for the stateless tier
12	and autoscaling); or to put one public address in front of a private fleet, with
13	TLS termination, health-gating, and routing in one place.
14
15	## When NOT to
16	One backend that comfortably handles peak load needs no balancer yet — adding one
17	is a new component, a new failure mode, and a new thing to operate (YAGNI). If the
18	real bottleneck is the database or a single hot shard, a balancer in front of the
19	web tier solves nothing; diagnose the actual constraint first (→ `back-of-the-envelope`).
20	Cross-region traffic steering is usually DNS/anycast at the edge, not an L4/L7
21	balancer (→ `content-delivery`). Per-client request limiting is a policy concern
22	owned by `resilience-failure`, not the balancing algorithm.
23
24	## Clarify first
25	- Protocol & layer need — raw TCP/UDP throughput (L4) or HTTP-aware routing by
26	path/host/header/cookie (L7)? This picks the balancer type.
27	- State — is the backend stateless, or does a session live on one server
28	(forcing affinity)? Moving state out is almost always the better answer.
29	- Peak QPS & connection count — one fat connection stream or many short
30	requests? Drives algorithm and balancer sizing (→ `back-of-the-envelope`).
31	- Health signal — what does "healthy" mean (TCP accept? a `/healthz` 200? a
32	deep dependency check?) and how fast must a dead node leave the pool?
33	- TLS — terminate at the balancer (offload backends, inspect L7) or pass
34	through end-to-end (compliance/mTLS)?
35
36	## The options
37
38	Layer of inspection
39	- L4 (transport): route by IP/port; forward packets via NAT or DSR without
40	reading payload. Use when you need raw throughput, non-HTTP protocols, or the
41	lowest added latency.
42	- L7 (application): terminate the connection, read HTTP (host, path, headers,
43	cookies), then route. Use when you need content-based routing, per-route pools,
44	TLS termination, or request rewriting.
45
46	Distribution algorithm
47	- Round robin / weighted round robin: even rotation, weighted by capacity.
48	Use for uniform, stateless backends.
49	- Least connections / least response time: send to the least-busy node. Use
50	when request cost varies widely (long-lived connections, mixed workloads).
51	- IP hash / consistent hash: map a client (or key) to a stable backend. Use
52	for affinity without server-side session storage, or to keep cache locality.
53	- Random (two-choices): pick two at random, take the lighter — cheap and
54	surprisingly even at scale.
55
56	Topology
57	- Active-passive: one balancer serves, a standby takes the VIP on failure.
58	Simple HA.
59	- Active-active: multiple balancers share load (via DNS or anycast). Removes
60	the balancer's own SPOF and adds headroom.
61
62	Reverse proxy role: an L7 balancer is also a reverse proxy — one public face
63	that hides backends, terminates TLS, compresses, caches, and centralizes routing.
64	A reverse proxy is worth it even with a single backend for those benefits;
65	load balancing is the multi-backend case of the same component.
66
67	## Trade-offs
68
69	\| Option \| What it solves \| What it worsens \| Change it when \|
70	\|---\|---\|---\|---\|
71	\| L4 balancing \| Max throughput, low latency, any protocol \| Blind to content; no path/header routing, no TLS inspection \| You need content-based routing or TLS termination → L7 \|
72	\| L7 balancing \| Content routing, TLS offload, rewrites, observability \| Higher latency/CPU; terminates connections (more state) \| Raw throughput dominates and routing is trivial → L4 \|
73	\| Round robin \| Dead simple, even for uniform work \| Ignores actual load; a slow node still gets its share \| Request costs vary a lot → least-connections \|
74	\| Least connections \| Adapts to uneven request cost \| Needs live connection state; can herd onto a just-recovered node \| Backend

People who liked this also viewed

6 picks

Type

load-balancing

45installs1forks

View on GitHub

How to install load-balancing?

proyecto26/system-design-skills/load-balancing

$npx -y skills add proyecto26/system-design-skills --skill load-balancing

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Load Balancing
2
3	Spread incoming requests across a pool of identical backends so no single server
4	is the bottleneck or the single point of failure. Get it wrong and the balancer
5	becomes the SPOF it was meant to remove, routes traffic to dead servers, or
6	amplifies an outage by hammering a backend that is already on its knees.
7
8	## When to reach for this
9	Reach for a balancer when more than one backend serves the same role and traffic
10	must be split across them; when a single entry point is a SPOF to eliminate; to
11	add or remove servers without clients noticing (the enabler for the stateless tier
12	and autoscaling); or to put one public address in front of a private fleet, with
13	TLS termination, health-gating, and routing in one place.
14
15	## When NOT to
16	One backend that comfortably handles peak load needs no balancer yet — adding one
17	is a new component, a new failure mode, and a new thing to operate (YAGNI). If the
18	real bottleneck is the database or a single hot shard, a balancer in front of the
19	web tier solves nothing; diagnose the actual constraint first (→ `back-of-the-envelope`).
20	Cross-region traffic steering is usually DNS/anycast at the edge, not an L4/L7
21	balancer (→ `content-delivery`). Per-client request limiting is a policy concern
22	owned by `resilience-failure`, not the balancing algorithm.
23
24	## Clarify first
25	- Protocol & layer need — raw TCP/UDP throughput (L4) or HTTP-aware routing by
26	path/host/header/cookie (L7)? This picks the balancer type.
27	- State — is the backend stateless, or does a session live on one server
28	(forcing affinity)? Moving state out is almost always the better answer.
29	- Peak QPS & connection count — one fat connection stream or many short
30	requests? Drives algorithm and balancer sizing (→ `back-of-the-envelope`).
31	- Health signal — what does "healthy" mean (TCP accept? a `/healthz` 200? a
32	deep dependency check?) and how fast must a dead node leave the pool?
33	- TLS — terminate at the balancer (offload backends, inspect L7) or pass
34	through end-to-end (compliance/mTLS)?
35
36	## The options
37
38	Layer of inspection
39	- L4 (transport): route by IP/port; forward packets via NAT or DSR without
40	reading payload. Use when you need raw throughput, non-HTTP protocols, or the
41	lowest added latency.
42	- L7 (application): terminate the connection, read HTTP (host, path, headers,
43	cookies), then route. Use when you need content-based routing, per-route pools,
44	TLS termination, or request rewriting.
45
46	Distribution algorithm
47	- Round robin / weighted round robin: even rotation, weighted by capacity.
48	Use for uniform, stateless backends.
49	- Least connections / least response time: send to the least-busy node. Use
50	when request cost varies widely (long-lived connections, mixed workloads).
51	- IP hash / consistent hash: map a client (or key) to a stable backend. Use
52	for affinity without server-side session storage, or to keep cache locality.
53	- Random (two-choices): pick two at random, take the lighter — cheap and
54	surprisingly even at scale.
55
56	Topology
57	- Active-passive: one balancer serves, a standby takes the VIP on failure.
58	Simple HA.
59	- Active-active: multiple balancers share load (via DNS or anycast). Removes
60	the balancer's own SPOF and adds headroom.
61
62	Reverse proxy role: an L7 balancer is also a reverse proxy — one public face
63	that hides backends, terminates TLS, compresses, caches, and centralizes routing.
64	A reverse proxy is worth it even with a single backend for those benefits;
65	load balancing is the multi-backend case of the same component.
66
67	## Trade-offs
68
69	\| Option \| What it solves \| What it worsens \| Change it when \|
70	\|---\|---\|---\|---\|
71	\| L4 balancing \| Max throughput, low latency, any protocol \| Blind to content; no path/header routing, no TLS inspection \| You need content-based routing or TLS termination → L7 \|
72	\| L7 balancing \| Content routing, TLS offload, rewrites, observability \| Higher latency/CPU; terminates connections (more state) \| Raw throughput dominates and routing is trivial → L4 \|
73	\| Round robin \| Dead simple, even for uniform work \| Ignores actual load; a slow node still gets its share \| Request costs vary a lot → least-connections \|
74	\| Least connections \| Adapts to uneven request cost \| Needs live connection state; can herd onto a just-recovered node \| Backend

People who liked this also viewed

6 picks

Type