Load Balancing

Purpose: spread incoming requests across many servers so no single server is overwhelmed. It's the front door of horizontal scaling — without it, adding servers doesn't help because everything still hits one box.

How it works

A reverse proxy (nginx, HAProxy, cloud LB, or a service-mesh sidecar) sits in front of your servers. It accepts traffic, picks a healthy backend, forwards the request, and returns the response. This is called L4 (TCP) or L7 (HTTP) load balancing depending on how much of the request it inspects.

Balancing algorithms

Algorithm	How it picks	Use when
Round-robin	Next server in turn	Uniform servers, uniform work
Weighted	Round-robin by capacity	Unevenly sized servers
Least-connections	Fewest active connections	Requests have variable cost
Least-response-time	Fastest recent responder	Latency-sensitive
IP/consistent hashing	Hash of client IP → fixed server	Sticky sessions / cache affinity

The catches a senior knows

Health checks are mandatory. An LB that forwards to a dead server is an outage multiplier. Health checks + connection draining (stop new traffic, let in-flight finish) make deploys safe.
Hashing gives affinity, not balance. Hash-based routing fixes a client to one server — great for sticky sessions and cache locality — but a hot client (or shard) skews load, the same hot-key problem as partitions.
The LB is a single point of failure unless it's a redundant pair. That's why your design must survive "the load balancer itself died."

Where it fits your systems

In the notification platform, the LB fronts the notification API and the workers. In the transaction-alert flow, the async queue is the load leveler — the queue lets you load-balance work by pulling instead of by pushing, which is more robust than a strict round-robin against a synchronous service.

Ask yourself

Is my load balancer itself a SPOF?
Do I need sticky sessions — and if so, what's my consistent-hash key (and what's my hot-key risk)?
Push (LB) or pull (queue) — which matches the latency and burst profile?