notes

Appearance

Load Balancers ​

Load balancing is the practice of distributing network traffic or computational workloads accross multiple servers to improve an applications performance and reliability.

Load balancing is handled by a tool or application called load balancer. A load balancer can be either hardware-based or software-based. Hardware load balancers require the installation of a dedicated load balancing device. Software based load balancers can run on a server on a virtual machine or in the cloud. Content delivery networks often include load balancing features.

Load Balancing Algorithms ​

When a request arrives from a client, the load balancer assigns the request to a given server, and this process repeats for each request. Load balancers determine which server should handle each request based on algorithms. These algorithms fall into two categories:

Static load balancing algorithms ​

These distribute workloads without taking into account the current state of the system and assigns workload based on a predetermined plan.

  • Round Robin: Round Robin load balancing distributes traffic to a list of servers in rotation using domain name system. An authoritative nameserver will have a list of different A record for a domain and provides a different ip address to each dns query.
  • Weighted Round Robin: Also done using DNS but each record is assigned a weight depending upon the server that can handle more requests.
  • IP Hash: Combines incoming traffic and destination IP addresses and uses a mathematical function to convert it into a hash. Based on the hash, the connection is assigned to a specific server.

Dynamic load balancing algorithms ​

These take into account the current availability, workfload and health of each server and balance the requests to each server accordingly.

  • Least connection: Checks which servers have the fewest connections open at the time and sends traffic to those servers. This assumes all connections require roughly equal processing power
  • Weighted least connection: Gives administrators the ability to assign different weights to each server, assuming that some servers can handle more connections than others.
  • Weighted response time: Averages the response time takes for a request to get fullfilled and determines which server to redirect the request for fastest response for the user
  • Resource-based: Distributes load based on what resources each server has available at the time. Specialized software (called an "agent") running on each server measures that server's available CPU and memory, and the load balancer queries the agent before distributing traffic to that server

Layer 4 Load Balancer ​

It is the type of load balancers that works on Layer 4 of the OSI model i.e. Transport Layer. It uses IP address and port to make routing decisions. Eg: AWS NLB, HAProxy (TCP), Linux iptables, etc. There are two modes for L4 Loadbalancer.

  • Passthrough mode: The load balancers forwards the packet using nat and doesnt create a new tcp request and terminate the connection. Selection in this is based on connection-level metadata such as source ip, destination ip, port number, protocol or some other metadata. The algorithms in this are: random, hash-based, etc. For eg: Linux Iptables, nftables, etc. Its flow is as follow:
    • The client opens a tcp connection to the load balancer.
    • The load balancer forwards the request to the server.
    • Backend responds with appropiate response which is then forwarded to client. Client thinks they are still talking to the load balancer.
  • Proxy Mode: The load balancer terminatest the connection and starts a new tcp connection with the server. The load balancer is responsive for tcp handshake and then creates a second tcp session to the backend. Since load balancer handles each request it can you more intelligent algorithms such as round robin, least connection, weighted round robin, ip hash, etc. Eg: HAProxy, Envoy, Nginx, etc.

Layer 7 Load Balancer ​

It is the type of load balancer that workds on Layer 7 of the OSI model. i.e the application layer. It inspects the http headers, paths, cookies, etc and makes decision based on the content. For example it can forward the request for a video to the video server and payment request to more secure payment server. Eg: Nginx, Envoy, AWS ALB, HAProxy (HTTP mode).