In a previous article, we explored the importance of load balancing in IT systems. In this article, we’re going to put that theory into practice by setting up a load-balancing system using Nginx.

Nginx is commonly known as a server, but it’s also incredibly versatile as a load balancer. It offers several load-balancing methods:

1. Round Robin: This is the default method where requests are distributed evenly across all servers, considering each server’s capacity.

2. Least Connections: Requests go to the server with the fewest active connections, again considering server capacity.

3. IP Hash: The client’s IP address determines which server receives the request. This method ensures that requests from the same address go to the same server if it’s available.

4. Generic Hash: The server is selected based on a user-defined key, which could be a text string or a variable.

5. Least Time (NGINX Plus only): NGINX Plus selects the server with the lowest response time and fewest active connections.

6. Random: Requests are sent to a randomly chosen server. If two parameters are specified, NGINX randomly selects two servers considering their capacity, then picks one using the specified method.

For more in-depth information about Nginx load balancing, check out the Nginx documentation.

Our System Architecture

We’ll build a system with the following components:

• 2 Containers running the same Flask API: These will process our requests.

• 1 Container running Nginx as a Load Balancer: This serves as the entry point and distributes incoming requests between the two Flask containers.

The diagram above illustrates how our setup will function. Notice how each time you refresh the page, a different container processes your request. This demonstrates load balancing in action.

version: '3.7'

services:
  esperluet_api_node_1:    
    build:
      context: app
      target: builder
    stop_signal: SIGINT
    ports:
      - '8001:8000'
    container_name: esperluet_api_node_1
    networks:
      - esperluet_network
  esperluet_api_node_2:    
    build:
      context: app
      target: builder
    stop_signal: SIGINT
    ports:
      - '8002:8000'
    container_name: esperluet_api_node_2
    networks:
      - esperluet_network
  esperluet_nginx:
    image: nginx
    container_name: esperluet_nginx
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "8082:80"
    environment:
      - NGINX_HOST=esperluet.local
      - NGINX_PORT=80
    networks:
      - esperluet_network

networks:
  esperluet_network:
    driver: bridge

To explore the code and set up this system yourself, visit our GitHub repository.

Below screenshot of the tests

containers 1

containers 2

Conclusion

Implementing load balancing with Nginx and Flask might sound complex, but it’s quite straightforward once you understand the basics. This setup ensures efficient handling of requests and better resource utilization.