Case Study: Building a Scalable E-Commerce Application using Docker Compose

Background: An e-commerce company is looking to deploy a scalable, reliable, and isolated development environment for their web application. The application includes a frontend, a backend, a database, a caching layer, and a queue for handling background tasks like email notifications or order processing. The goal is to ensure that each component of the application can scale independently, interact with one another, and be easily configured across multiple environments (development, staging, production).

The solution is to use Docker Compose to orchestrate the multiple services required for the e-commerce application. Docker Compose will help streamline the development, testing, and deployment processes by ensuring a consistent environment across different stages of the software lifecycle.

Step 1: Application Overview

The application consists of the following components:

• Frontend: A React-based application served by an Nginx container.
• Backend: A REST API written in Node.js that handles business logic and communicates with other services.
• Database: A MySQL database to store user data, product information, and orders.
• Caching: Redis is used for caching frequently accessed data (e.g., product details).
• Queue: RabbitMQ is used to handle background tasks like sending order confirmations or notifications.
• Monitoring: A logging and monitoring service (e.g., Prometheus, Grafana) to track performance and issues.
• The goal is to use Docker Compose to define these services, handle networking, manage dependencies, and configure them for scalability.

Step 2: Define the docker-compose.yml File

A docker-compose.yml file is created to define all the services, volumes, and networks required for the application. The services are designed to be isolated but can communicate with one another via Docker’s default networking.

Here’s an example of how the docker-compose.yml file could be structured:

version: ‘3.8’

services:
frontend:
image: react-app:latest
build:
context: ./frontend
ports:
– “80:80”
depends_on:
– backend
networks:
– app-network

backend:
image: node-api:latest
build:
context: ./backend
environment:
– DB_HOST=db
– DB_USER=root
– DB_PASSWORD=secret
– REDIS_HOST=redis
– RABBITMQ_HOST=rabbitmq
ports:
– “5000:5000”
depends_on:
– db
– redis
– rabbitmq
networks:
– app-network

db:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: ecommerce
volumes:
– db_data:/var/lib/mysql
networks:
– app-network

redis:
image: redis:alpine
networks:
– app-network

rabbitmq:
image: rabbitmq:management
networks:
– app-network

monitoring:
image: prom/prometheus
container_name: prometheus
ports:
– “9090:9090”
volumes:
– ./prometheus.yml:/etc/prometheus/prometheus.yml
networks:
– app-network

networks:
app-network:
driver: bridge

volumes:
db_data:

Step 3: Key Components of the docker-compose.yml

Frontend:
The frontend service is a React application that serves the UI.
It depends on the backend, meaning the backend must be started before the frontend.
It exposes port 80 to the host machine.

Backend:
The backend is a Node.js application that handles API requests.
It connects to the MySQL database (db), Redis cache (redis), and RabbitMQ message broker (rabbitmq).
It exposes port 5000 for external API access.
Database (MySQL):

MySQL is used to store the e-commerce data, including products, orders, and customer information.
It is configured with a root password and a predefined database (ecommerce).
Data persistence is handled through Docker volumes to ensure data is not lost on container restarts.

Redis:

Redis is used for caching, improving the performance of the backend by caching frequently accessed data.
It is connected to the backend service to store and retrieve cached data.

RabbitMQ:

RabbitMQ handles background tasks such as sending order confirmations or processing background jobs (e.g., email notifications).
It is connected to the backend, allowing the backend to queue tasks.

Monitoring:
Prometheus is used for monitoring the application.
It collects and stores metrics, which are visualized using Grafana (not defined here, but could be added).
Prometheus is connected to the application network to access metrics exposed by backend services.
Networking:

All services are connected to the same custom network (app-network), ensuring they can communicate securely and easily within the Docker environment.
Volumes:

The db_data volume is used to persist MySQL data, ensuring that even if the database container is removed or restarted, the data remains intact.

Step 4: Handling Service Dependencies and Scaling

One of the main advantages of Docker Compose is handling service dependencies. For example, the backend relies on the database, Redis, and RabbitMQ services. The depends_on keyword ensures that these services are started before the backend, although additional logic might be required to ensure that the database is fully initialized and accepting connections before the backend starts.

Scaling the Backend Service:
As the application needs to handle more traffic, we can scale the backend service to run multiple instances using the –scale option.

docker-compose up --scale backend=3

This command will start three instances of the backend service. For load balancing between the multiple backend instances, a reverse proxy like Nginx could be used, or Docker’s built-in load balancing could manage traffic across the backend containers.

Example of Adding Nginx for Load Balancing:

If you want to add a reverse proxy for load balancing, you could define an Nginx service in the docker-compose.yml:

  nginx:
    image: nginx:latest
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    ports:
      - "8080:80"
    depends_on:
      - backend
    networks:
      - app-network

In the nginx.conf file, you would configure load balancing to route traffic to the multiple backend instances.

Step 5: Development, Staging, and Production Environments

Docker Compose simplifies the transition from development to staging and production environments:

• Development: Developers can run docker-compose up locally, with minimal configuration. The docker-compose.override.yml file can be used to customize settings (like debug mode) for local development.
• Staging and Production: For staging and production environments, you can configure a different docker-compose.yml file, with optimized settings (e.g., production-ready databases, optimized image builds, environment-specific configurations).
• You can also use Docker Compose in a CI/CD pipeline (e.g., GitLab CI, Jenkins) to automatically build and deploy the application to different environments.

Step 6: Conclusion

• Using Docker Compose, the e-commerce application can be easily configured, deployed, and scaled. It ensures that all services can interact in a defined network, and each component is isolated in its container. With the flexibility to scale services independently and the ability to integrate caching, messaging, and monitoring, Docker Compose provides a powerful tool for managing complex applications.
• By leveraging Docker Compose in the development pipeline, the application can achieve consistency across different environments, streamline the development process, and simplify deployment. Docker Compose makes it easy to manage service dependencies, scale components as needed, and ensure the system remains reliable and performant.