In the previous article we learned how to create a Rails 6 Docker image and run it locally. While useful for demonstration purposes, it is not enough for a real-world web application. In addition to our main application code, there are other services that we also need, like a database, a queue system, storage and so on.
Docker Compose provides a way for us to describe how our entire application works using declarative code. Declarative code means that we specify the final state of our application, rather than specifying what steps are needed to create our application. This results in a simpler, more intuitive configuration.
docker-compose.yml
Just like we used a Dockerfile to instruct Docker on how to create our custom image, we use a docker-compose.yml file when using Compose.
First, let’s take a look at a sample Compose file for a Rails 6 application:
version: '3'
services:
app:
build: .
ports:
- 3000:3000
volumes:
- .:/usr/src/app
- gem_cache:/bundler_gems
env_file:
- .env/development
environment:
- WEBPACKER_DEV_SERVER_HOST=webpack_dev_server
db:
image: postgres
env_file:
- .env/development
volumes:
- db_data:/var/lib/postgresql/data
webpack_dev_server:
build: .
command: ./bin/webpack-dev-server
ports:
- 3035:3035
volumes:
- .:/usr/src/app
- gem_cache:/bundler_gems
env_file:
- .env/development
environment:
- WEBPACKER_DEV_SERVER_HOST=0.0.0.0
redis:
image: redis
volumes:
db_data:
gem_cache:
Next, we will break it down into their specific components and explain how it works in more detail. We will only discuss configuration that is used in our example. If you are interested in how the other configurations work, the official documentation for the Compose file is found here.
Compose Version
version: '3'
This indicates the version of the supported file format when using Compose. Each version also corresponds to a supported Docker Engine version, which means that newer versions may include features and syntax that is not available for older versions. You will need to use the latest version as much as possible to take advantage of improvements in the Docker ecosystem.
Services
Compose allows you to define services, which are the components of your application as a whole. Your Rails application can be one service, your database is another service, and so on.
In our example, you can see four services: app, db, redis, and webpack_dev_server. You can name your service anything you want as long as you can easily identify it in your Compose file. The name of your service has significance however, due to the fact that Compose’s internal DNS can resolve any service just by using its name. For instance, we can use the service name “db” instead of an IP address when connecting the database via the database.yml file.
Application Service
app:
build: .
ports:
- 3000:3000
volumes:
- .:/usr/src/app
- gem_cache:/bundler_gems
env_file:
- .env/development
environment:
- WEBPACKER_DEV_SERVER_HOST=webpack_dev_server
This service is our main Rails application. This hosts our application code and runs the application server (like Puma).
build
The build configuration indicates where our image will be built from. Similar to our Docker build command:
docker build .
This instructs Compose to build the app image using the Dockerfile in the current directory.
ports
This connects the port 3000 of the app container to the host port 3000 so we can access the container using our browser in the host machine. This is similar to what we used before to run our Rails application:
docker run --rm -p 3000:3000 myapp:1.0
volumes
Mounts a Docker volume into a path in the container. The syntax is familiar, with the local path in the left, and the container path in the right separated by a colon. However, you may notice that in the example, one local path is actually a name and not a path:
volumes:
- .:/usr/src/app
- gem_cache:/bundler_gems
Where did “gem_cache” come from? This is the volume name, and it enables us to reuse the same volume in multiple services. In our case, this will allow us to reuse the bundler gem cache in both our app service and our development webpack service. This makes sense as both services use the same gems and application code in order to work.
Volume names are declared under the volumes definition in our Compose file which we will discuss later.
environment
environment:
- WEBPACKER_DEV_SERVER_HOST=webpack_dev_server
When using Docker, we make use of environment variables extensively. Using this configuration, we can set individual environment variables that can be used by the service. If you have multiple variables or need to share them with several services, it is better to use env_file instead.
env_file
env_file:
- .env/development
This configuration enables us to specify a file to set our environment variables instead of adding them one by one. We can also use the same file in multiple services that need the same environment variables.
Database Service
db:
image: postgres
env_file:
- .env/development
volumes:
- db_data:/var/lib/postgresql/data
Now we go to our database service. In our example, we are using PostgreSQL, but the concept is the same even if you are using another database such as MySQL or SQLite. We have already covered the env_file and volumes sections, so we will skip those.
image
image: postgres
The image configuration tells Docker where to pull the image to run the containers from. In our example, this will pull the latest postgres image from DockerHub (by default) if you do not have the image available in your local machine.
It is also possible to specify a particular image tag to use, or even your own custom image using your account in the image repository:
image: ubuntu:18.04
image: marvs/myapp:1.1
You can specify both an image and a build configuration in the same service. Docker will try to pull the image first from the image repository. If the image is not found, then it will try to build the image itself.
Utility Service
redis:
image: redis
One of the advantages to using Compose (and containers in general) is the ease of adding additional services in our application. In our example, we add a Redis service that we can use for background processing for instance.
Here we just specify the image name redis, and as it is an official image, it will be downloaded from DockerHub automatically. We will discuss how we can connect this “redis” service to our application in a later section.
Development Webpack Service
webpack_dev_server:
build: .
command: ./bin/webpack-dev-server
ports:
- 3035:3035
volumes:
- .:/usr/src/app
- gem_cache:/bundler_gems
env_file:
- .env/development
environment:
- WEBPACKER_DEV_SERVER_HOST=0.0.0.0
If you are developing an application with a Javascript frontend, Webpack is almost a given nowadays. In development, we use Webpack to compile our Javascript files before serving it into the browser in real-time. In production, these compiled files are generated and stored first (such as in a CDN) before they are served to end users.
As we continuously change our application code during development, we will need a local Webpack server to automatically compile our Javascript code. This is the purpose of this service. In a production environment, this service is no longer needed as we just serve static, pre-compiled files.
command
command: ./bin/webpack-dev-server
You may notice that we are using the build configuration in this service, meaning the container for our webpack server is using the same image as our Rails application. This is because we will be using the webpack-dev-server executable provided by Rails to start Webpack.
The command configuration overrides the default command of an image. Since our application image’s default command is to start the Puma server, we will need to replace it with the webpack-dev-server command instead.
Volumes
volumes:
db_data:
gem_cache:
As discussed earlier, we use volumes to store persistent data in our application. These volumes can be mounted into one or many services as necessary. Volumes are treated separately from containers and so they are not destroyed when you destroy a container.
In our example, we have two defined volumes: db_data and gem_cache. The db_data volume is mounted to our PostgreSQL service, meaning this is where the actual database data gets stored. The gem_cache volume is used to store the gems installed by Bundler. This allows the app, webpack, and other services that use the application code to reuse the installed gems, eliminating the slow and redundant bundle installs when running these services.
Environment Variables
Using the env_file configuration, we can create a file that contains all of the environment variables that are needed in our services.
As an example, here is an environment variable file that is saved in .env/development. The values here are used by both the db and the app services.
# DB
POSTGRES_USER=postgres
POSTGRES_PASSWORD=root
POSTGRES_DB=myapp_development
#APP
DATABASE_HOST=db
You may notice that in the app service, it refers to the database host by “db” instead of an IP address or a URL. How does this work?
How services communicate to each other
When you run Docker Compose, it also sets up a network for your application. All services connect to this network and so they can communicate with other services and also be discovered by these services.
This network also includes an internal DNS, meaning services can find others just by referring to it by its name. So if the app service wants to connect to the database, it just calls it using the name db.
#APP
DATABASE_HOST=db
Your application connects to the database service using its IP address that was resolved by the db name. The same goes for the redis service. To connect your application to the redis container, just refer to it using its name.
Redis.new(host: 'redis', port: 6379)
You can use any name you want when you define services in your docker-compose.yml file. Just make sure that they are descriptive and unique so it is easy to refer to them and to troubleshoot issues.
Basic Compose Commands
Compose has many useful commands that you can use during development. Here we will discuss some of the more common ones that you may use frequently.
docker-compose up
This command is the Swiss army knife of Compose. When you run this command, Docker will parse your docker-compose.yml file and do the following automatically:
- Builds all images that are specified in your services. If they are using a custom Dockerfile, the images are built. If they come from an image repository, they are pulled. As always, if the image already exists in your system, then the existing images are used instead.
- Creates the containers for all your services. Compose will also try to reuse the containers you have created before, and will automatically detect if any containers need re-creating.
- Creates a network for your application in order for your services to communicate with each other. Networks are also reused as much as possible.
- Starts all containers for your services, so there is no need to run a separate command to start your application.
As you can see, the up command basically does everything you need to start your application. In a typical scenario, if someone else needs to set up the application on their own machine from scratch, then they just need to do the following steps:
- Install Docker and Compose
- Clone your repository
- Create the environment variable files
- Run docker-compose up
And that’s it! There is no need for the other person to install anything else or run any scripts apart from the Compose commands.
docker-compose down
If there’s an up command, then we can expect to have a down command as well. What this command does is to clean up the containers/services used in your application.
- Stops all running containers that are specified in your services
- Destroys all containers in your application defined in the Compose file
- Removes networks created by Compose for your application
The images that were downloaded and the volumes that were created are not destroyed by this command. This means that even if we ran the down command, our application will be quickly restored the next time we run the up command.
Compose uses a standard way of defining container names based on the application and service name. Given this, you can be sure that the new container names are going to be the same as the previous ones. This is useful when you have aliases for common commands, such as:
docker attach myapp_app_1
This attaches input/output/error streams from a container into a terminal. When using debuggers in Rails, this is used so that you can interact with the debug console. If you have an application name of “myapp” and a service name of “app”, then the created container will be “myapp_app_1”. The 1 at the end indicates the instance count of the container, as you can spin up as many app services as needed.
docker-compose exec
The exec command allows you to run arbitrary commands to a specific container. This is very similar to how docker exec works, but with a few major improvements:
- You can use the service name instead of specifying the container ID or name. This makes it easy to remember the command, as you can just do docker-compose exec app {command}. This then runs the command on the app service.
- Compose exec commands are already interactive by default, meaning you no longer need to add the -i and -t flags like what we did in the base docker exec command. This means that if you want to access the terminal of the app container, you just need to run docker-compose exec app sh.
docker-compose stop
If we wanted to run our application in Compose, we use the up command. By default, this will run within the current terminal and you can stop it by sending the SIGINT command to the terminal, which is usually some form of CTRL+c.
You can also choose to run Compose in the background by running it in detached mode:
docker-compose up -d
If you do this, you will no longer be able to stop it by doing CTRL+c as it won’t reach the process. Instead, you do it by issuing the stop command:
docker-compose stop
This stops all running containers. However, compared to the down command, it doesn’t destroy the containers but instead keeps them in the stopped state so you can run the same containers again when you need them.
Compose is a Great Tool
If you are as old as I am, then you may also remember the time when setting up a development environment can take you at least a whole day. Carefully reading the set-up documentation, making sure that your local dependencies are properly installed, configuring the application, and so on could take a lot of time.
When using containers, especially when using a tool like Docker Compose, this gets much easier. In one single command (docker-compose up), you can have your entire development environment set up in just a few minutes.
This does not mean it is all goodness and rainbows though. As you are using several containers to run your services, this means that it can also take a toll on your computing resources. Images take up disk space, and running several containers at once can consume a lot of memory. However, in my opinion, the advantages to using Compose greatly outweigh the disadvantages. Repeatability of development environments and the ease of adding and connecting services make it worthwhile.
Compose is not limited to development use however. You can also run it in a production environment using Docker Swarm. If your application is simple enough, you can choose to use Compose+Swarm instead of a full-fledged orchestration tool like Kubernetes.
All in all, Compose is a really great tool, and a great fit for development using the Ruby on Rails framework. I highly recommend you to use it at work or in your side projects as it can optimize not just your development flow but also potentially your deployments as well.