Category Archives: Docker

A Docker “hello world” app

Leave a reply

What we’re going to do

In this tutorial we’re going to illustrate what I consider to be Docker’s central and most magical ability: to let you run software on your computer that you don’t actually have installed on your computer.

The specific software we’re going to run is the Lisp REPL (read-evaluate-print loop). The reason I chose Lisp is because you’re unlikely to happen to have Lisp already installed. If I had chosen to use a language that you might already have installed on your computer, like Ruby or Python, the illustration would lose much of its sizzle.

Here are the steps we’re going to carry out. Don’t worry if you don’t understand each step right now because we’re going to be looking at each step in detail as we go.

1. Add a Dockerfile
2. Build an image from our Dockerfile
3. Run our image, which will create a container
4. Shell into that container
5. Start a Lisp REPL inside the container
6. Run a Lisp “hello world” inside the REPL
7. Exit the container
8. Delete the image

Prerequisites

Before you follow this tutorial you’ll of course have to have Docker installed. You might also like to get familiar with basic Docker concepts and terminology.

Adding a Dockerfile

A Dockerfileis a specification for building a Docker image. We’re going to write a Dockerfile, use the Dockerfileto build a custom image, create a container using the image, and finally shell into the container and start the Lisp REPL.

First I’m going to show you the Dockerfileand then I’ll explain each individual part of it.

# Dockerfile

FROM ubuntu:20.04

RUN apt update && apt install -y sbcl

WORKDIR /usr/src

FROM

FROM ubuntu:20.04

The FROMdirective tells Docker what image we want to use as our base image. A base image, as the name implies, is the image that we want to use as a starting point for our custom image. In this case our base image is just providing us with an operating system to work with.

The FROMdirective takes the form of <image>:<tag>. In this case our image is ubuntuand our tag is 20.04. When Docker sees ubuntu:20.04, it will look on Docker Hub for an image called ubuntuthat’s tagged with 20.04.

RUN

RUN apt update && apt install -y sbcl

The RUNcommand in a Dockerfilesimply takes whatever it’s given and runs it on the command line.

In this case the command we’re running is apt update && apt install -y sbcl. The &&in between the two commands means “execute the first command, then execute the second command if and only if the first command was successful”. Let’s deal with each of these commands individually.

The apt updatecommand is a command that downloads package information. If we were to skip the apt updatecommand, we would get an error that says Unable to locate package sbclwhen we try to install sbcl. So in other words, running apt updatemakes our package manager aware of what packages are available to be installed.

The apt install -y sbclcommand installs a package called sbcl. SBCL stands for Steel Bank Common Lisp which is a Common Lisp compiler. (Common Lisp itself is a popular dialect of the Lisp language.)

The -ypart of apt install -y sbclmeans “don’t give me a yes/no prompt”. If we were to leave off the -ywe’d get an “are you sure?” prompt which would be no good because the Dockerfileisn’t executed in an interactive way that would actually allow us to respond to the prompt.

WORKDIR

WORKDIR /usr/src

The WORKDIR /usr/srcdirective specifies which directory to use as the working directory inside the container. Imagine being logged into a Linux machine and running cd /usr/src. After running that command, you’re “in” /usr/srcand /usr/srcis your working directory. Similar idea here.

Listing our existing images

Before we use our Dockerfileto build our image, let’s list our existing Docker images. If this is your first time doing anything with Docker then the list of existing images will of course be empty.

In any case, let’s run the docker image lscommand:

$ docker image ls

Listing our existing containers

In addition to the docker image lscommand which lets us list any images we have, there’s an analogous command that lets us list our containers.

$ docker container ls

Building our image

We can use the docker buildcommand to build our image. The --tag lisppart says “give the resulting image a tag of lisp“. The .part says “when you look for the Dockerfileto build the image with, look in the current directory”.

$ docker build --tag lisp .

After you run this command you’ll see some entertaining output fly across the screen while Docker is building your image for you.

Confirming that our image was created

Assuming that the build was successful, we can now use the docker image lscommand once again to list all of our existing images, which should now include the image we just built.

$ docker image ls

You should see something like the following:

REPOSITORY   TAG      IMAGE ID       CREATED          SIZE  
lisp         latest   91f4fa2a754a   11 minutes ago   140MB

Creating and shelling into a container

Run the following command, which will place you on the command line inside a container based on your image. You should of course replace <image id>with the image id that you see when you run docker image ls.

$ docker run --interactive --tty <image id> /bin/bash

The docker runcommand is what creates a container from the image. The /bin/bashargument says “the thing you should run on this container is the /bin/bashprogram”.

Viewing our new container

Now that we’ve invoked the docker runcommand, we have a new container. Open a separate terminal window/tab and run docker container ls.

$ docker container ls

You should see a new container there with an image ID that matches the ID of the image you saw when you ran docker image ls.

CONTAINER ID   IMAGE          COMMAND       CREATED         STATUS
5cee4af0cfa9   91f4fa2a754a   "/bin/bash"   4 seconds ago   Up 3 seconds

Poking around in our container

Just for fun, run a few commands in the container and see what’s what.

$ pwd    # show the current directory
$ ls -la # show the contents of the current directory
$ whoami # show the current user

Running the Lisp REPL

Finally, let’s run the Lisp REPL by running the sbclcommand inside our container.

$ sbcl

Once you’re inside the REPL, run this piece of “hello, world!” Lisp code.

(format t "hello, world!")

Exiting the container

Press CTRL+D to exit the Lisp REPL and then CTRL+D again to exit the container. The container will delete itself once exited.

Deleting the image

Run the following command to delete the image.
The rmipart stands for “remove image” and the -fflag stands for “force”.

$ docker rmi -f <image id>

Recap

You’ve completed this exercise. You’re now capable of the following things:

  • Writing a Dockerfilethat specifies an operating system to use and some software to install
  • Listing Docker images and containers
  • Building Docker images
  • Shelling into a container
  • Using the software that was installed on the container

If you possess these capabilities and have at least a little bit of a grasp of the underlying concepts, then you’re well on your way to being able to use Docker to accomplish real work.

Docker concepts and terminology

5 Replies

Host machine

The host machine is the machine that Docker is running on. When Dockerizing for development, the host machine is typically the laptop that you’re working on. In a production environment, the host machine could be (for example) an AWS EC2 instance.

Images and containers

Images and containers are sufficiently interrelated that it makes sense to explain them together.

A Docker image is like a blueprint. A container is like the house that you build with that blueprint. You can build any number of houses off of the same blueprint. If you want to, you can even build a house today, demolish it tomorrow, change the blueprint, and build a new version of the house the next day.

Let’s take that analogy and make it concrete. A Docker image (which again is the blueprint) might specify something like “use the Ubuntu operating system, install PostgreSQL, and get PostreSQL running.”Using that image, you can then create a container which, according to the image, will run on Ubuntu, have PostgreSQL installed and have PostgreSQL running.

A container is like a computer inside your computer. If you have a container running, you can do things like shell into it or use a web browser to visit a website hosted on the container, for example.

Dockerfile

A Dockerfile is a specification for building an image.

For example, here’s a Dockerfile that says “Use Ubuntu as the operating system, and install PHP”.

FROM ubuntu:20.04

RUN apt update && apt install -y php

WORKDIR /usr/src

Volume

One of the defining characteristics of a Docker container is that it’s ephemeral. Containers can be started, stopped, created and deleted, and they often are. This creates a problem if you want to save any files or make permanent changes to existing files. Once your container gets deleted, anything you’ve done on the container’s filesystem is gone.

The solution to this problem is volumes. The idea is that instead of storing things on the container’s filesystem, you store things somewhere on the host machine’s filesystem.

To use volumes, you can tell Docker that a certain service should store its files in a certain location on the host machine. For example, you can say, “Hey Docker, whenever you need to store PostgreSQL data, store it in /var/lib/postgresql/data on the host machine”.

This way your data can survive containers being stopped or deleted.

Docker Compose

Docker Compose is a tool for composing an environment from one or more services.

For example, if my development environment needs PostgreSQL, Redis and Elasticsearch, I could tell Docker via a Docker Compose config file that it should install all three of these services and get them running for me. I can also tell Docker that my environment includes e.g. a custom Rails application.

In addition to specifying that all these services exist, I can use a Docker Compose file to specify how these services are supposed to be configured and how they are to talk to one another.

Docker Hub

Docker Hub is a place where Docker images are stored. You can think of it analogously to open source libraries being hosted on GitHub.

If I want to use Redis, for example, I can tell Docker to grab me the redis:4.0.14-alpine image and the default place that Docker will look for this image is on Docker Hub.

My hybrid approach to Dockerizing Rails applications

3 Replies

Why I started using Docker

I was a latecomer to the Docker craze. I simply didn’t have a problem that I left like Docker would solve for me.

Then, in 2020, a scenario arose for me that finally made me feel a need for Docker. My team at work was growing. I was finding that onboarding new developers was painful due to all the setup work we had to do on each new developer’s machine. I suspected that Docker could ease my pain.

The details of my Docker use case

First of all, I want to explain the exact use case that I used Docker for. There are two ways to Dockerize a Rails application. You can either Dockerize a Rails application to help with your development environment or your production infrastructure. My use case was to make my development environment easier.

I was tired of having each use have to install RVM, Ruby, RubyGems, PostgreSQL, Redis, etc. I wanted developers to be able to run a single command and have a complete development ready to go.

My good experiences with Docker

Regarding the objective of a complete development environment ready to go, I must say that Docker delivered on its promise. It took me a lot of difficult work but once I Dockerized my Rails application, it just worked. It was magical.

It was really helpful not to have to spend $HOURS to $DAYS getting a new developer set up with the development environment. It was nice not having to pair with people while we googled esoteric errors that only came up on their machine.

It was also really nice not to have to juggle services on my local machine. No more manually starting Rails, Sidekiq and Redis separately.

My bad experiences with Docker

Sadly, Docker’s benefits didn’t come without costs. Here were the downsides in rough descending order of severity.

Worse performance when running Rails commands

When a Rails app is Dockerized, you can no longer run commands like rails db:migrate because your local computer (the “host machine”) is not the one running Rails, your Docker container is. So you have to run docker-compose run web rails db:migrate.

Running Rails commands via Docker Compose took noticeably longer than running Rails commands straight on the host machine. These commands were so sluggish as to be intolerable.

Since tests are a big part of the workflow where I work, it was a real drag to have to pay this “performance tax” every time I wanted to run a test.

Worse performance when interacting with the app in the browser

Clicking around on stuff in the browser was slower as well. This issue wasn’t quite as bad as the command line issue but it was still bad enough to be a noticeable bummer.

No easy way to run tests non-headlessly

Most of the time I run my system specs headlessly. But it’s not uncommon for me to run across a difficult test which I need to run non-headlessly in order to see what’s going on with it.

Running tests non-headlessly directly on my host machine works fine. Trying to get my Docker container to open a browser for my was a nightmare though and I never did get it figured out.

No easy binding.pry

Since the docker-compose up command runs all services under the same parent process or whatever, you can’t just stick a binding.pry in your code and drop into the console inside the rails server process like you can if you’re not using Docker. To be fair, I understand there are ways around this, and I didn’t try very hard to solve this particular problem, so I might be griping about nothing with this one.

The hybrid Docker solution I developed instead

The solution I ultimately landed on was a hybrid approach. I decided to use Rails natively and Docker for Redis and PostgreSQL. That way I don’t have that impenetrable seal between me and my Rails application, but I still don’t have to manually install and run PostgreSQL and Redis.

This still leaves Sidekiq and webpack-dev-server. Luckily I found an easy fix for this. I just reverted to an old-fashioned solution, Foreman.

My Docker Compose config

Here’s what my docker-compose.yml looks like.

---
# Docker Compose 2.4 is for local development
# https://www.heroku.com/podcasts/codeish/57-discussing-docker-containers-and-kubernetes-with-a-docker-captain - Source on that.
version: '2.4'

services:
  postgres:
    image: postgres:13.1-alpine
    mem_limit: 256m
    volumes:
      - postgresql:/var/lib/postgresql/data:delegated
    ports:
      - "127.0.0.1:5432:5432"
    environment:
      PSQL_HISTFILE: /root/log/.psql_history
      POSTGRES_USER: mednote_development
      POSTGRES_PASSWORD: pgpassword
    restart: on-failure
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 2s
      retries: 10
    logging:
      driver: none

  redis:
    image: redis:4.0.14-alpine
    mem_limit: 64m
    volumes:
      - redis:/data:delegated
    ports:
      - "127.0.0.1:6379:6379"
    restart: on-failure
    logging:
      driver: none

volumes:
  postgresql:
  redis:
  storage:

That’s all I need for the Docker portion.

My Procfile

docker:    docker-compose up
web:       bundle exec puma -p 3000
worker:    bundle exec sidekiq
webpacker: bundle exec bin/webpack-dev-server

I can run all these things by doing foreman start -f Procfile.dev. Then, in a separate terminal window (or rather, a separate tmux pane) I run rails server so I can still use binding.pry tidily.

The trade-offs

The downsides of this approach are that each developer will still have to install RVM, Ruby, and RubyGems manually. But to me that’s much less of a downside than the downsides I experienced when my app was “fully” Dockerized.

Takeaways

  • Docker can be a great help in making it easier for each developer to get set up with a development environment.
  • Dockerizing an app “fully” can unfortunately come with some side effects that you may find unacceptable.
  • Using a hybrid approach can provide a sensible balance of the areas where using Docker is better versus the areas where going outside of Docker is better.

How to run a PostgreSQL database in a Docker container

Leave a reply

Why you would want to do this

There are two main reasons for Dockerizing an application: for production use and for development use.

When Dockerizing an application for development, sometimes you might want to package up every single thing in the development environment. For example, in a Rails app, you might want to run each of PostgreSQL, Redis, and Ruby/Rails inside containers.

The drawbacks of Dockerizing everything

But then again you might not want to do that. To continue with the Rails app example, it can be very slow to e.g. run your tests inside a Docker container every single time you want to run a test.

You may instead wish to run PostgreSQL and Redis inside containers but leave Rails out of it, and instead connect Rails to these various containerized components.

This particular blog post covers the most basic building block of the above scenario: running PostgreSQL inside of a Docker container. I’m not going to cover actually connecting an application to the database in this post. This post is not meant to give you a useful result, but rather to help you gain an understanding of this building block.

What we’re going to do

Here’s what we’re going to do:

  1. Create a Docker image that can run PostgreSQL
  2. Run a container based on that image
  3. Connect to the PostgreSQL instance running inside that container to verify that it’s working

Let’s get started.

Creating the image

The first step is to create a Dockerfile. Name this file Dockerfile and put it inside a fresh empty directory.

FROM library/postgres
COPY init.sql /docker-entrypoint-initdb.d/

The first line says to use the postres image from Docker Hub as our base image.

The second line says to take our init.sql file (shown below) and copy it into the special /docker-entrypoint-initdb.d/ directory. Anything inside the /docker-entrypoint-initdb.d/ directory will get run each time we start a container.

Here are the contents of init.sql, which should go right next to our Dockerfile.

CREATE USER docker_db_user;
CREATE DATABASE docker_db_user;
GRANT ALL PRIVILEGES ON DATABASE docker_db_user TO docker_db_user;

As you can see, this file contains SQL code that creates a user, creates a database, and then grants permissions to that user on the database. It’s necessary to do all this each time a container starts because a container doesn’t “remember” all this stuff from run to run. It starts as a blank slate each time.

Building the image

Now let’s build our image.

docker build .

Once this runs, we can run docker images to see the image we just created.

docker images

You should see something that looks like this:

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
<none>              <none>              396bfb8e3373        7 minutes ago       314MB

Running the container

We can use the docker run command to run a container based on our image. There are a few arguments that need to be passed, which I’ve annotated below.

docker run \
  --name dev-postgres \             # the arbitrary name of the container we're starting
  -e POSTGRES_PASSWORD=mypassword \ # the arbitrary db superuser password, which must be set to something
  396bfb8e3373                      # the image ID from our previous docker build step

Connecting to the database

Finally, we can connect to the database to verify that it works.

docker exec --interactive dev-postgres psql -U docker_db_user

If running this command puts you inside the PostgreSQL CLI, you’re good!