Category Archives: Programming

Rails model spec tutorial, part one

Overview

Because there are so many tools and concepts involved in Rails model testing, I’ve separated this beginner tutorial into two parts.

Part One will deal with writing tests just for plain old Ruby objects (POROs) in order to get comfortable with the process without bringing specifics of Rails into the picture.

Part Two will repeat what was done in Part One but in the context of an actual Rails model instead of a PORO.

Before we get into the tutorial itself we’ll discuss a little bit of necessary context: what a model is and how model specs are different from other types of specs.

What a model is

It may seem obvious what a Rails model is. To many Rails developers, the model is the MVC layer that talks to the database.

But in my experience, there are actually a lot of different conceptions as to what Rails models are, and not all of them agree with each other. I think it’s important for us to firmly establish what a Rails model is before we start talking about how to test Rails models.

To me, a model is an abstraction that represents a small corner of reality in a simplified way. Models exist to make it easier for a programmer to think about and work with the concepts in a program.

Models are not a Rails idea or even an OOP idea. A model could be represented in any programming language and in any programming paradigm.

In the context of Rails, models don’t have to be just the files that inherit from ActiveRecord::Base. I create most of my models as plain old Ruby objects (POROs) and put them in app/models right alongside Active Record models because to me the distinction between Active Record models and PORO models isn’t very important of a distinction.

Why model specs are different from other types of specs

Because models aren’t a Rails idea but rather a programming idea, testing models in Rails isn’t that conceptually different from testing models in any other language or framework.

In a way this is a great benefit to a learner because it means that if you know how to test models in one language, your testing skills will easily translate to any other language.

On the other hand this is a difficulty because, relative to other types of Rails tests, model tests don’t have such a straight and narrow path to success. The generic-ness of Rails models means that there’s not a rigid template that can be followed in order to write Rails models, unlike, for example, system specs.

System specs are relatively easy to get started with because you can more or less follow a certain step-by-step formula for writing system specs for CRUD features and be well on your way. There’s not as much of a step-by-step formula for writing model tests.

Nonetheless, there are certain principles and tactics you can learn. Let’s take a look at some of the ideas that can help you get started.

The tutorial

Learning objectives

Here are the things you can expect to have a better understanding of after completing this tutorial.

  1. How to come up with test cases for a model based on the model’s desired behavior
  2. How to translate those test cases into actual working test code, in a methodical and repeatable manner
  3. How to use a test-first approach to make it easier both to write the tests and to write the application code

The scenario

The model we’re going to be writing tests for is a model that represents a phone number. Our phone number model will be designed, at least for starters, to mainly deal with just one particular challenge that tends to be common in applications that deal with phone numbers.

This challenge is that it’s usually desired to display phone numbers in a consistent format (e.g. “(555) 555-5555”), but phone numbers are often entered by users in a variety of formats (e.g. “555-555-5555” or “1 555 555 5555”).

A common way of dealing with this problem is to strip down the user input so that it’s all numbers and store the all-numbers version in the database. So, for example, 555-555-5555 would become 5555555555.

We want our phone number model to be able to take phone numbers in any of the following formats:

555-856-8075
(555) 856-8075
+1 555 856 8075

And strip them down to look like this:

5558568075

Our PhoneNumber class won’t know anything about databases, it will just be responsible for converting a “messy” phone number to a normalized one.

The approach

We’re going to take the following three scenarios

555-856-8075
(555) 856-8075
+1 555 856 8075

and write tests for each one.

The shell of the spec and PhoneNumber class

Our beginning spec will contain the bare minimum: an outer describe block and a require_relative that includes our “application code”, phone_number.rb.

require_relative './phone_number.rb'

RSpec.describe PhoneNumber do
end

The phone_number.rb file will only contain the shell of a class.

class PhoneNumber
end

Our first test

A big part of the art of model testing is coming up with various scenarios and deciding how our code should behave under those scenarios.

The first scenario we’ll test here is: “When we have a phone number where the digits are separated by dashes, the dashes should all get stripped out.”

We’ll add a context block that describes our scenario and an it block that describes our expected behavior. Inside the it block, we’ll create our test data (an instance of PhoneNumber) and an expectation: the expectation that we’ll get a dashless version of our original dashed phone number.

require_relative './phone_number.rb'

RSpec.describe PhoneNumber do
  context "phone number contains dashes" do
    it "strips out the dashes" do
      phone_number = PhoneNumber.new("555-856-8075")

      expect(phone_number.value).to eq("5558568075")
    end
  end
end

We haven’t yet defined an initializer for PhoneNumber, so passing a phone number to PhoneNumber when we instantiate an object couldn’t possibly work, but let’s defer worrying about that to the future. We don’t want to try to think about too many things at once. For now all we care about is that our test is constructed properly.

Let’s run the spec.

$ rspec phone_number_spec.rb

Very unsurprisingly, we can an error.

ArgumentError: wrong number of arguments (given 1, expected 0)

Let’s write just enough code to make this error go away and nothing else.

class PhoneNumber
  def initialize(value)
  end
end

When we run the test again we get a new error:

NoMethodError: undefined method 'value' for #<PhoneNumber:0x00007f8541abcb60>

Indeed, when we do expect(phone_number.value).to eq("5558568075"), we’re sending a message (value) to which PhoneNumber doesn’t respond. Let’s fix this error by making PhoneNumber respond to value. The simplest way we can do this is by adding an attr_reader.

class PhoneNumber
  attr_reader :value

  def initialize(value)
    @value = value
  end
end

Now we get yet a different error when we run the spec.


Failures:

  1) PhoneNumber phone number contains dashes strips out the dashes
     Failure/Error: expect(phone_number.value).to eq("5558568075")
     
       expected: "5558568075"
            got: "555-856-8075"

At this point it finally feels like we’re getting somewhere. The error feels like something we can work with. Instead of getting the expected value of 5558568075, we’re getting 555-856-8075.

Let’s write the code that fixes this error—and actually makes the test pass.

class PhoneNumber
  attr_reader :value

  def initialize(value)
    @value = value.gsub(/-/, "")
  end
end

Now when we run the test it passes.

The other two formats

Of our three scenarios, we have so far covered just the first one. Let’s address the next two.

555-856-8075
(555) 856-8075
+1 555 856 8075

We can add a second context block to our test, very similar to our first context block.

require_relative "./phone_number.rb"

RSpec.describe PhoneNumber do
  context "phone number contains dashes" do
    it "strips out the dashes" do
      phone_number = PhoneNumber.new("555-856-8075")

      expect(phone_number.value).to eq("5558568075")
    end
  end

  context "phone number contains parentheses" do
    it "strips out the non-numeric characters" do
      phone_number = PhoneNumber.new("(555) 856-8075")

      expect(phone_number.value).to eq("5558568075")
    end
  end
end

When we run this, it fails. We get:

1) PhoneNumber phone number contains parentheses strips out the dashes
     Failure/Error: expect(phone_number.value).to eq("5558568075")
     
       expected: "5558568075"
            got: "(555) 8568075"
     
       (compared using ==)

This makes sense because we’re of cours not doing anything yet to strip out parentheses or spaces. Let’s add parentheses and spaces to our regex.

class PhoneNumber
  attr_reader :value

  def initialize(value)
    @value = value.gsub(/[- ()]/, "")
  end
end

Now, when we run the test, it passes.

Refactoring

And in fact, we can use the safety net of this test to do a little refactoring. Our regular expression is a little bit more complicated than it needs to be. We can change it to just \D, which means “anything that’s not a number”.

class PhoneNumber
  attr_reader :value

  def initialize(value)
    @value = value.gsub(/\D/, "")
  end
end

The last scenario

The final scenario we want to address is when the phone number has a country code. Again, we can add a context block that’s similar to the other two context blocks.

require_relative "./phone_number.rb"

RSpec.describe PhoneNumber do
  context "phone number contains dashes" do
    it "strips out the dashes" do
      phone_number = PhoneNumber.new("555-856-8075")

      expect(phone_number.value).to eq("5558568075")
    end
  end

  context "phone number contains parentheses" do
    it "strips out the non-numeric characters" do
      phone_number = PhoneNumber.new("(555) 856-8075")

      expect(phone_number.value).to eq("5558568075")
    end
  end

  context "phone number contains country code" do
    it "strips out the country code" do
      phone_number = PhoneNumber.new("+1 555 856 8075")

      expect(phone_number.value).to eq("5558568075")
    end
  end
end

When we run this, it fails. The plus sign and spaces get taken care of but not the 1.

1) PhoneNumber phone number contains country code strips out the country code
     Failure/Error: expect(phone_number.value).to eq("5558568075")
     
       expected: "5558568075"
            got: "15558568075"
     
       (compared using ==)

To get rid of the 1, we can just grab everything that’s not the 1, i.e. the last 10 digits of the phone number.

class PhoneNumber
  attr_reader :value

  def initialize(value)
    @value = value.gsub(/\D/, "").split("").last(10).join
  end
end

The 10 is a “magic number” though, which is bad, so let’s assign it to a constant instead of letting it be mysterious.

class PhoneNumber
  attr_reader :value
  EXPECTED_NUMBER_OF_DIGITS = 10

  def initialize(value)
    @value = value.gsub(/\D/, "")
      .split("")
      .last(EXPECTED_NUMBER_OF_DIGITS)
      .join
  end
end

Now all our tests pass and our code is pretty good as well.

Takeaways

  1. A Rails model is an abstraction that represents a small corner of reality in a simplified way.
  2. Because models are so much more of an open-ended concept than certain other components of Rails (e.g. controllers), writing model specs can often be a more ambiguous task for beginners.
  3. One good way to write model specs is to list the specific fine-grained behaviors that you want to exist and to write test cases for each of those behaviors.
  4. Writing tests before we write the application code can make the process of writing the application code easier.

What’s next

In Part Two of the tutorial, we’ll go through the same process we did in Part One, but with a real Active Record model instead of just a plain old Ruby object.

Continue to Part Two

A beginner-friendly Rails model spec tutorial

What this is

Using you’re using RSpec to write Rails tests, there are several different types of tests you can write. There are system specs, request specs, view specs, model specs, and a number of others.

Among all these types of tests, I consider model specs to be perhaps the most important. Models are where I put the “meat” of my application’s code. Since model code is my most important code, model specs are my most important specs.

If haven’t done much model testing before and you aren’t sure how to come at it, this tutorial will show you how to take an idea for a feature and translate it into model specs.

Who this is for

This is a very beginner-level tutorial, intended for developers with very little model testing experience.

If you’ve never done any testing at all before, I would first recommend my Rails testing “hello world” tutorial and my Getting Started with Rails Testing articles.

But if you have a little bit of familiarity with RSpec and want to understand how to get started with testing models, let’s get started with Part One of this two-part tutorial.

Continue to Part One

Models are a convenient approximation, not a complete account

Someone recently criticized some code I wrote in a class called SomeNamespace::File. (The particular namespace is not important.) Here’s the criticism:

I’d argue that the class name is misleading here: it does not represent a file, it does not read it or check its chmods. It’s a file writer.

The implication seems to be that a class can’t truthfully be called File unless the object does certain “filey” things, like reading the file and checking its chmods, in addition to just writing to the file.

I think this is a mistaken outlook. However, it is intuitive outlook, and one I used to share myself. I’ll explain why I think the outlook is mistaken via a couple examples.

Two pocketknives

Let’s say I own a pocketknife. In different scenarios this pocketknife can serve me in different ways. For example, I might take my knife camping and use it to chop vegetables for dinner. At another time, I might take my knife into a dark alley where I use the knife to stab people.

In one scenario I could conceive of my knife as a Camping::PocketKnife. In the other scenario I would conceive of it as a DarkAlley::PocketKnife. It wouldn’t serve me well to merge all of the knife’s potential uses into one class. This would make the code harder to understand and change.

And it certainly wouldn’t be necessary or desirable to model all the characteristics of the knife including the material it’s made out of, the brand of the knife or any other details that aren’t relevant to the aspects of the knife that we need to work with.

A model is a convenient approximation, not a complete account. Just because a complex item exists in the real world doesn’t mean that a model has to represent all parts of that item’s complexity. It’s also true that a real-world item can have multiple different models that each look at the item from different angles.

Appointments in different contexts

I think my pocketknife example is a good illustration because it’s easy to understand but it’s also a bad example because we’d never model a pocketknife in such a way. Here’s a more realistic example.

Let’s say we have a medical application. In our application, patients schedule appointments with doctors. Sometimes we care about appointments in a clinical sense. Sometimes we’re interested in an appointment in a scheduling sense. At other times we’re interested in an appointment in a billing sense.

If I have a model called Billing::Appointment, someone could conceivably criticize that class as being inappropriately called an Appointment, because the class doesn’t include anything to do with scheduling. They’d be missing the point though. I’m not trying to model a whole appointment at a time, in all its possible aspects, just as I’m never trying to model a whole pocketknife at a time in all its possible aspects. I’m only trying to model the part I care about in a certain context.

Takeaway

A model is a convenient approximation, not a complete account. A model only needs to represent the slice of reality that we care about in that particular context.

How I code without service objects

What service objects are

Service objects and Interactors are fashionable among Rails developers today. I can’t speak for everyone’s motives but the impression I get is that service objects are seen as a way to avoid the “fat models” you’re left with if you follow the “skinny controllers, fat models” guideline. If your models are too fat you can move some of their bulk to some service objects to thin them out.

Unfortunately it’s not quite possible to say what service objects are because the term “service object” means different things to different people. To some developers, “service object” is just a synonym for plain old Ruby object (PORO). To other developers, a service object is more or less an implementation of the command pattern. Interactors are similarly close to the command pattern.

For the purposes of this post we’ll go with the “command pattern” conception of a service object: a Ruby class with a single method, call, which carries out a sequence of steps.

I have some gripes with the way service objects are advocated in the Ruby community. I’ll explain why.

Pattern vs. paradigm

The command pattern has its time and place. Sometimes it makes plenty of sense to bundle up an action into a single object and let that object orchestrate the steps that need to happen for that action.

But the command pattern is a pattern, not a paradigm. A software design pattern is a specific solution meant to meet a specific problem. A programming paradigm (like object-oriented programming or functional programming) is a style that can be applied at the level of entire applications.

What pains me a little bit to see is that many developers seem to have confused the idea of a pattern with the idea of a paradigm. They see service objects or Interactors and think, “This is how we’ll code now!” They have a service-object-shaped hammer and now everything is a nail.

Imperative vs. declarative

An imperative programming style expresses code mainly in terms of what to do. A declarative programming style expresses code mainly in terms of what things are.

For those not very familiar with imperative vs. declarative, here’s an explanation by way of example.

An imperative sandwich

If I were to tell a robot to make me a peanut butter and jelly sandwich using the imperative style, I would say, “Hey robot, get two pieces of bread, put some peanut butter on one, put some jelly on the other, put them together, put it on a plate, and give it to me.”

A declarative sandwich

If I were to do the same thing in a declarative style, I would say, “A sandwich is a food between two pieces of bread. It comes on a plate. A peanut butter and jelly sandwich is a sandwich that contains peanut butter and jelly. Make me one.”

My preference for declarative

I’m strongly of the view that declarative code is almost always easier to understand than imperative code. I think for the human brain it’s much easier to comprehend a static thing than a fleeting action.

At some point the things do have to take action (the peanut butter and jelly sandwich has to get made) but we can save the action part for the very highest level, the tip of the pyramid. Everything below can be declarative.

Service objects are all imperative

Service objects and Interactors are basically all imperative. Their names tend to be verbs, like NotifySlack or AuthenticateUser. Their guts tend to be imperative as well. To me this is a pity because, like I said above, I think most things are more clearly expressed in a declarative style. Even things that seem at first glance to lend themselves more naturally to an imperative style can often be expressed even more naturally in a declarative way.

Takeaways

  • The command pattern is a pattern, not a paradigm. It’s not a style, like OOP or FP, that should be applied to all your code.
  • Declarative code is often easier to understand than imperative code.
  • Regular old OOP is a fine way to program. Not everything in Rails has to utilize a fancy tool or concept.

What is a Rails model?

In the model-view-controller pattern that Rails is built on, it’s pretty clear what views and controllers are, but models are a little less clear. In my years with Rails I’ve had the opportunity to come across a number of different conceptions regarding what models are. These conceptions often overlap but often don’t share the same exact contours.

Our conception of what a model is is super important because it has a huge influence on how we structure our Rails applications, which in turn determines how easy and economical it’s going to be to maintain our applications.

Here’s how I think about what Rails models are.

How I conceive of Rails models

First of all, forget about Rails

I think one of the most helpful mindset shifts a Rails developer can make with regard to models is to think about models outside the concept of Rails, at least for a moment. Instead, let’s consider: What is a model in general? Why do we use models?

And in fact, not to get too crazy, but we can even think about models even outside the context of programming. Scientists and mathematicians use models as well. Why do people in these professions use models? If you asked a scientist what a model is, what would they say?

Non-programming models

According to Wikipedia, “Scientific modeling is a scientific activity, the aim of which is to make a particular part or feature of the world easier to understand, define, quantify, visualize, or simulate by referencing it to existing and usually commonly accepted knowledge.”

Let’s see if we can draw inspiration from this to describe models in programming.

My definition of a model

I might adapt the above definition of a scientific model to programming as follows: “Modeling is a programming activity, the aim of which is to make a particular part of a program easier to understand and work with by representing parts of reality in a simplified way that can be expressed in code.”

A model is not a simulation

It’s common for object-oriented programming tutorials to use examples like Dog and Cat each inheriting from Animal, and with methods like bark and meow. I find these kinds of examples misleading. These entities aren’t models, they’re simulations, and we programmers almost never build simulations.

Instead we create models with names more like User, PatientPayment or HashWithIndifferentAccess. These models are useful precisely because they break off a tiny part of the infinite hugeness of reality and provide us with a simplified abstraction we can work with, rather than having to deal with all of reality at once.

In programming we would never try to model an entire dog or cat. Rather, we try to model the tiny slices of reality related to dogs and cats that we care about. For example, for a pet store application, we might keep track of a dog’s name, the dog’s owner’s contact information, and other things like that, but never e.g. its skeletal structure or biological taxonomy. And we certainly wouldn’t create a method that invokes a bark (whatever that could even mean).

A model doesn’t need to have anything to do with Active Record

I used to think that Rails applications were supposed to have exactly one model per database table and that every model file was supposed to inherit from ActiveRecord::Base.

Now, with my broader conception of models, not only do I not think models have to inherit from Active Record, but most of my models don’t. Most of my models are just plain old Ruby objects (POROs). In fact, I don’t even see why models should have to take the form of objects at all, but since I’m personally a big OOP fan and so I almost always write my models in the form of objects.

A model doesn’t have to have anything to do with business/domain logic

Some developers define models as the part of the application that has to do with domain logic. I don’t see it this way.

To me, a model can model anything. A model can model domain logic (i.e. stuff that has to do with the peculiarities of the organization/industry the application serves) or it can model application logic (i.e. stuff that’s purely technical, and doesn’t particularly have anything to do with your specific domain).

A Rails app can be made almost entirely of models

I view Rails controllers as just very thin entry points that deal mostly with HTTP requests and responses. The real “meat” of the application is in its models. And I find that models, in the form of POROs, can account for 95%+ of my needs.

I don’t use Interactors or service objects or any other pseudo-patterns like that. In fact, it could be argued that Interactors and service objects are models, they’re just models that have been adulterated with some confused ideas and therefore are less effective at the job models are intended to do: to simplify and abstract reality to make reality easier to work with in code.

Takeaways

  • Modeling is an activity that aims to make a particular part of a program easier to understand and work with by representing parts of reality in a simplified way that can be expressed in code.
  • A model is not a simulation. Rather, it’s an abstraction.
  • A model doesn’t have to inherit from Active Record.
  • A model doesn’t have to specifically deal with domain/business logic. A model can deal with any aspect of reality, including imagined realities that only exist in the context of your program.
  • Rails apps can be made almost entirely of models. No Interactors or service objects are necessary.

Testing implementation vs. behavior in Rails

One of the biggest mistakes I see in Rails testing (especially model testing) is testing implementation instead of behavior.

Let’s say I run a factory that makes cars. I want to make sure that each car I make actually works. How should I go about making sure each car works?

Here are two possible methods.

Testing implementation

One way I could try to make sure each car works is to check to see if it has all the right stuff. I could check for the presence of an engine, an ignition, a brake system, wheels, and everything else that’s needed in order to get from point A to point B. If the car has all this stuff, then it’s all set.

Testing behavior

Another way I could try to make sure the car works is to start it up and try to drive it somewhere.

The first way is bad and the second way is good. Here’s why.

Why testing behavior is better

If I “verify” that my car works by checking for the presence of various parts, then I haven’t really actually verified anything. I haven’t demonstrated that the system under test (the car) actually meets spec (can drive).

If I test the car by actually driving it, then the questions of whether the car has various components become moot. If for example the car can travel down the road, we don’t need to ask if the car has wheels. If it didn’t have wheels it wouldn’t be moving.

All of our “implementation” questions can be translated into more meaningful “behavior” questions.

  • Does it have an ignition? -> Can it start up?
  • Does it have an engine and wheels? -> Can it drive?
  • Does it have brakes? -> Can it stop?

Lastly, behavior tests are better than implementation tests because behavior tests are more loosely coupled to the implementation. I ask “Can it start up?” instead of “Does it have an engine?” then I’m free to, for example, change my car factory from a gasoline-powered car factory to an electric car factory without having to change the set of tests that I perform. In other words, behavior tests enable refactoring.

Different granularities of behavior testing

I want to be sure that my argument isn’t misconstrued as a discussion between unit tests and integration tests. The implementation tests being performed weren’t actually verifying the proper behavior of the various components (engine, ignition, etc.) in isolation. The implementation tests were merely verifying the presence of components.

In fact, performing some “unit” tests and some “integration” tests in my car factory would probably be a good idea. It would be smart to make sure each part works individually before I combine them all. I’d just want to make sure my unit and integration tests were testing behavior, not implementation.

Verifying that my car can drive is a test at the highest granularity. At a lower level of granularity, I might want to verify that the engine works, even before the engine is connected to an actual car. To test my engine, I wouldn’t want to check to see whether the engine has six cylinders, each with a piston inside. Instead I would want to give my engine some fuel and actually see if it runs.

So please don’t confuse the implementation vs. behavior question with the unit test vs. integration test question. We sometimes want to perform unit tests and we sometimes want to perform integration tests, but we pretty much always want to test behavior instead of implementation.

How this relates to Rails

I sometimes come across examples of Rails model tests that I consider pointless. These tests verify the presence of validations, associations, callbacks and other implementation details. These tests are pointless because they don’t actually test anything. Having these tests is like checking for the presence of your engine and brakes rather than actually just driving your car.

It’s also pointless to write system specs that e.g. check for the presence of various form fields and buttons, for the same exact reason.

Takeaway

Test behavior, not implementation.

My hybrid approach to Dockerizing Rails applications

Why I started using Docker

I was a latecomer to the Docker craze. I simply didn’t have a problem that I left like Docker would solve for me.

Then, in 2020, a scenario arose for me that finally made me feel a need for Docker. My team at work was growing. I was finding that onboarding new developers was painful due to all the setup work we had to do on each new developer’s machine. I suspected that Docker could ease my pain.

The details of my Docker use case

First of all, I want to explain the exact use case that I used Docker for. There are two ways to Dockerize a Rails application. You can either Dockerize a Rails application to help with your development environment or your production infrastructure. My use case was to make my development environment easier.

I was tired of having each use have to install RVM, Ruby, RubyGems, PostgreSQL, Redis, etc. I wanted developers to be able to run a single command and have a complete development ready to go.

My good experiences with Docker

Regarding the objective of a complete development environment ready to go, I must say that Docker delivered on its promise. It took me a lot of difficult work but once I Dockerized my Rails application, it just worked. It was magical.

It was really helpful not to have to spend $HOURS to $DAYS getting a new developer set up with the development environment. It was nice not having to pair with people while we googled esoteric errors that only came up on their machine.

It was also really nice not to have to juggle services on my local machine. No more manually starting Rails, Sidekiq and Redis separately.

My bad experiences with Docker

Sadly, Docker’s benefits didn’t come without costs. Here were the downsides in rough descending order of severity.

Worse performance when running Rails commands

When a Rails app is Dockerized, you can no longer run commands like rails db:migrate because your local computer (the “host machine”) is not the one running Rails, your Docker container is. So you have to run docker-compose run web rails db:migrate.

Running Rails commands via Docker Compose took noticeably longer than running Rails commands straight on the host machine. These commands were so sluggish as to be intolerable.

Since tests are a big part of the workflow where I work, it was a real drag to have to pay this “performance tax” every time I wanted to run a test.

Worse performance when interacting with the app in the browser

Clicking around on stuff in the browser was slower as well. This issue wasn’t quite as bad as the command line issue but it was still bad enough to be a noticeable bummer.

No easy way to run tests non-headlessly

Most of the time I run my system specs headlessly. But it’s not uncommon for me to run across a difficult test which I need to run non-headlessly in order to see what’s going on with it.

Running tests non-headlessly directly on my host machine works fine. Trying to get my Docker container to open a browser for my was a nightmare though and I never did get it figured out.

No easy binding.pry

Since the docker-compose up command runs all services under the same parent process or whatever, you can’t just stick a binding.pry in your code and drop into the console inside the rails server process like you can if you’re not using Docker. To be fair, I understand there are ways around this, and I didn’t try very hard to solve this particular problem, so I might be griping about nothing with this one.

The hybrid Docker solution I developed instead

The solution I ultimately landed on was a hybrid approach. I decided to use Rails natively and Docker for Redis and PostgreSQL. That way I don’t have that impenetrable seal between me and my Rails application, but I still don’t have to manually install and run PostgreSQL and Redis.

This still leaves Sidekiq and webpack-dev-server. Luckily I found an easy fix for this. I just reverted to an old-fashioned solution, Foreman.

My Docker Compose config

Here’s what my docker-compose.yml looks like.

---
# Docker Compose 2.4 is for local development
# https://www.heroku.com/podcasts/codeish/57-discussing-docker-containers-and-kubernetes-with-a-docker-captain - Source on that.
version: '2.4'

services:
  postgres:
    image: postgres:13.1-alpine
    mem_limit: 256m
    volumes:
      - postgresql:/var/lib/postgresql/data:delegated
    ports:
      - "127.0.0.1:5432:5432"
    environment:
      PSQL_HISTFILE: /root/log/.psql_history
      POSTGRES_USER: mednote_development
      POSTGRES_PASSWORD: pgpassword
    restart: on-failure
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 2s
      retries: 10
    logging:
      driver: none

  redis:
    image: redis:4.0.14-alpine
    mem_limit: 64m
    volumes:
      - redis:/data:delegated
    ports:
      - "127.0.0.1:6379:6379"
    restart: on-failure
    logging:
      driver: none

volumes:
  postgresql:
  redis:
  storage:

That’s all I need for the Docker portion.

My Procfile

docker:    docker-compose up
web:       bundle exec puma -p 3000
worker:    bundle exec sidekiq
webpacker: bundle exec bin/webpack-dev-server

I can run all these things by doing foreman start -f Procfile.dev. Then, in a separate terminal window (or rather, a separate tmux pane) I run rails server so I can still use binding.pry tidily.

The trade-offs

The downsides of this approach are that each developer will still have to install RVM, Ruby, and RubyGems manually. But to me that’s much less of a downside than the downsides I experienced when my app was “fully” Dockerized.

Takeaways

  • Docker can be a great help in making it easier for each developer to get set up with a development environment.
  • Dockerizing an app “fully” can unfortunately come with some side effects that you may find unacceptable.
  • Using a hybrid approach can provide a sensible balance of the areas where using Docker is better versus the areas where going outside of Docker is better.

The two ways to Dockerize a Rails application

Here’s something that should be Docker 101, but for some reason people don’t really tell you: there’s no such thing as just “Dockerizing” a Rails app. There are two use cases for using Docker with Rails. Before you go down the path of Dockerizing your app, you have to pick which use case you’re after. The implementation for each is very different from the other.

You can Dockerize an app for development or you can Dockerize it for production.

Dockerizing for development

The reason you would want to Dockerize an app for development is to make it easier for a new developer to get a development environment set up on their machine.

When you have your app Dockerized for development, Docker can install and run services for you. For example, if your app uses PostgreSQL, Redis, webpack-dev-server, and Sidekiq, Docker will run all those processes for you, and before that Docker will install PostgreSQL and Redis if you don’t already have them.

This means that each new developer who starts the project doesn’t have to go through a potentially long and painful process of installing all the dependencies for your app before getting to work. They can just clone the repo, run a single command, and be up and running.

Dockerizing for production

The benefits of Dockerizing for production overlap with Dockerizing for development, but they’re not exactly the same.

In my development example I mentioned that you might have certain services running locally like PostgreSQL, Redis, webpack-dev-server, and Sidekiq. In a production environment, you of course don’t have all these processes running on a single machine or VM. Rather, you might have (if you’re using AWS for example), your database hosted on RDS, a Redis instance on ElasticCache, Sidekiq running on a worker EC2 instance, and no webpack-dev-server because that doesn’t apply in production. So the need is very different.

So unlike a development use case, a production Docker use case doesn’t include installing and running various services for you, because running all kinds of services on a host machine is not how a “modern” production infrastructure configuration works.

Takeaways

  • There are two ways to Dockerize a Rails app: for development and for production.
  • Dockerizing for development makes it easier to spin up a development environment.
  • Dockerizing for production helps avoid the problem of having snowflake servers.
  • Dockerizing for development could probably benefit just about any team, but Dockerizing for production is probably much more dependent on your app’s unique hosting needs.

Developer success == individual * environment

I used to think that a good developer could work pretty much anywhere and do good work. If the developer is good, then they’ll of course be good anywhere.

I’ve learned in recent years that this is not actually the case. If a developer gets hired somewhere and fails to do good work, it’s not always because that developer is a bad developer. Sometimes it’s because the developer’s employer fails to provide an environment conducive to success.

What a bad environment looks like

Chaos

In my experience, one of the common characteristics of a bad developer environment is chaos. There’s no development methodology, there’s no issue tracking, there’s no intelligent long-term project planning. Everybody just kind of does stuff.

If a really experienced developer enters this sort of environment, they might be able to help introduce some order to the organization or at least keep their little corner of the world orderly in spite of everyone else. But someone who’s earlier in their career, even if they’re really smart, might not have the skills and experience to do that. I’ve talked with junior developers before who have worked in very chaotic organizations who just thought it was normal. It was all they had ever known and so they had nothing to compare it to.

Managers in these types of organizations might expect a certain level of productivity out of their developers, not realizing that their organization’s environment makes good productivity unlikely, and conclude that the fault lies with the developers. Sadly, the developers might share this conclusion, ignorant of the fact that the chaotic environment is responsible for most of the productivity shortcoming.

Vague expectations

Along with chaos typically come vague expectations. This can happen at a low or high level, or of course both.

As an example of a low-level vague expectation, a developer might get assigned a ticket that simply says “Ability to add products” with few or no other details. There’s no way to tell when the ticket is complete, or whether the work done matches the expectation.

As an example of a high-level vague expectation, a developer might get hired with a title that implies role X when, all along, his superiors expected him to fulfill role Y, but never explicitly stated this expectation until several months after he started. (This exact scenario happened to me once.)

When a developer isn’t told what’s expected of them, it’s really hard for them to be successful. If what the developer does matches what’s expected, it’s only by chance.

Poor technical practices

If you put a good developer on a team that uses poor development practices, it’s going to be hard for that developer to make a good contribution.

I’ve worked at places where they didn’t use version control or practice continuous deployment. Obviously these poor practices can cause a good developer to produce bad work.

Lack of education/culture of ignorance

Some developers are smart but lack knowledge. If care isn’t taken to educate these developers, they often won’t produce very good work.

Some organizations simply fail to provide education to their employees. Others actively discourage it. I once worked at a place where my boss denied a request of mine to buy some books with company money because “you can read that stuff online”.

What a good environment looks like

Order

The best places I’ve worked at have used some sort of development methodology. Usually that development methodology was “agile-ish”, some form of not-quite-by-the-book agile. Even agile done badly is better than no methodology at all though.

When requirements are thoroughly thought out and specified for the developers in great detail before the work is assigned, it makes the developer’s work much clearer. The developer doesn’t have to spend much energy on parsing ambiguities and can instead just focus on the actual implementation work.

Clear expectations

When assigning user stories to developers, I like to include a bulleted list of acceptance criteria in the description for each story. That way it’s trivially easy to tell whether a story is complete and correct. Just go down the list of acceptance criteria and see if each item matches.

The specificity of the requirements should be tailored to the experience level of the developer. Unless the developer is very senior, it’s often a good idea to include not only high-level requirements but also some guidance regarding things like database structure.

I also like to include a UI mockup of the feature along with the story, sometimes even if the story is trivial. Miscommunication is super easy and so it’s better to over-communicate than under-communicate. Visual communication often helps clear up ambiguities.

There are certain other high-level types of expectations I like to set as well. For example, on the team I lead, I’ve set certain expectations around when and how to refactor. (Refactoring is encouraged. We don’t mix refactoring with behavior changes. We perform refactorings right before or right after behavior changes.)

Good technical practices

Things can go so much more smoothly when the technical practices are good.

Where I work, it goes like this:

  1. You’re assigned a small story with clear acceptance criteria
  2. You work on the story, which could take a couple hours or as much as a day, typically not more
  3. You open a PR which is reviewed 1 to 24 hours later (usually more like 1)
  4. Your PR is merged and your work is immediately deployed and verified in production
  5. You start again at step 1 with a new story

We’re able to move quickly like this because we have a solid test suite and we practice continuous delivery.

Things go much better when we work this way than if we were to try to assign them big projects that would only be verified, merged and deployed weeks or months after they were started.

Good education/culture of learning

When developers are told that their work output is the most important thing, and that “it doesn’t have to be perfect, it just has to work“, then the quality of the work output suffers.

I believe that developers can actually work faster when they’re allowed to take the time to learn how to do their work the right way rather than just cramming things in any old way. As Bob Martin has said, “the only way to go fast is to go well.”

Takeaways

A programmer’s ability to be successful is the product of the quality of the programmer and the quality of the programmer’s environment. I mean “product” in the mathematical sense. A programmer who is “90% good” who is put into an environment that is only “10% good” will produce work that’s only 90% * 10% = 9% good.

If you work with developers who aren’t producing good work, maybe it’s their fault, but maybe it’s the environment they’re in. And if you yourself feel like you’re not producing good work, it might be your fault or it might be that you’re in a bad environment. If you can find a better environment to put yourself in, you might find yourself able to do better work.

Don’t wrap instance variables in attr_reader unless necessary

I’ve occasionally come across advice to wrap instance variables in attr_reader. There are two supposed benefits of this practice, which I’ll describe shortly. First I’ll provide a piece of example code.

Example of wrapping instance variables in attr_reader

Original version

Here’s a tiny class that has just one instance variable, @name. You can see that @name is used in the loud_name method.

class User
  def initialize(name)
    @name = name
  end

  def loud_name
    "#{@name.upcase}!!!"
  end
end

attr_reader version

Here’s that same class with an attr_reader added. Notice how loud_name now references name rather than @name.

class User
  attr_reader :name

  def initialize(name)
    @name = name
  end

  def loud_name
    "#{name.upcase}!!!"
  end
end

The purported benefits

Advocates of this technique seem to find two certain benefits in it. (I’ve also come across other supported benefits but I don’t find them strong enough to merit mentioning.)

It makes refactoring easier

Rationale: If you ever want to change @name from being defined by an instance variable to being defined by a method, then you don’t have to go changing all the instances of @name to name.

This reasoning isn’t wrong but it is weak. First, it’s very rare that a value will start its life as an instance variable and then at some later point need to change to a method. This has happened to me so few times that I can’t recall a single instance of it happening.

Second, the refactoring work that the attr_reader saves is only a trivial amount of work. The cost of skipping the attr_reader is that you have to e.g. change a handful of instances of @name, in one file, to name. Considering that this is a tiny amount of work and that it needs to happen perhaps once every couple years per developer, this justification seems very weak.

It saves you from typo failures

Rationale: If you’re using instance variables and you accidentally type @nme instead of @name, @nme will just return nil rather than raising an error. If you’re using attr_reader and you accidentally type nme instead of name, nme will in fact raise an error.

This justification is also true, but also weak. If typo-ing an instance variable allows a bug to silently enter your application, then your application is not tested well enough.

I would be in favor of saving myself from the typo problem if the attr_reader technique hardly cost anything to use, but as we’ll see shortly, the attr_reader technique’s cost is too high to justify its benefit. Since the benefit is so tiny, the cost would have to be almost nothing, which it’s not.

Reasons why the attr_reader technique is a bad idea

Adding a public attr_reader throws away the benefits of encapsulation

Private instance variables are useful for the same reason as private methods: because you know they’re not depended on by outside clients.

If I have a class that has an instance variable called @price, I know that I can rename that instance variable to @cost or change it to @price_cents (changing the whole meaning of the value) or even kill @price altogether. What I want to do with @price is 100% my business. This is great.

But if I add attr_reader :price to my class, my class suddenly has responsibilities. I can no longer be sure that the class where @price is defined is the only thing that depends on @price. Other clients throughout my application may be referring to @price. I’m no longer free to do away with @price or change its meaning. This makes my code riskier and harder to change.

You can add a private attr_reader, but that’s unnatural

If you want to make use of the attr_reader technique but you don’t want to throw away the benefits of encapsulation, you can add a private attr_reader. Here’s what that would look like.

# attr_reader version

class User
  def initialize(name)
    @name = name
  end

  def loud_name
    "#{name.upcase}!!!"
  end

  private

  attr_reader :name
end

This solves the encapsulation problem, but what have we really gained on balance? In exchange for not having to change @name to name on the off chance that we change name from an instance variable to a method, we have to pay the price of having this weird private attr_reader :name thing at the bottom of our class.

And consider that we would have to do this on every single class that has at least one instance variable!

Don’t wrap instance variables in attr_reader

Wrapping your instance variables in a public attr_reader changes your instance variables from private to public, increasing the public surface area of your class’s API and making your application a little bit harder to understand.

Wrapping your instance variables in a private attr_reader adds an unnatural piece of boilerplate to all your Ruby classes.

Given the tiny and dubious benefits that the attr_reader technique provides, this cost isn’t worth it.

Using attr_reader is good and necessary for values that really need to be public. As a default policy, wrapping instance variables in attr_reader is a bad idea.