The problem to be solved

As a Rails application grows, its controllers tend to accumulate actions beyond the seven RESTful actions (index, show, new, edit, create, update and destroy). The more “custom” actions there are, the harder it can be to understand and work with the controller.

Here are three tactics I use to keep my Rails controllers organized.

First, a note about “skinny controllers, fat models”

The concept of “skinny controllers, fat models” is well-known in the Rails community at this point. For the sake of people who are new to Rails, I want to mention that one good way to keep controllers small is to put complex business logic in models rather than controllers. For more on this topic, I might suggest my posts “What is a Rails model?” and (since service objects are a common but I think misguided recommendation) “How I code without service objects“, as well as the original skinny controllers/fat models post by Jamis Buck.

But even if you put as much business logic as possible in models rather than controllers, you’re still left with some challenges regarding overly large controllers. Here are the main three tactics I use to address these challenges.

Tactic 1: extracting a “hidden resource”

Sometimes, when a controller collects too many custom actions, it’s a sign that there’s some “hidden resource” that’s waiting to be identified and abstracted.

For example, in the application that I work on at work, we have an Active Record model called Message which exists so that internal employees who use the application can message each other. At one point we added the concept of a PTO request, which under the hood is really just a Message, but created through a different UI than regular messages.

We could have put these PTO-request-related actions right inside the main MessagesController but that would have made MessagesController too big and muddy. Instead of just being about regular messages, MessagesController would contain some code about regular messages, some code about PTO requests, and some code that relates to both things. So we didn’t want to do that.

Instead, we created a separate controller called PTORequestsController. Even though we decided to have a resource called a PTO request, we didn’t create a separate Active Record model for that. The PTORequestsController just uses the Message model. Here’s what the controller looks like.

module Messaging
  class PTORequestsController < ApplicationController
    def new
      @message = Message.new
    end

    def create
      @message = Message.new(
        message_params.merge(
          sender: current_user,
          recipients: [User.practice_administrator],
          body: "PTO request from #{current_user.first_name} #{current_user.last_name}\n\n#{@message.body}"
        )
      )

      if @message.save
        @message.send_any_email_notifications(@message)
        redirect_to submitted_messaging_pto_requests_path
      else
        render :new
      end
    end

    def submitted
    end

    private

    def message_params
      params.require(:messaging_message).permit(:body)
    end
  end
end

Sometimes, like in this case, the “hidden resource” is pretty obvious from the start and so the original controller can just be left untouched. Other times (and probably more commonly), the original controller slowly grows over time and then a “last straw” moment prompts us to identify a hidden resource and move that resource to a new controller. Sometimes it’s easy to identify a hidden resource and sometimes it’s not.

Tactic 2: same resource, different “lenses”

To review the “hidden resource” example, we had one resource (messages) and then we added a new resource (PTO requests). Making the distinction between messages and PTO requests allowed us to think about the two resources separately and keep their code in separate places. This allowed “regular messages” and PTO requests to be thought about and worked with separately, lowering the overall cognitive cost of the code.

This second tactic applies to a different scenario. Sometimes we don’t want to have a different resource but rather we want to treat the same resource differently. I’ll give another example from the application that I work on at work.

In this application, which is an application for running a medical clinic, we have the concept of an appointment, which can have different meanings in different contexts. For example, in a scheduling context, we care about what time the appointment is for. In a clinical context, we care about the notes the doctor makes regarding the patient’s condition. In a billing context, we care about the charges associated with the appointment.

Early in the application’s life, it was okay for there just to be one single AppointmentsController. But over time AppointmentsController started to get cluttered and harder to understand. So we added a couple new controllers, Billing::AppointmentsController and Chart::AppointmentsController, so that each of these concerns could be dealt with separately. As I’m writing this post, I even realize that it would probably be smart for us to rename AppointmentsController to Schedule::AppointmentsController because almost everything that’s in AppointmentsController is related to scheduling.

Unlike the case of messages and PTO requests, the idea here is not to come up with a new resource but rather to look at the same resource through different lenses. There’s no separate model called Billing::AppointmentsController or Chart::AppointmentsController. The benefit comes from being able to have separate places to deal with separate contexts for the same model.

Tactic 3: dealing with collections separately

This one is a simple one but useful and common enough to be worth mentioning. Sometimes I end up with controllers with actions like bulk_create, bulk_update, etc. in addition to the regular create and update actions. In this case I often create a “collections” controller.

For example, in my application I have a Billing::InsurancePaymentsController and also a Billing::InsurancePaymentCollectionsController. Here’s what the latter controller looks like.

module Billing
  class InsurancePaymentCollectionsController < ApplicationController
    before_action { authorize %i(billing insurance_payment) }

    def create
      @insurance_deposit = InsuranceDeposit.find_by(id: params[:insurance_deposit_id])

      if params[:file].present?
        RemittanceFileParser.new(
          content: params[:file].read,
          insurance_deposit: @insurance_deposit
        ).perform

        redirect_to new_billing_insurance_deposit_insurance_deposit_reconciliation_path(
          insurance_deposit_id: @insurance_deposit.id
        )
      else
        redirect_to request.referer
      end
    end

    def destroy
      InsurancePayment.where(id: params[:ids]).destroy_all
      redirect_to request.referer
    end
  end
end

Takeaways

• Controllers often have a tendency to grow over time. When they do, they usually become hard to understand.
• It’s helpful to put as much business logic as possible in models rather than controllers.
• A large controller can sometimes be made smaller by extracting a “hidden resource” that uses the same Active Record model but clothed in a different idea.
• Another way that large controllers can sometimes be broken up is to think about looking at the same resource through different lenses.
• It can also sometimes be helpful to deal with “bulk actions” in a separate controller.

Overview

Influenced by the experiences I’ve had last over many years of building and maintaining Rails applications, combined with my experiences using other technologies, I’ve developed some ways of structuring Rails applications that have worked out pretty well for me.

Some of my organizational tactics follow conventional wisdom, like keeping controllers thin. Other of my tactics are ones I haven’t really seen in others’ applications but wish I would.

Here’s an overview of the topics I touch on in this post.

• Controllers
• Namespaces
• Models
• ViewComponents
• The lib folder
• Concerns
• Background jobs
• JavaScript
• Tests
• Service objects
• How I think about Rails code organization in general

Let’s start with controllers.

Controllers

The most common type of controller in most Rails applications is a controller that’s based on an Active Record resource. For example, if there’s a customers database table then there will also be a Customer model class and a controller called CustomersController.

The problem

Controllers can start to get nasty when there get to be too many “custom” actions beyond the seven RESTful actions of index, new, create, edit, update, show and destroy.

Let’s say we have a CustomersController that, among other things, allows the user to send messages about a customer (by creating instances of a Message, let’s say). The relevant actions might be called new_message and create_message. This is maybe not that bad, but it clutters up the controller a little, and if you have enough custom actions on a controller then the controller can get pretty messy and hard to comprehend.

The solution

What I like to do in these scenarios is create a “custom” controller called e.g. CustomerMessagesController. There’s no database table called customer_messages or class called CustomerMessage. The concept of a “customer message” is just something I made up. But now that this idea exists, my CustomersController#new_message and CustomersController#create_message actions can become CustomerMessagesController#new and CustomerMessagesController#create. I find this much tidier.

And as long as I’m at it, I’ll even create a PORO (plain old Ruby object) called CustomerMessage where I can handle the business of creating a new customer message as not to clutter up either Customer or Message with this stuff which is really not all that relevant to either of those classes. I might put a create or create! method on CustomerMessage which creates the appropriate Message for me.

Furthermore, I’ll also often put include ActiveModel::Model into my PORO so that I can bind the PORO to a form as though it were a regular old Active Record model.

Namespaces

Pieces of code are easier to understand when they don’t require you to also understand other pieces of code as a prerequisite. To use an extreme example to illustrate the point, it would obviously be impossible to understand a program so tangled with dependencies that understanding any of it required understanding all of it.

So, anything we can do to allow small chunks of our programs understandable in isolation is typically going to make our program easier to work with.

Namespaces serve as a signal that certain parts of the application are more related to each other than they are to anything else. For example, in the application I work on at work, I have a namespace called Billing. I have another namespace called Schedule. A developer who’s new to the codebase could look at the Billing and Schedule namespaces and rightly assume that when they’re thinking about one, they can mostly ignore the other.

Contexts

Some of my models are sufficiently fundamental that it doesn’t make sense to put them into any particular namespace. I have a model called Appointment that’s like this. An Appointment is obviously a scheduling concern a lot of the time, but just as often it’s a clinical concern or a billing concern. An appointment can’t justifiably be “owned” by any one namespace.

This doesn’t mean I can’t still benefit from namespaces though. I have a controller called Billing::AppointmentsController which views appointments through a billing lens. I have another controller called Chart::AppointmentsController which views appointments through a clinical lens. For scheduling, we have two calendar views, one that shows one day at a time and one that shows one month at a time. So I have two controllers for that: Schedule::ByDayCalendar::AppointmentsController and Schedule::ByMonthCalendar::AppointmentsController. Imagine trying to cram all this stuff into a single AppointmentsController. This idea of having namespaced contexts for broad models has been very useful.

Models

I of course keep my models in app/models just like everybody else. What’s maybe a little less common is the way I conceive of models. I don’t just think of models as classes that inherit from ApplicationRecord. To me, a model is anything that models something.

So a lot of the models I keep in app/models are just POROs. According to a count I did while writing this post, I have 115 models in app/models that inherit from ApplicationRecord and 439 that don’t. So that’s about 20% Active Record models and 80% POROs.

ViewComponents

Thanks to the structural devices that Rails provides natively (controllers, models, views, concerns, etc.) combined with the structural devices I’ve imposed myself (namespaces, a custom model structure), I’ve found that most code in my Rails apps can easily be placed in a fitting home.

One exception to this for a long time for me was view-related logic. View-related logic is often too voluminous and detail-oriented to comfortably live in the view, but too tightly coupled with the DOM or other particulars of the view to comfortably live in a model, or anywhere else. The view-related code created a disturbance wherever it lived.

The solution I ultimately settled on for this problem is ViewComponents. In my experience, ViewComponents can provide a tidy way to package up a piece of non-trivial view-related logic in a way that allows me to maintain a consistent level of abstraction in both my views and my models.

The lib folder

I have a rough rule of thumb is that if a piece of code could conceivably be extracted into a gem and used in any application, I put it in lib. Things that end up in lib for me include custom form builders, custom API wrappers, custom generators and very general utility classes.

Concerns

In a post of DHH’s regarding concerns, he says “Concerns are also a helpful way of extracting a slice of model that doesn’t seem part of its essence”. I think that’s a great way to put it and that’s how I use concerns as well.

Like any programming device, concerns can be abused or used poorly. I sometimes come across criticisms of concerns, but to me what’s being criticized is not exactly concerns but bad concerns. If you’re interested in what those criticisms are and how I write concerns, I wrote a post about it here.

Background jobs

I keep my background job workers very thin, just like controllers. It’s my belief that workers shouldn’t do things, they should only call things. Background job workers are a mechanical device, not a code organization device.

JavaScript

I use JavaScript as little as possible. Not because I particularly have anything against JavaScript, but because the less “dynamic” an application is, and the fewer technologies it involves, the easier I find it to understand.

When I do write JavaScript, I use a lot of POJOs (plain old JavaScript objects). I use Stimulus to help keep things organized. To test my JavaScript code, I exercise it using system specs. The way I see it, it’s immaterial from a testing perspective whether I implement my features using JavaScript or Ruby. System specs can exercise it all just fine.

Tests

Being “the Rails testing guy”, I of course write a lot of tests. I use RSpec not because I necessarily think it’s the best testing framework from a technical perspective but rather just to swim with the current. I practice TDD a lot of the time but not all the time. Most of my tests are model specs and system specs.

If you’re curious to learn more about how I do testing, you can read my many Rails testing articles or check out my Rails testing book.

Service objects

Since service objects are apparently so popular these days, I feel compelled to mention that I don’t use service objects. Instead, I use regular old OOP. What a lot of people might model as procedural service object code, I model as declarative objects. I write more about this here.

How I think about Rails code organization in general

I see Rails as an amazingly powerful tool to save me from repetitive work via its conventions. Rails also provides really nice ways of organizing certain aspects of code: controllers, views, ORM, database connections, migrations, and many other things.

At the same time, the benefits that Rails provides have a limit. One you find yourself past that limit (which inevitably does happen if you have a non-trivial application) you either need to provide some structure of your own or you’re likely going to end up with a mess. Specifically, once your model layer grows fairly large, Rails is no longer going to help you very much.

The way I’ve chosen to organize my model code is to use OOP. Object-oriented programming is obviously a huge topic and so I won’t try to convey here what I think OOP is all about. But I think if a Rails developer learns good OOP principles, and applies them to their Rails codebase, specifically in the model layer, then it can go a long way toward keeping a Rails app organized, perhaps more than anything else.

Why to Dockerize your database

Dockerizing helps ease the pain of dependencies

Getting a new developer set up with a Rails app (or any app) can be tedious. Part of the tedium is the chore of manually installing all the dependencies: Ruby, RVM, Rails, PostgreSQL, Redis, etc.

It would be really nice if that developer could just run a single command and have all that app’s dependencies installed on their computer rather than having to install dependencies manually.

This ideal is in fact possible if you fully Dockerize your app. If you want to, you can create a Docker setup that will make it so you don’t have to install anything manually: no Ruby, no Rails, no RVM, no PostgreSQL, no Redis, nothing. It’s very nice.

Fully Dockerizing has drawbacks

Unfortunately, fully Dockerizing a Rails app isn’t without trade-offs. When working with a Dockerized app, there’s a performance hit, there are some issues with using binding.pry, and system specs/system tests in such a way that you can see them run in a browser is next to impossible.

None of these obstacles is insurmountable, but if you don’t want to deal with these issues, you can choose to Dockerize just some of your app’s dependencies instead of all of them.

Partial Dockerization

The Docker setup I use at work is a hybrid approach. I let Docker handle my PostgreSQL and Redis dependencies. I install all my other dependencies manually. This makes it so I don’t have to live with the downsides of full Dockerization but I still get to skip installing some of my dependencies. Any dependency I can skip is a win.

The example I’m going to show you shortly is an even simpler case. Rather than Dockerizing PostgreSQL and Redis, we’re only going to Dockerize PostgreSQL. I’m doing it this way in the interest of showing the simplest possible example.

Dockerizing for development vs. production

I want to add a note for clarity. The Docker setups I’ve been discussing so far are all development setups. There are two ways to Dockerize an app: for a development environment and for a production environment. Development environments and production environments of course have vastly different needs and so a different Docker setup is required for each. In a production environment we wouldn’t run PostgreSQL and a Rails server on the same machine. We’d have a separate database server instead. So I want to be clear that this Docker setup is for development only.

How to Dockerize your database

In order to Dockerize our database we’re going to use Docker Compose. Docker Compose is a tool that a) lets you specify and configure your development environment’s dependencies, b) installs those dependencies for you, and c) runs those dependencies.

Initializing the Rails app

Before we do anything Docker-related, let’s initialize a new Rails app that uses PostgreSQL.

$ rails new my_app -d postgresql

Adding a Docker Compose config file

Here’s the Docker Compose config file. It’s called docker-compose.yml and goes at the project root. This file, again, is what specifies our development environment’s dependencies. I’ve annotated the file to help you understand what’s what.

# docker-compose.yml
---
version: '3.8'

# The "services" directive lists all the services your
# app depends on. In this case there's only one: PostgreSQL.
services:

  # We give each service an arbitrary name. I've called
  # our PostgreSQL service "postgresql".
  postgres:

    # Docker Hub hosts images of common services for
    # people to use. The postgres:13.1-alpine is an
    # image that uses the Alpine Linux distribution,
    # very lightweight Linux distribution that people
    # often use when Dockerizing development environments.
    image: postgres:13.1-alpine

    # PostgreSQL has to put its data somewhere. Here
    # we're saying to put the data in /var/lib/postgresql/data.
    # The "delegated" part specifies the strategy for
    # syncing the container's data with our host machine.
    # (Another option would be "cached".)
    volumes:
      - postgresql:/var/lib/postgresql/data:delegated

    # This says to make our PostgreSQL service available
    # on port 5432.
    ports:
      - "127.0.0.1:5432:5432"

    # This section specifies any environment variables
    # that we want to exist on our Docker container.
    environment:
      # Use "my_app" as our PostgreSQL username.
      POSTGRES_USER: my_app

      # Set POSTGRES_HOST_AUTH_METHOD to "trust" to
      # allow passwordless authentication.
      POSTGRES_HOST_AUTH_METHOD: trust

volumes:
  postgresql:
  storage:

Next we’ll have to change config/database.yml ever so slightly in order to get it to be able to talk to our PostgreSQL container. We need to set the username to my_app and set the host to 127.0.0.1.

default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

  # This must match what POSTGRES_USER was set to in docker-compose.yml.
  username: my_app

  # This must be 127.0.0.1 and not localhost.
  host: 127.0.0.1

development:
  <<: *default
  database: my_app_development

test:
  <<: *default
  database: my_app_test

production:
  <<: *default
  database: my_app_production
  username: my_app
  password: <%= ENV['DB_PASSWORD'] %>

init.sql

If we put a file called init.sql at the project root, Docker will find it and execute it. It’s necessary to have an SQL script that creates a user called my_app or else PostgreSQL will give us an error saying (truthfully) that there’s no user called my_app.

CREATE USER my_app SUPERUSER;

It’s very important that the init.sql file is in place before we proceed. If the init.sql file is not in place or not correct, it can be a difficult error to recover from.

Using the Dockerized database

Run docker-compose up to start the PostgreSQL service.

$ docker-compose up

Now we can create the database.

$ rails db:create

As long as the creation completed successfully, we can connect to the database.

$ rails db

Now we’re connected to a PostgreSQL database without having had to actually install PostgreSQL.

If you’d like to keep your finger on the pulse in the Ruby world, here’s a list of Rubyists you can follow. The list is in alphabetical order by last name.

The list

Bozhidar Batsov
Author of RuboCop.

Rob Bazinet
Rubyist, blogger.

Nate Berkopec
Rails performance expert, author of the Complete Guide to Rails Performance.

Mike Buckbee
Creator of Expedited Security.

Jason Charnes
Co-host of the Remote Ruby podcast.

Ken Collins
Principal Engineer at Custom Ink.

Dave Copeland
Author of Sustainable Web Development
with Ruby on Rails.

Peter Cooper
Creator of Ruby Weekly.

Andy Croll
Organizer of Brighton Ruby.

Andrew Culver
Creator of Bullet Train.

Vladimir Dementyev
Developer at Evil Martians, blogger and conference speaker.

Andrea Fomera
Creator of Learn Hotwire by Building a Forum.

Rafael França
Rails core team member.

Noah Gibbs
Author of Rebuilding Rails.

Justin Gordon
Founder of ShakaCode.

Avdi Grimm
Creator of Ruby Tapas.

Kirk Haines
Principal DevRel at New Relic.

David Heinemeier Hansson
Creator of Ruby on Rails.

Michael Hartl
Creator of The Ruby on Rails Tutorial.

Nate Hopkins
Creator of CableReady and StimulusReflex.

Jemma Issroff
Writes and speaks on Ruby garbage collection, writes weekly tips for Ruby Weekly.

Ross Kaffenberger
Rubyist and blogger.

Ufuk Kayserilioglu
Production Engineering Manager at Shopify.

Dave Kimura
Creator of Drifting Ruby.

Brittany Martin
Host of the 5by5 Ruby on Rails Podcast.

Joe Masilotti
Creator of Mugshot Bot.

Andrew Mason
Co-host of the Remote Ruby podcast.

Yukihiro Matsumoto AKA Matz
Creator of the Ruby language.

Dan Mayer
Rubyist, blogger.

Adam McCrea
Creator of Rails Autoscale.

Sandi Metz
Author of POODR and 99 Bottles of OOP.

Penelope
Directory at Ruby Central, blogger.

Mike Perham
Creator of Sidekiq.

Chad Pytel
COO of thoughtbot.

Charles Nutter
Co-lead of the JRuby project.

Chris Oliver
Creator of Go Rails, Jumpstart, HatchBox and Rails Bytes. Co-host of the Remote Ruby podcast.

Aaron Patterson
Teller of jokes.

Noel Rappin
Author of Modern Front-End Development for Rails and other Rails-related books.

Frank Rietta
CEO of Rietta Inc..

Mike Rogers
I know Mike best for his Docker-Rails template.

Robby Russell
CEO of Planet Argon.

Richard Schneeman
Creator of CodeTriage, a maintainer of Puma.

Colleen Schnettler
Creator of Simple File Upload.

Justin Searls
Co-founder of Test Double.

Prathamesh Sonpatki
Prolific blogger and conference speaker.

Jesse Spevack
Rubyist, conference speaker.

Kelly Sutton
Blogger, conference speaker, engineer at Gusto.

Matt Swanson
Creator of Boring Rails.

Ernesto Tagwerker
Founder of FastRuby.io and OmbuLabs.

Eileen Uchitelle
Principal Engineer at GitHub, Rails core team member, conference speaker.

Brandon Weaver
Rubyist, blogger, conference speaker.

Jared White
Rubyist, blogger.

Samuel Williams
Member of Ruby core team.

Josh Wood
Co-founder of Honeybadger, co-host of FounderQuest.

Jake Yesbeck
Rubyist, blogger.

Ender Ahmet Yurt
Co-Organizer of Ruby Turkey.

Rob Zolkos
Rails developer, prolific help-provider in the Ruby on Rails Slack.

Jason Swett
That would be me.