The difference between integration tests and controller tests in Rails

by Jason Swett,

I recently came across a Reddit post asking about the difference between integration tests and controller tests. Some of the comments were interesting:

“I always write controller tests. I only write integration tests if there’s some JavaScript interacting with the controller or if the page is really fucking valuable.”

“In this case I would say it depends if you are building a full rails app (front-end and back-end) or only and API (back-end). For the first I would say an integration test should go through the front-end (which eventually calls the controllers). If you are doing an API, integration and controller tests would be the same.”

“Controller tests attempt to test the controller in isolation, and integration tests mimic a browser clicking through the app, i.e. they touch the entire stack. Honestly I only write integration tests. Controller tests are basically exactly the same thing but worse, and they are harder to write. “

“As far as I understand for the direction of Rails, controller tests will be going away and you’ll need to use integration tests.”

“From my experience integration tests are always harder to maintain and are much more fragile. Not trying to neglect it’s value but it comes with a price. I wonder how will people going to test api only rails apps if controller tests are gone?”

Which of these things are accurate and which are BS? I’ll do my best here to clarify. I’ll also add my own explanation of the difference between integration tests and controller tests.

My explanation of integration tests vs. controller tests

Before I even share my explanation I need to provide some context. There are two realities that make questions like “What’s the difference between integration tests and controller tests?” hard to answer.

Problem: terminology

First, there’s no consensus in the testing world on terminology. What one person calls an integration test might be what another person would call an end-to-end test or an acceptance test. No one can say whether any particular definition of a term is right or wrong because there’s no agreed-upon standard.

Problem: framework differences

Second, in what context are we talking about integration tests vs. controller tests? RSpec? MiniTest? Something else? The concepts of integration tests and controller tests map slightly differently onto each framework. Here’s how I’d put it:

General Testing Concept Relevant MiniTest Concept Relevant RSpec Concept
Integration Test Integration Test Feature Spec
End-to-End Test System Test Feature Spec
Controller Test Functional Test Request Spec (and, previously, Controller Spec)

Yikes. In order to talk about the difference between integration tests and controller tests I needed to involve no fewer than eight different terms: integration test, end-to-end test, system test, feature spec, controller test, functional test, request spec and controller spec.

I could share a treatment of integration and controller tests that’s very precise and technically correct but first let me try, for clarity’s sake, to share a useful approximation to the truth.

My approximately-true explanation

If a Rails application can be thought of a stack listed top-to-bottom as views-controllers-models, controller tests exercise controllers and everything “lower”. So controller tests test the “controllers-models” part of the “views-controllers-models” stack.

Integration tests test the views and everything “lower”. So in the views-controllers-models stack, integration tests test all three layers.

The messier truth

The messier truth is that since RSpec is (as far as I can tell) substantially more popular than MiniTest for commercial Rails applications, then you, dear reader, are more likely to be an RSpec user than a MiniTest user.

So instead of “integration tests”, the term that’s applicable to you is feature specs. Luckily the definition is pretty much the same, though. Feature specs exercise the whole application stack (views-controllers-models).

When we start to talk about controller tests (or controller specs in RSpec terminology), things get a little confusing. In addition to the concept of controller specs, there now exists the concept of request specs. What’s the difference between the two? As far as I can tell, the main difference is that request specs run in a closer simulation to a real application environment than controller specs. Therefore, the RSpec core team recommends using request specs over controller specs. So the main thing to be aware of is that when someone says “controller tests”, the RSpec concept to map that to is “request specs”.

To sum up: in RSpec, it’s not integration tests vs. controller tests, it’s feature specs vs. request specs. (For the record, I’m not a big fan of RSpec’s somewhat arcane terminology.)

When to use integration tests vs. controller tests

Now that we’ve covered what these two things are, how do we know when to use them?

When I use controller tests/request specs

I’ll start with controller tests (or again in RSpec terminology, request specs). As I discussed in detail in a different article, I only use controller/request specs in two specific scenarios: 1) when I’m maintaining a legacy project and 2) when I’m maintaining an API-only application.

Why are these the only two scenarios in which I use request specs? Because when I’m doing greenfield development, I try very hard to make my controllers do almost nothing by themselves. I push all possible behavior into models. Legacy projects, however, often possess bloated controllers containing lots of code. I find tests useful in teasing apart those bloated controllers so I can refactor the code to mostly use models instead. The reason I use request specs when developing API-only applications is because integration tests/feature specs just aren’t possible. There’s no browser with which to interact.

When I use integration tests/feature specs

Practically every feature I write gets an integration test. For example, when I’m building CRUD functionality for a resource (let’s say a resource called `Customer`), I’ll write a feature spec for attempting to create a new customer record (using both valid and invalid inputs, checking for either success or failure), a feature spec for attempting to update a customer record, and perhaps a feature spec for deleting a customer record. Since feature specs exercise the whole application stack including controllers, I pretty much always find redundant the idea of writing both a feature spec and a request spec for a particular feature.

Addressing the comments

Finally I want to address some of the comments I saw on the Reddit post I referenced at the beginning of this article because I don’t think all of them are accurate.

Page value

“I always write controller tests. I only write integration tests if there’s some JavaScript interacting with the controller or if the page is really fucking valuable.”

I personally don’t buy this idea. To me, one of the main benefits of having integration test/feature spec coverage is that I can automatically check the whole application for regressions every time I make a new commit. I hate it when a client of mine has to point out an error to me, even if it’s something trivial. I’d much rather have my code get tested by automated tests than by my client. (And yes, I get that tests can’t prove the absence of bugs and that my test suite won’t always catch all regressions, but I think something is a lot better than nothing.)

API-only applications

“In this case I would say it depends if you are building a full rails app (front-end and back-end) or only and API (back-end). For the first I would say an integration test should go through the front-end (which eventually calls the controllers). If you are doing an API, integration and controller tests would be the same.”

Let me address the last sentence first. Yes, if you’re writing an API-only application, the max you can test is “from the controller down”, so the idea of adding an integration test that tests an additional layer doesn’t apply.

Now let me address the first part of the comment. The idea behind the comment makes sense to me. I think what the comment author is saying is that if the application is a “traditional” Rails application, then an integration test would hit the front-end, the user-facing part of the application, and exercise all parts of the code from that starting point.

“The same thing but worse”

“Controller tests attempt to test the controller in isolation, and integration tests mimic a browser clicking through the app, i.e. they touch the entire stack. Honestly I only write integration tests. Controller tests are basically exactly the same thing but worse, and they are harder to write”

I believe I basically agree with this comment but I’d like to get a little more specific. I wouldn’t exactly agree that “controller tests are basically the exact same thing” as integration tests. As I said above, integration tests/feature specs test one additional layer of the application beyond controllers. There’s a lot in that extra layer. It makes a big difference.

As for the second part, “Controller tests are basically the same thing but worse,” it would be helpful to say why they’re “worse”. I would repeat what I said earlier in that controller tests/requests specs for me are usually redundant to any integration tests/feature specs I might have. The exceptions, again, are legacy projects and API-only applications.

Controller tests going away?

“As far as I understand for the direction of Rails, controller tests will be going away and you’ll need to use integration tests.”

Unless I’m mistaken I believe this comment is a little confused. As I mentioned earlier in this article, it’s true that the RSpec core team no longer recommends using controller specs. The recommended replacement though isn’t integration tests but request specs. However, I personally tend to favor integration tests/feature specs over controller specs/request specs anyway.

I’m not able to find any evidence that MiniTest controller tests are going away.

Integration tests fragile and harder to maintain?

From my experience integration tests are always harder to maintain and are much more fragile. Not trying to neglect it’s value but it comes with a price. I wonder how will people going to test api only rails apps if controller tests are gone?

I wouldn’t necessarily disagree. Out of the test types I use, I find integration tests/feature specs to be the most expensive to maintain. I would, however, suggest that the additional value integration tests/feature specs provide over controller tests/request specs more than makes up for their extra cost.

2 thoughts on “The difference between integration tests and controller tests in Rails

  1. Joe Horsnell

    Thanks for the article – you make some valid points about the confusion caused by the different terminology used in Rails vs RSpec Rails.

    A couple of suggestions/corrections however for your summary table in “FRAMEWORK DIFFERENCES”:
    1. System tests in Rails (aka Feature specs in RSpec Rails) are the highest level tests, since they drive the entire application through its external interface using a real/headless browser. I therefore suggest you put them first in the table, followed by Integration tests, then finally Functional tests, which are the lowest level (in terms of testing isolation) of these three types of test
    2. You have Request specs in the wrong place – they are (and always have been) a thin wrapper around Rails Integration tests (https://relishapp.com/rspec/rspec-rails/v/3-8/docs/request-specs/request-spec)
    3. Controller specs in RSpec Rails are still (and always have been) the equivalent of Rails Functional tests (https://relishapp.com/rspec/rspec-rails/v/3-8/docs/controller-specs) so nothing has changed there.

    The only thing that has really changed ‘recently’ (where recently is since Rails 5, so two and a half years ago) is that in Rails 5, two things were deprecated around Functional tests (ie RSpec Rails controller tests), which was discussed with some community disagreement here https://github.com/rails/rails/issues/18950:

    1. The ability to assert what ivars (instance variables) have been assigned in a controller (for access in the view template) using `assigns` (see how the Four became the Three hashes of the Apocalypse from https://guides.rubyonrails.org/v4.2/testing.html#the-four-hashes-of-the-apocalypse to https://guides.rubyonrails.org/testing.html#the-three-hashes-of-the-apocalypse)
    2. The ability to assert which template the controller action rendered, using `assert_template`

    Both of these are are available for backwards compatibility through another gem for existing apps, but for new Rails apps the official Rails advice is to use Integration tests (aka RSpec Rails Request specs) instead, since unlike Functional tests, they exercise the full application stack, including routes, Rack middleware, etc. and are (apparently) faster on Rails 5 than Functional tests were on Rails 4.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *