What exactly are integration tests?
The many different types of RSpec specs
When I started learning about Rails testing and RSpec I discovered that there are many different kinds of tests. I encountered terms like “model spec”, “controller spec”, “view spec”, “request spec”, “route spec”, “feature spec”, and more. I asked myself: Why do these different types of tests exist? Do I need to use all of them? Are some more important than others?
As I’ve gained experience with Rails testing I’ve come to believe that yes, some types of tests (or, to use the RSpec terminology, “specs”) are more important than other types of specs and no, I don’t need to use all of types of specs that RSpec offers. Some types of specs I never use at all (e.g. view specs).
There are two types of tests I use way more than any other types of specs: model specs and feature specs. Let’s focus on these two types of specs for a bit. What are model specs and feature specs for, and what’s the difference?
Feature specs and model specs
Speaking loosely, model specs test ActiveRecord models by themselves and feature specs test the whole “stack” including model code, controller code, and any HTML/CSS/JavaScript together. Neither type of spec is better or worse (and it’s not an “either/or” situation anyway), the two types of specs just have different strengths and weaknesses in different scenarios.
The strengths and weaknesses of model specs
An advantage of model specs is that they’re “inexpensive”. Compared to feature specs, model specs are relatively fast to run and relatively fast to write. It tends to be slower to actually load the pages of an application and switch among them than to just test model code by itself with no browser involved. That’s an advantage of model specs. A disadvantage of model specs is that they don’t tell you anything about whether your whole application actually works from the user’s perspective. You can understand what I mean if you imagine a Rails model that works perfectly by itself but doesn’t have have any working HTML/CSS/JavaScript plugged into that model to make the whole application work. In such a case the model specs could pass even though there’s no working feature built on top of the model code.
The strengths and weaknesses of feature specs
Feature specs have basically the opposite pros and cons. Unlike model specs, feature specs do tell you whether all the parts of your application are working together. There’s a cost to this, though, in that feature specs are relatively “expensive”. Feature specs are relatively time-consuming to run because you’re running more stuff than in a model spec. (It’s slower to have to bring up a web browser than to not have to.) Feature specs are also relatively time-consuming to write. The reason feature specs are time-consuming to write is that, unlike a model spec where you can exercise e.g. just one method at a time, a feature spec has to exercise a whole feature at a time. In order to test a feature it’s often necessary to have certain non-trivial conditions set up—if you want to test, for example, the creation of a hair salon appointment you first have to have a salon, a stylist, a client, and maybe more. This necessity makes the feature spec slower to write and slower to run than a model spec where you could test e.g. a method on a `Stylist` class all by itself.
Where integration tests come into the picture, and what integration tests are
If you’re okay with being imprecise, we can say that feature specs and integration tests are roughly the same thing. And I hope you’re okay with being imprecise because when it comes to testing terminology in general there’s very little consensus on what the various testing terms mean, so it’s kind of impossible to be precise all the time. We do have room to be a little more precise in this particular case, though.
In my experience it’s commonly agreed that an integration test is any test that tests two or more parts of an application together. What do we mean when we say “part”? A “part” could be pretty much anything. For example, it has been validly argued that models specs could be considered integration tests because model specs typically test Ruby code and database interaction, not just Ruby code in isolation with no database interaction. But it could also be validly argued that a model spec is not an integration test because a model spec just tests “one” thing, a model, without bringing controllers or HTML pages into the picture. (For our purposes though let’s say model specs are NOT integration tests, which is the basic view held, implicitly or explicitly, by most Rails developers.)
So, while general agreement exists on what an integration test is in a broad sense, there still is a little bit of room for interpretation once you get into the details. It’s kind of like the definition of a sandwich. Most people agree that a sandwich is composed of a piece of food surrounded by two pieces of bread, but there’s not complete consensus on whether e.g. a hamburger is a sandwich. Two different people could have different opinions on the matter and no one could say either is wrong because there’s not a single authoritative definition of the term.
Since feature specs exercise all the layers of an application stack, feature specs fall solidly and uncontroversially into the category of integration tests. This is so true that many developers (including myself) speak loosely and use the terms “feature spec” and “integration test” interchangeably, even though we’re being a little inaccurate by doing so. Inaccurate because all feature specs are integration specs but not all integration tests are feature specs. (For example, a test that exercises an ActiveRecord model interacting with the Stripe API could be considered an integration test even though it’s not a feature spec.)
I hope at this point you have some level of understanding of how feature specs and integration tests are different and where they overlap. Now that we’ve discussed feature specs vs. integration tests, let’s bring some other common testing terms into the picture. What about end-to-end tests and system tests?
What’s the difference between integration tests, acceptance tests, end-to-end tests, system tests, and feature specs?
A useful first step in discussing these four terms may be to name the origin of each term. Is it a Ruby/Rails-specific term or just a general testing term that we happen to use in the Rails world sometimes?
| Integration tests |
General testing term, not Ruby/Rails-specific |
| Acceptance tests |
General testing term, not Ruby/Rails-specific |
| End-to-end tests |
General testing term, not Ruby/Rails-specific |
| System tests |
Rails-specific, discussed in the official Rails guides |
| Feature specs |
An RSpec concept/term |
Integration tests, acceptance tests and end-to-end tests
Before we talk about where system tests and feature specs fit in, let’s discuss integration tests, acceptance tests and end-to-end tests.
Like most testing terms, I’ve heard the terms integration tests, acceptance tests, and end-to-end tests used by different people to mean different things. It’s entirely possible that you could talk to three people and walk away thinking that integration tests, acceptance tests and end-to-end tests mean three different things. It’s also entirely possible that you could talk to three people and walk away thinking all three terms mean the same exact thing.
I’ll tell you how I interpret and use each of these three terms, starting with end-to-end tests.
To me, an end-to-end test is a test that tests all layers of an application stack under conditions that are very similar to production. So in a Rails application, a test would have to exercise the HTML/CSS/JavaScript, the controllers, the models, and the database in order to qualify as an end-to-end test. To contrast end-to-end tests with integration tests, all end-to-end tests are integration tests but not all integration tests are end-to-end tests. I would say that the only difference between the terms “end-to-end test” and “feature spec” are that feature spec is an RSpec-specific term while end-to-end test is a technology-agnostic industry term. I don’t tend to hear the term “end-to-end test” in Rails circles.
Like I said earlier, an integration test is often defined as a test that verifies that two or more parts of an application behave correctly not only in isolation but also when used together. For practical purposes, I usually hear Rails developers say “integration test” when they’re referring to RSpec feature specs.
The purpose of an acceptance tests is quite a bit different (in my definition at least) from end-to-end tests or integration tests. The purpose of an acceptance test is to answer the question, “Does the implementation of this feature match the requirements of the feature?” Like end-to-end tests, I don’t ever hear Rails developers refer to feature specs as acceptance tests. I have come across this usage, though. One of my favorite testing books, Growing Object-Oriented Software, Guided by Tests, uses the term “acceptance test” to refer to what in RSpec/Rails would be a feature spec. So again, there’s a huge lack of consensus in the industry around testing terminology.
System tests and feature specs
Unlike many distinctions we’ve discussed so far, the difference between system tests and feature specs is happily pretty straightforward: when an RSpec user says integration test, they mean feature spec; when a MiniTest user says integration test, they mean system test. If you use RSpec you can focus on feature specs and ignore system tests. If you use MiniTest you can focus on system tests and ignore feature specs.
My usage of “integration test”
For the purposes of this article I’m going to swim with the current and use the terms “integration test” and “feature spec” synonymously from this point forward. When I say “integration test”, I mean feature spec.
What do I write integration tests for, and how?
We’ve discussed what integration tests are and aren’t. We’ve also touched on some related testing terms. If you’re like many developers getting started with testing, your next question might be: what do I write integration tests for, and how?
This question (what do I write tests for) is probably the most common question I hear from people who are new to testing. It was one of my own main points of confusion when I myself was getting started with testing.
What to write integration tests for
I can tell you what I personally write integration tests for in Rails: just about everything. Virtually every time I build a feature that a user will interact with in a browser, I’m going to write at least one integration test for that feature. Unfortunately this answer of mine, while true, is perhaps not very helpful. When someone asks what they should write tests for, that person is probably somewhat lost and that person is probably looking for some sort of toehold. So here’s what’s hopefully a more helpful answer.
If you’ve never written any sort of integration test (which again I’m sloppily using interchangeably with “feature spec”) then my advice would be to first do an integration test “hello world” just so you can see what’s what and get a tiny bit of practice under your belt. I have another post, titled A Rails testing “hello world” using RSpec and Capybara, that will help you do just that.
What if you’re a little further? What if you’re already comfortable with a “hello world” level of integration testing and you’re more curious about how to add real integration tests to your Rails application?
My advice would be to start with what’s easiest. If you’re working with a legacy project (legacy project here could just be read as “any existing project without a lot of test coverage”), it’s often the case that the structure of the code makes it particularly difficult to add tests. So in these cases you’re dealing with two separate challenges at once: a) you’re trying to learn testing (not easy) and b) you’re trying to overcome the obstacles peculiar to adding tests to legacy code (also not easy). To the extent that you can, I would encourage you to try to separate these two endeavors and just focus on getting practice writing integration tests first.
If you’re trying to add tests to an existing project, maybe you can find some areas of the application where adding integration tests is relatively easy. If there’s an area of your application where the UI simply provides CRUD operations on some sort of resource that has few or no dependencies, then that area might be a good candidate to begin with. A good clue would be to look for how many `has_many`/`belongs_to` calls you find in your various models (in other words, look for how many associations your models have). If you have a model that has a lot of associations, the CRUD interface for that model is probably not going to be all that easy to test because you’re going to have to spin up a lot of associated records in order to get the feature to function. If you can find a model with fewer dependencies, a good first integration test to write might be a test for updating a record.
How to write integration tests
Virtually all the integration tests I write follow the same pattern:
- Generate some test data
- Log into the application
- Visit the page I’m interested in
- Perform whatever clicks and typing need to happen in order to exercise the feature I’m testing
- Perform an assertion
If you can follow this basic template, you can write integration tests for almost anything. The hardest part is usually the step of generating test data that will help get your program/feature into the right state for testing what you want to test.
Most of the Rails features I write most of the time are pretty much just some variation of a CRUD interface for a Rails resource. So when I’m developing a feature, I’ll usually write an integration test for at least creating and updating the resource, and maybe deleting an instance of that resource. (I’ll pretty much never write a test for simply viewing a list of resources, the “R” in CRUD, because that functionality is usually exercised anyway in the course of testing other behavior.) Sometimes I write my integration tests before I write the application code that makes the tests pass. Usually I write generate the CRUD code first using Rails scaffolds and add the tests afterward. To put it another way, I usually don’t write my integration tests in a TDD fashion. In fact, it’s not really possible to use scaffolds and TDD at the same time. In the choice between having the benefits of scaffolds and having the benefits of TDD, I choose having the benefits of scaffolds. (I do, however, practice TDD when writing other types of tests besides integration tests.)
Now that we’ve discussed in English what to write integration tests for and how, let’s continue fleshing out this answer using some actual code examples. The remainder of this article is a Rails integration test tutorial.
Tutorial overview
Project description
In this short tutorial we’re going to create a small Rails application which is covered by a handful of integration tests. The application will be ludicrously small as far as Rails applications go, but a ludicrously small Rails application is all we’re going to need.
Our application will have just a single model, City, which has just a single attribute, name. Since the application has to be called something, we’ll call it “Metropolis”.
Tutorial outline
Here are the steps we’re going to take:
- Initialize the application
- Create the `city` resource
- Write some integration tests for the `city` resource
That’s all! Let’s dive in. The first step will be to set up our Rails project.
Setting up our Rails project
First let’s initialize the application. The `-T` flag means “no test framework”. We need the `-T` flag because, if it’s not there, Rails will assume we want MiniTest, which in this case is not the case. I’m using the `-d postgresql` flag because I like PostgreSQL, but you can use whatever RDBMS you want.
$ rails new metropolis -T -d postgresql
Next let’s `cd` into the project directory and create the database.
$ cd metropolis
$ rails db:create
With that groundwork out of the way we can add our first resource and start adding our first integration tests.
Writing our integration tests
What we’re going to do in this section is generate the city scaffold, then pull up the city index page in the browser, then write some tests for the city resource.
The tests we’re going to write for the city resource are:
- Creating a city (with valid inputs)
- Creating a city (with invalid inputs)
- Updating a city (with valid inputs)
- Deleting a city
Creating the city resource
The City resource will have only one attribute, name.
$ rails g scaffold city name:string
$ rails db:migrate
Let’s set our root route to cities#index so Rails has something to show when we visit localhost:3000.
Rails.application.routes.draw do
resources :cities
root 'cities#index'
end
If we now run the rails server command and visit http://localhost:3000, we should see the CRUD interface for the City resource.
$ rails server
Integration tests for City
Before we can write our tests we need to install some certain gems.
Installing the necessary gems
Let’s add the following to our Gemfile under the :development, :test group.
group :development, :test do
# The RSpec testing framework
gem 'rspec-rails'
# Capybara, the library that allows us to interact with the browser using Ruby
gem 'capybara'
# This gem helps Capybara interact with the web browser.
gem 'webdrivers'
end
Remember to bundle install.
$ bundle install
In addition to running bundle install which will install the above gems, we also need to install RSpec into our Rails application which is a separate step. The rails g rspec:install command will add a couple RSpec configuration files into our project.
$ rails g rspec:install
The last bit of plumbing work we have to do before we can start writing integration tests is to create a directory where we can put the integration tests. I tend to create a directory called spec/features and this is what I’ve seen others do as well. There’s nothing special about the directory name features. It could be called anything at all and still work.
$ mkdir spec/features
Writing the first integration test (creating a city)
The first integration test we’ll write will be a test for creating a city. The steps will be this:
- Visit the “new city” page
- Fill in the Name field with a city name (Minneapolis)
- Click the Create City button
- Visit the city index page
- Assert that the city we just added, Minneapolis, appears on the page
Here are those four steps translated into code.
visit new_city_pathfill_in 'Name', with: 'Minneapolis'click_on 'Create City'visit cities_pathexpect(page).to have_content('Minneapolis')
Finally, here are the contents of a file called spec/features/create_city_spec.rb with the full working test code.
# spec/features/create_city_spec.rb
require 'rails_helper'
RSpec.describe 'Creating a city', type: :feature do
scenario 'valid inputs' do
visit new_city_path
fill_in 'Name', with: 'Minneapolis'
click_on 'Create City'
visit cities_path
expect(page).to have_content('Minneapolis')
end
end
Let’s create the above file and then run the test.
rspec spec/features/create_city_spec.rb
The test passes. There’s kind of a problem, though. How can we be sure that the test is actually doing its job? What if we accidentally wrote the test in such a way that it always passes, even if the underlying feature doesn’t work? Accidental “false positives” certainly do happen. It’s a good idea to make sure we don’t fool ourselves.
Verifying that the test actually does its job
How can we verify that a test doesn’t give a false positive? By breaking the feature and verifying that the test no longer passes.
There are two ways to achieve this:
- Write the failing test before we write the feature itself (test-driven development)
- Write the feature, write the test, then break the feature
Method #1 is not an option for us in this case because we already created the feature using scaffolding. That’s fine though. It’s easy enough to make a small change that breaks the feature.
In our CitiesController, let’s replace the line if @city.save with simply if true. This way the flow through the application will continue as if everything worked, but the city record we’re trying to create won’t actually get created, and so the test should fail when it looks for the new city on the page.
# app/controllers/cities_controller.rb
def create
@city = City.new(city_params)
respond_to do |format|
#if @city.save
if true
format.html { redirect_to @city, notice: 'City was successfully created.' }
format.json { render :show, status: :created, location: @city }
else
format.html { render :new }
format.json { render json: @city.errors, status: :unprocessable_entity }
end
end
end
If we run the test again now, it does in fact fail.
F
Failures:
1) Creating a city valid inputs
Failure/Error: expect(page).to have_content('Minneapolis')
expected to find text "Minneapolis" in "Cities\nName\nNew City"
# ./spec/features/create_city_spec.rb:9:in `block (2 levels) in <top (required)>'
Finished in 0.23035 seconds (files took 0.89359 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/features/create_city_spec.rb:4 # Creating a city valid inputs
Now we can change if true back to if @city.save, knowing that our test really does protect against a regression should this city saving functionality ever break.
We’ve just added a test for attempting (successfully) to create a city when all inputs are valid. Now let’s add a test that verifies that we get the desired behavior when not all inputs are valid.
Integration test for trying to create a city with invalid inputs
In our “valid inputs” case we followed the following steps.
- Visit the “new city” page
- Fill in the Name field with a city name (Minneapolis)
- Click the Create City button
- Visit the city index page
- Assert that the city we just added, Minneapolis, appears on the page
For the invalid case we’ll follow a slightly different set of steps.
- Visit the “new city” page
- Leave the Name field blank
- Click the Create City button
- Assert that the page contains an error
Here’s what these steps might look like when translated into code.
visit new_city_pathfill_in 'Name', with: ''click_on 'Create City'expect(page).to have_content("Name can't be blank")
A comment about the above steps: step 2 is actually not really necessary. The Name field is blank to begin with. Explicitly setting the Name field to an empty string is superfluous and doesn’t make a bit of change in how the test actually works. However, I’m including this step just to make it blatantly obvious that we’re submitting a form with an empty Name field. If we were to jump straight from visiting new_city_path to clicking the Create City button, it would probably be less clear what this test is all about.
Here’s the full version of the “invalid inputs” test scenario alongside our original “valid inputs” scenario.
# spec/features/create_city_spec.rb
require 'rails_helper'
RSpec.describe 'Creating a city', type: :feature do
scenario 'valid inputs' do
visit new_city_path
fill_in 'Name', with: 'Minneapolis'
click_on 'Create City'
visit cities_path
expect(page).to have_content('Minneapolis')
end
scenario 'invalid inputs' do
visit new_city_path
fill_in 'Name', with: ''
click_on 'Create City'
expect(page).to have_content("Name can't be blank")
end
end
Let’s see what we get when we run this test.
$ rspec spec/features/create_city_spec.rb
The test fails.
.F
Failures:
1) Creating a city invalid inputs
Failure/Error: expect(page).to have_content('error')
expected to find text "Name can't be blank" in "City was successfully created.\nName:\nEdit | Back"
# ./spec/features/create_city_spec.rb:16:in `block (2 levels) in <top (required)>'
Finished in 0.19976 seconds (files took 0.96104 seconds to load)
2 examples, 1 failure
Failed examples:
rspec ./spec/features/create_city_spec.rb:12 # Creating a city invalid inputs
Instead of finding the text Name can't be blank on the page, it found the text City was successfully created.. Evidently, Rails happily accepted our blank Name input and created a city with an empty string for a name.
To fix this behavior we can add a presence validator to the name attribute on the City model.
# app/models/city.rb
class City < ApplicationRecord
validates :name, presence: true
end
The test now passes.
Integration test for updating a city
The steps for this test will be:
- Create a city in the database
- Visit the “edit” page for that city
- Fill in the Name field with a different value from what it currently is
- Click the Update City button
- Visit the city index page
- Assert that the page contains the city’s new name
Here are these steps translated into code.
nyc = City.create!(name: 'NYC')visit edit_city_path(id: nyc.id)fill_in 'Name', with: 'New York City'click_on 'Update City'visit cities_pathexpect(page).to have_content('New York City')
Here’s the full test file which we can put at spec/features/update_city_spec.rb.
# spec/features/update_city_spec.rb
require 'rails_helper'
RSpec.describe 'Updating a city', type: :feature do
scenario 'valid inputs' do
nyc = City.create!(name: 'NYC')
visit edit_city_path(id: nyc.id)
fill_in 'Name', with: 'New York City'
click_on 'Update City'
visit cities_path
expect(page).to have_content('New York City')
end
end
If we run this test (rspec spec/features/update_city_spec.rb), it will pass. Like before, though, we don’t want to trust this test without seeing it fail once. Otherwise, again, we can’t be sure that the test isn’t giving us a false positive.
Let’s change the line if @city.update(city_params) in CitiesController to if true so that the controller continues on without actually updating the city record. This should make the test fail.
# app/controllers/cities_controller.rb
def update
respond_to do |format|
#if @city.update(city_params)
if true
format.html { redirect_to @city, notice: 'City was successfully updated.' }
format.json { render :show, status: :ok, location: @city }
else
format.html { render :edit }
format.json { render json: @city.errors, status: :unprocessable_entity }
end
end
end
The test does in fact now fail.
F
Failures:
1) Updating a city valid inputs
Failure/Error: expect(page).to have_content('New York City')
expected to find text "New York City" in "Cities\nName NYC Show Edit Destroy\nNew City"
# ./spec/features/update_city_spec.rb:10:in `block (2 levels) in <top (required)>'
Finished in 0.17966 seconds (files took 0.89554 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/features/update_city_spec.rb:4 # Updating a city valid inputs
Integration test for deleting a city
This will be the last integration test we write. Here are the steps we’ll follow.
- Create a city in the database
- Visit the city index page
- Assert that the page contains the name of our city
- Click the “Destroy” link
- Accept the “Are you sure?” alert
- Assert that the page no longer contains the name of our city
Translated into code:
City.create!(name: 'NYC')visit cities_pathexpect(page).to have_content('NYC')click_on 'Destroy'accept_alertexpect(page).not_to have_content('NYC')
Here’s the full test file, spec/features/delete_city_spec.rb.
# spec/features/delete_city_spec.rb
require 'rails_helper'
RSpec.describe 'Deleting a city', type: :feature do
scenario 'success' do
City.create!(name: 'NYC')
visit cities_path
expect(page).to have_content('NYC')
click_on 'Destroy'
accept_alert
expect(page).not_to have_content('NYC')
end
end
If we run this test, it will pass.
$ rspec spec/features/delete_city_spec.rb
To protect against a false positive and see the test fail, we can comment out the line @city.destroy in CitiesController.
# app/controllers/cities_controller.rb
def destroy
#@city.destroy
respond_to do |format|
format.html { redirect_to cities_url, notice: 'City was successfully destroyed.' }
format.json { head :no_content }
end
end
Now the test fails.
F
Failures:
1) Deleting a city success
Failure/Error: expect(page).not_to have_content('NYC')
expected not to find text "NYC" in "City was successfully destroyed.\nCities\nName NYC Show Edit Destroy\nNew City"
# ./spec/features/delete_city_spec.rb:9:in `block (2 levels) in <top (required)>'
Finished in 0.18866 seconds (files took 0.88234 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/features/delete_city_spec.rb:4 # Deleting a city success
Remember to change that line back to the test passes again.
At this point we’ve written four test cases:
- Creating a city (with valid inputs)
- Creating a city (with invalid inputs)
- Updating a city (with valid inputs)
- Deleting a city
Let’s invoke the rspec command to run all four test cases. Everything should pass.
$ rspec
....
Finished in 0.23001 seconds (files took 0.88377 seconds to load)
4 examples, 0 failures
Where to go next
If you want to learn more about writing integration tests in Rails, here are a few recommendations.
First, I recommend good old practice and repetition. If you want to get better at writing integration tests, write a whole bunch of integration tests. I would suggest building a side project of non-trivial size that you maintain over a period of months. Try to write integration tests for all the features in the app. If there’s a feature you can’t figure out how to write a test for, give yourself permission to skip that feature and come back to it later. Alternatively, since it’s just a side project without the time pressures of production work, give yourself permission to spend as much time as you need in order to get a test working for that feature.
Second, I’ll recommend a couple books.
Growing Object-Oriented Software, Guided by Tests. This is one of the first books I read when I was personally getting started with testing. I recommend it often and hear it recommended often by others. It’s not a Ruby book but it’s still highly applicable.
The RSpec Book. If you’re going to be writing tests with RSpec, it seems like a pretty good idea to read the RSpec book. I also did a podcast interview with one of the authors where he and I talk about testing. I’m recommending this book despite the fact that it advocates Cucumber, which I am pretty strongly against.
Effective Testing with RSpec 3. I have not yet picked up this book although I did do a podcast interview with one of the authors.
My last book recommendation is my own book, Rails Testing for Beginners. I think my book offers a pretty good mix of model-level testing and integration-level testing with RSpec and Capybara.