The four phases of a test

by Jason Swett,

When writing tests, or reading other people’s tests, it can be helpful to understand that tests are often structured in four distinct phases.

These phases are:

  1. Setup
  2. Exercise
  3. Assertion
  4. Teardown

Let’s illustrate these four phases using an example.

Test phase example

Let’s say we have an application that has a list of users that can receive messages. Only active users are allowed to receive messages. So, we need to assert that when a user is inactive, that user can’t receive messages.

Here’s how this test might go:

  1. Create a User record (setup)
  2. Set the user’s “active” status to false (exercise)
  3. Assert that the user is not “messageable” (assertion)
  4. Delete the User record we created in step 1 (teardown)

In parallel with this example, I’ll also use another example which is somewhat silly but also less abstract. Let’s imagine we’re designing a sharp-shooting robot that can fire a bow and accurately hit a target with an arrow. In order to test our robot’s design, we might:

  1. Get a fresh prototype of the robot from the machine shop (setup)
  2. Allow the robot to fire an arrow (exercise)
  3. Look at the target to make sure it was hit by the arrow (assertion)
  4. Return the prototype to the machine shop for disassembly (teardown)

Now let’s take a look at each step in more detail.

The purpose of each test phase

Setup

The setup phase typically creates all the data that’s needed in order for the test to operate. (There are other things that could conceivably happen during a setup phase but for our current purposes we can think of the setup phase’s role as being to put data in place.)In our case, the creation of the User record is all that’s involved in the setup step, although more complicated tests could of course create any number of database records and potentially establish relationships among them.

Exercise

The exercise phase walks through the motions of the feature we want to test. With our robot example, the exercise phase is when the robot fires the arrow. With our messaging example, the exercise phase is when the user gets put in an inactive state.

Side note: the distinction between setup and exercise may seem blurry, and indeed it sometimes is, especially in low-level tests like our current example. If someone were to argue that setting the user to inactive should actually be part of the setup, I’m not sure how I’d refute them. To help with the distinction in this case, imagine if we instead were writing an integration test that actually opened up a browser and simulated clicks. For this test, our setup would be the same (create a user record) but our exercise might be different. We might visit a settings page, uncheck an “active” checkbox, then save the form.

Assertion

The assertion phase is basically what all the other phases exist in support of. The assertion is the actual test part of the test, the thing that determines whether the test passes or fails.

Teardown

Each test needs to clean up after itself. If it didn’t, then each test would potentially pollute the world in which the test is running and affect the outcome of later tests, making the tests non-deterministic. We don’t want this. We want deterministic tests, i.e. tests that behave the same exact way every single time no matter what. The only thing that should make a test go from passing to failing or vice-versa is if the behavior that the test tests changes.

In reality, Rails tests tend not to have an explicit teardown step. The main pollutant we have to worry about with our tests is database data that gets left behind. RSpec is capable of taking care of this problem for us by running each test in a database transaction. The transaction starts before each test is run and aborts after the test finishes. So really, the data never gets permanently persisted in the first place. So although I’m mentioning the teardown step here for completeness’ sake, you’re unlikely to see it in the wild.

A concrete example

See if you can identify the phases in the following RSpec test.

RSpec.describe User do
  let!(:user) { User.create!(email: 'test@example.com') }

  describe '#messageable?' do
    context 'is inactive' do
      it 'is false' do
        user.update!(active: false)
        expect(user.messageable?).to be false
        user.destroy!
      end
    end
  end
end

Here’s my annotated version.

RSpec.describe User do
  let!(:user) { User.create!(email: 'test@example.com') } # setup

  describe '#messageable?' do
    context 'is inactive' do
      it 'is false' do
        user.update!(active: false)           # exercise
        expect(user.messageable?).to be false # assertion
        user.destroy!                         # teardown
      end
    end
  end
end

Takeaway

Being familiar with the four phases of a test can help you overcome the writer’s block that testers sometimes feel when staring at a blank editor. “Write the setup” is an easier job than “write the whole test”.

Understanding the four phases of a test can also help make it easier to parse the meaning of existing tests.

One thought on “The four phases of a test

  1. Dmitry

    First, I like the idea of such structuring!

    But I politely disagree with your first example even if you said that “the distinction between setup and exercise may seem blurry”. I think to make this distinction more obvious each step needs some formalization. For example, “in a setup phase we can prepare the state needed for a test without actual interaction needed to achieve this state in the application, in other words we can prepare a needed DB state). In that case the exercise will be just a call to `#messageable?` and the assertion will be an expectation for the result of such call.

    So I’d write the example in such way
    “`ruby
    RSpec.describe User do
    let!(:user) { User.create!(email: ‘test@example.com’) }

    describe ‘#messageable?’ do
    context ‘with inactive status’ do
    before do
    user.update!(active: false)
    end

    after do
    user.destroy!
    end

    it ‘returns false’ do
    expect(user.messageable?).to be false
    end
    end
    end
    end
    “`

    So for our story, where “Only active users are allowed to receive messages..”
    we don’t care how the user got an inactive status and we prepare the needed DB state in a setup phase.

    Moreover, I expect that any interaction with `user` in an exercise phase will be related to the interface of sending/receving the messages(ex. some `user.send_message` call) but we don’t have any in a such simple example.

    And, as you can see above, in RSpec I’d use `before/after` blocks for setup and teardown phases.

    Anyway, thanks for the information, it’s actually an interesting topic and I’ve had similar thoughts in this direction 🙂

    Reply

Leave a Reply

Your email address will not be published.