Author Archives: Jason Swett

Things you can ignore when getting started with Rails testing

Here’s an incomplete list of tools and concepts you might encounter when first trying to learn Rails testing: Capybara, Cucumber, Database Cleaner, factory_bot, Faker, MiniTest, RSpec, system tests, Test::Unit, acceptance tests, end-to-end-tests, mocks, stubs, unit tests and TDD.

That’s a lot of stuff. If you’re like most humans you might look at this long list of things, feel confused about where to start, and say, “I don’t know what to do. I’ll just deal with this later.”

The challenge of getting started with testing would be much easier if you knew exactly what you needed to know and what you could safely ignore. What follows is a list of what you can safely ignore.

Framework Decisions

You don’t need to get hung up on which framework to use. You literally can’t go wrong. The principles of testing are the same no matter which testing framework you use. Plus you can always change your mind later.

When I first got started with Rails, the way I decided on a testing framework was very simple. I noticed that most Rails developers used RSpec, so I just picked RSpec.

(I actually used Test::Unit for a while before realizing most Rails developers used RSpec. So I just switched. It wasn’t a very big deal.)

Cucumber, Capybara and System Tests

Most Rails test suites have two main sub-suites: a suite of model tests and a suite of integration tests.

Model tests test the behavior of ActiveRecord models by themselves. Integration tests actually spin up a browser and do things like fill out forms and click links and buttons as if a human is actually using the application.

Between these two, the bar is lower for model tests. For integration tests you have to do almost all the same stuff as model tests plus more. For this reason I suggest that if you’re a newcomer to Rails testing that you start with model tests (or even just Ruby tests without Rails) and ignore integration tests altogether until you get more comfortable with testing.

So you can ignore Cucumber (which I don’t recommend using at all), Capybara and system tests, which are all integration testing tools.

View Specs

I’ve never written a view spec. To me they seem tautological. I’ve also never encountered any other Rails developer who advocates writing view specs.

If for some reason I had a wildly complicated view, I could see a view spec potentially making sense. But I haven’t yet encountered that case.

Helper Specs

I don’t tend to write helper specs because I don’t tend to write helpers. Helpers are a fairly peripheral area of Rails that you can safely disregard altogether when you’re getting started with Rails testing.

Routing Specs

Like view specs, I find routing specs to be tautological. I don’t write them.

Request Specs

Request specs are a great boon and even necessity if your application has an API or if your controllers do anything non-trivial. But I don’t think you should worry about request specs when you’re just getting started.

Controller Specs

Controller specs are deprecated in favor of request specs.

What Not to Ignore

The main thing I would recommend learning when you’re getting started with Rails testing is model tests.

When you’re learning about model tests you’ll naturally have to get acquainted with RSpec syntax (or whichever framework you choose), factory_bot and Database Cleaner (or analogous tools). But other than the actual testing techniques, that’s about it as far as model test tooling goes.

If you want to make life even easier on yourself you can learn just Ruby and RSpec with no Rails involved. Then, after you get comfortable with RSpec syntax and basic testing techniques, you can approach Rails testing with more confidence.

Why I recommend against using Cucumber

Around the time I first started using Rails in 2011, I noticed that a lot of developers, seemingly all Rails developers, were using Cucumber to assist with testing.

I bought into the idea—describing test cases in plain English—but in practice I found Cucumber not to be very valuable. In fact, my experience has been that Cucumber adds a negative amount of value.

In recent years I’ve noticed (although this is anecdotal and might just be my perception) that fewer codebases seem to use Cucumber and that fewer Rails developers seem to be on board with Cucumber. I had thought Cucumber was pretty much dead. But lately, to my surpise, I’ve seen Cucumber recommended to testing noobs more than a few times. Since I consider Cucumber to be a bad thing, I want to explain why I think so and why I don’t think other people should use it.

In my view there are two general ways Cucumber can be used: it can be used as intended or it can be abused. In the former case, I believe Cucumber has a small negative value. In the latter case I believe it has a large negative value.

Why Cucumber is bad when it’s not used as intended

Most production Cucumber scenarios I’ve seen look something like this:

Given a user exists with email "test@example.com" and password "mypassword"
And I visit "/sign_in"
And I fill in the "Email" field with "test@example.com"
And I fill in the "Password" field with "mypassword"
And I click "Sign In"
And I visit "/user/edit"
And I fill in the "First Name" field with "John"
And I fill in the "Last Name" field with "Smith"
And I fill in the "Age" field with "30"
And I click "Save"
And I visit "/profile"
Then I should see "John Smith, 30"

These kinds of tests, with fine-grained steps, arise when the developers seem to mistake Cucumber for a way to write Ruby in English. The above scenario provides exactly zero benefit, in my opinion, over the following equivalent Capybara scenario:

FactoryBot.create(:user, email: 'test@example.com', password: 'mypassword')

visit sign_in_path
fill_in 'Email', with: 'test@example.com'
fill_in 'Password', with: 'mypassword'
click_on 'Sign In'

visit edit_user_path
fill_in 'First Name', with: 'John'
fill_in 'Last Name', with: 'Smith'
fill_in 'Age', with: '30'
click_on 'Save'

visit profile_path
expect(page).to have_content('John Smith, 30')

The Cucumber/Gherkin version is no shorter nor more easily understandable.

To be fair to Cucumber, nobody who understands Cucumber advocates writing Cucumber scenarios in this way. The Cucumber creator himself, Aslak Hellesøy, wrote a post in 2011 saying not to do this. Other people have written similar things.

I think it’s telling that so many people have written blog posts advising against the very common practice of writing fine-grained Cucumber steps. To me it’s kind one of those gas station doors that looks for all the world like a pull door, so every single person who comes up to it pulls it instead of pushes it, feels like a dumbass, and then pushes it. So the gas station manager puts up a big sign that says “PUSH”, but most people don’t notice it and the problem persists. What instead should have been done is to make the push door look like a push door, without the big useless handle that you’re not supposed to pull. I get that the Cucumber maintainers tried to do that by removing `web_steps.rb`, but in my experience it didn’t seem to work.

And it doesn’t matter much anyway because Cucumber still sucks even if you don’t abuse it by writing fine-grained steps. I’ll explain why I think so.

Why Cucumber is bad even when it is used as intended

Here’s a version of the above Cucumber scenario that’s done in the way the Cucumber creators would intend. There are two parts.

First, the Gherkin steps:

Given I am signed in
And I provide my name and age details
Then I should see those details on my profile page

Second, the underlying Ruby steps:

Given /^I am signed in$/ do
  visit sign_in_path
  fill_in 'Email', with: 'test@example.com'
  fill_in 'Password', with: 'mypassword'
  click_on 'Sign In'
end

And /^I provide my name and age details$/ do
  visit edit_user_path
  fill_in 'First Name', with: 'John'
  fill_in 'Last Name', with: 'Smith'
  fill_in 'Age', with: '30'
  click_on 'Save'
end

Then /^I should see those details on my profile page$/ do
  visit profile_path
  expect(page).to have_content('John Smith, 30')
end

This is actually pretty decent-looking and appealing, at least at first glance. There are two problems, though. First, this way of doing things doesn’t really provide any clarity over doing it the Capybara way. Second, the step definitions usually end up in a single, flat file full of “step soup” where unrelated steps are mixed together willy-nilly.

Compare this again with the Capybara version:

FactoryBot.create(:user, email: 'test@example.com', password: 'mypassword')

visit sign_in_path
fill_in 'Email', with: 'test@example.com'
fill_in 'Password', with: 'mypassword'
click_on 'Sign In'

visit edit_user_path
fill_in 'First Name', with: 'John'
fill_in 'Last Name', with: 'Smith'
fill_in 'Age', with: '30'
click_on 'Save'

visit profile_path
expect(page).to have_content('John Smith, 30')

The sign in portion is usually abstracted away in Capybara, too, so the scenario would look more like this:

FactoryBot.create(:user, email: 'test@example.com', password: 'mypassword')

sign_in

visit edit_user_path
fill_in 'First Name', with: 'John'
fill_in 'Last Name', with: 'Smith'
fill_in 'Age', with: '30'
click_on 'Save'

visit profile_path
expect(page).to have_content('John Smith, 30')

That’s not too crazy at all. In order for Cucumber to be a superior solution to using bare Capybara, it would have to have some pretty strong benefits to compensate for the maintenance burden and cognitive overhead it adds. But it doesn’t.

So what do I recommend doing instead of using Cucumber? I think just using Capybara by itself is fine, and better than using Capybara + Cucumber. I also think Capybara + page objects is a pretty good way to go.

Rails testing resource roundup

.resource-item {
border-bottom: 1px solid #DDD;
padding-bottom: 50px;
margin-top: 30px;
margin-bottom: 50px;
}

Below is a list of testing resources I’ve either used myself or heard recommended. I tried to make the list as Ruby-centric as possible, although some resources from other ecosystems are so good that I didn’t want to exclude them.

I intend this to be a living list that grows over time. If you know of something that should be on this list but isn’t, please let me know, either in a comment on this page or via email.

Disclosure statement: None of the links below are affiliate links, although I do have a relationship with some of the authors/creators of these resources.

Section One: Ruby/Rails Specific Resources

Print Book: Rails 5 Test Prescriptions


Excerpt from Pragmatic Bookshelf summary:

“Does your Rails code suffer from bloat, brittleness, or inaccuracy? Cure these problems with the regular application of test-driven development. You’ll use Rails 5.2, Minitest 5, and RSpec 3.7, as well as popular testing libraries such as factory_bot and Cucumber.”

Details at The Pragmatic Bookshelf

Side note: you can also listen to my interview with author Noel Rappin on the Ruby Testing Podcast.

eBook: Everyday Rails Testing with RSpec


Summary from Leanpub:

“Real-world advice for adding reliable tests to your Rails apps with RSpec, complete with expanded, exclusive content and a full sample application. Updates for 2017 now available—RSpec 3.6, Rails 5.1, and more! Learn to test with confidence!”

Details at Leanpub

Screencast Series: Destroy All Software

Not everything on Destroy All Software (DAS) is testing-related but a lot of it is. I often see DAS recommended when people ask for testing-related resources.

Destroy All Software Catalog

Online Course Series: Upcase’s “Learn Testing” Courses

Summary from Upcase website:

“Test-driven development, or TDD, is the practice of writing your tests firsts, then using those tests to guide you as you write your actual production code. This may sound crazy, but it turns out that it makes writing code much easier. It provides a clear workflow and next steps while you’re building and has the added benefit of producing a test suite you can have confidence in. With these courses and videos we’ll teach you everything you need to know to get started with TDD.”

Details at Upcase

Print Book/eBook: Effective Testing with RSpec 3

Excerpt from Pragmatic Bookshelf summary:

“This definitive guide from RSpec’s lead developer shows you how to use RSpec to drive more maintainable designs, specify and document expected behavior, and prevent regressions during refactoring. Build a project using RSpec to design, describe, and test the behavior of your code. Whether you’re new to automated tests or have been using them for years, this book will help you write more effective tests.”

Details at The Pragmatic Bookshelf

Print Book/eBook: Practical Object-Oriented Design in Ruby

This isn’t specifically a testing book but I’ve seen it recommended a number of times as a book that will help you write better Ruby tests.

Excerpt from summary:
“[Practical Object-Oriented Design in Ruby] explains object-oriented design (OOD) using realistic, understandable examples. POODR* is a practical, readable introduction to how OOD can lower your costs and improve your applications.”

Details at Author’s Website

Section Two: Non-Rails-Specific Resources

Print Book: Growing Object-Oriented Software, Guided by Tests

This book is a classic in the testing world. I first read Growing Object-Oriented Software, Guided by Tests (GOOS) when I was clueless about testing. It helped me get oriented and learn what’s what. Among the most important concepts I learned from this book is the idea of a Walking Skeleton.

Details at Amazon

Print Book: Working Effectively with Legacy Code

I’ve worked on dozens of codebases so far in my career. Most of them have been legacy code. This book was super helpful in showing techniques like the Sprout Method technique to help get legacy code under control.

Details at Amazon

All code eventually gets tested, it’s just a question of when and how and by whom

All code eventually gets tested. If a bug is found, it can be found in one of the following ways, in descending order of cost and embarrassment.

A Customer Trying to Use the Feature in Production

Obviously the worst place to find a bug is in production, and the worst person to find it is an end user trying to do his or her job.

This is a bad way to find a bug not just because it blocks your user from using your feature successfully and thus hurts your reputation, but because you might not even immediately find out that the bug exists. Not every user who finds a bug immediately reports that bug to the vendor’s development team. When I find a bug in Gmail, for example, I don’t make an attempt to report the bug to the Gmail team, I just suffer through it.

Even if the customer reports the bug, the bug probably won’t get directly reported to you, the developer. It will probably get reported to a support person, then get communicated to your manager, then communicated to you. By this point the bug may gain high visibility inside your organization, providing embarrassment to you and your team.

Discovering bugs in production is of course inevitable but all possible measures should be taken to avoid it.

Your Boss/Client Trying to Use It

The next worst way to find a bug is for your boss or client to find it.

I don’t mean when your boss/client serves in a QA tester role. I mean when your boss/client tries to use the feature with the expectation that it’s already finished and verified by you to be working.

The reason that this is bad is obvious: it erodes the trust your boss or client has in your competence as a developer. It especially hurts to get back a comment like, “Hey, let me know when this is 100% working.”

Not only is there the embarrassment and trust erosion but a concrete cost as well. The back-and-forth between you and your boss/client lengthens the development cycle for the feature, and therefore increases the cost to develop that feature.

Manually Testing it Yourself

Catching a bug by manually testing the feature yourself is way better than letting your boss, client or end user find it.

If you discover a silly bug, you might still feel a little embarrassed, but it will be a small, private embarrassment. You also keep the development cycle shorter by skipping the back-and-forth of “Hey boss, this is done,” “Hey developer, it’s not actually done.”

An Automated Test

The least expensive way to catch a bug is to write an automated test that makes the bug physically impossible.

Some developers perceive that writing automated tests takes “extra” time. That’s an inaccurate way to look at it. All testing takes time. The difference is that with a manual test, the time is paid each time you carry out the test, and with automated testing the time is paid all at once upfront. It’s like the difference between paying cash for a $250,000 house versus getting a mortgage and paying $500,000 for the same house over the course of 30 years. It potentially only takes a small handful of runs of a manual test before you’ve used up more time than it would have taken to write an automated test.

“So You’re Saying Automated Tests Are The Answer?”

My point is not that automated tests are a better alternative to all the other forms of testing. My point is that I believe automated tests are, in most cases, the least expensive testing method and the best first line of defense.

Imagine a secret government research facility that’s protected by an armed guard, locking doors and safe containing sensitive documents. If a Russian spy is after the documents, it’s obviously better for the spy to get caught by the guard than for the spy to make it past the guard and locked doors and make it all the way to the safe. But that doesn’t mean it’s a smart idea to do away with the doors and safe and only have the guard. Defense should exist at all levels practicable.

Same thing with automated testing. Just because it’s cheaper for a bug to get caught by an automated test doesn’t mean it’s a good idea to do away with manual testing. Protection should exist at all levels. It’s just a good idea to try to catch as many bugs at the earliest levels as possible.

How to Get RSpec to Skip View Specs When You Generate Scaffolds

I personally don’t find much value in tests for views, helpers, routes or requests in most cases.

It’s annoying to have to delete these files each time you generate a new scaffold. Fortunately, it’s possible to configure RSpec not to generate these files.

Below is an example of how you can exclude these types of spec (and more) from being generated when you generate a new scaffold.

require_relative 'boot'

require "rails"
# Pick the frameworks you want:
require "active_model/railtie"
require "active_job/railtie"
require "active_record/railtie"
require "active_storage/engine"
require "action_controller/railtie"
require "action_mailer/railtie"
require "action_view/railtie"
require "action_cable/engine"
require "sprockets/railtie"
# require "rails/test_unit/railtie"

# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)

module MyApp
  class Application < Rails::Application
    # Initialize configuration defaults for originally generated Rails version.
    config.load_defaults 5.2

    # Settings in config/environments/* take precedence over those specified here.
    # Application configuration can go into files in config/initializers
    # -- all .rb files in that directory are automatically loaded after loading
    # the framework and any gems in your application.

    # Don't generate system test files.
    config.generators.system_tests = nil

    config.generators do |g|
      g.test_framework :rspec,
        fixtures:         false,
        view_specs:       false,
        helper_specs:     false,
        routing_specs:    false,
        request_specs:    false,
        controller_specs: false
    end
  end
end

Ruby/Rails testing glossary

Ruby-Specific Terms

Capybara
Integration testing tool. Capybara’s GitHub page describes it as an “acceptance test framework for web applications” but everyone I know uses the term integration test when they talk about Capybara.
Cucumber
A tool that lets you write test scenarios in English commands and then tie those English commands to code. For example, a Cucumber file could contain the line `Given I am on the login page`. In a different file, the text `Given I am on the login page` would map to some Ruby code that navigates to the login page.

I personally advise people not to use Cucumber.

Database Cleaner
The tests in a test suite should be runnable in any order. If a test leaves behind extra data or depends on data set up in a previous test, there’s a good chance the ability to run the tests in any order has been lost. Database Cleaner is a tool that helps ensure every test case starts and ends with a blank database.
factory_bot
“A library for setting up Ruby objects as test data”, according to factory_bot’s GitHub page. I think that’s a pretty good description. If I were to say it in my own words, I’d describe factory_bot as a library that helps define factories for the purpose of conveniently creating test data.
Faker
Faker is, for my purposes, a tool that generates random values so I don’t have to manually come up with arbitrary phone numbers, names of people, etc. when testing. Faker is also handy when I need test values to be unique, e.g. when having two people with the same name would be a problem. I use Faker in conjunction with factory_bot.
MiniTest
A Ruby testing framework.
RSpec
A Ruby testing framework. In my perception, RSpec is by far the most popular testing framework in the Rails community.
System Tests
A feature added to Rails core in Rails 5.1 to make integration testing easier. Uses Capybara. If you’d like to learn more about System Tests, Noel Rappin wrote a pretty good Rails System Tests guide.
Test::Unit
A Ruby testing framework.
timecop
A library that makes it easy to test time-dependent code. Offers features like “time travel” and “time freezing”.

Non-Ruby-Specific Terms

Acceptance Test
A test that checks whether a feature meets its business requirements. A certain feature may work exactly as the developer and designer intended, but if the developer and designer designed and built the wrong thing, the acceptance test would fail.
Assertion
A condition that evaluates to true if the feature behaves properly and false if the feature contains a bug. The analogous term in RSpec is “expectation”.
CI Server
A server that aids in the practice of continuous integration. A CI server typically runs the test suite anytime a code change is pushed and notifies the development team in the case of a test suite failure.
CircleCI
A CI server product.
Continuous Integration
The practice of continuously merging together the work of all the developers on a development team. The benefit of continuous integration is that it helps teams avoid the painful process of integrating large chunks of work, which is often tedious and error-prone.
Dependency Injection
A technique where an object’s dependencies are passed as arguments, making it easier to test objects in isolation. See this post of mine for an example.

Factory
There’s a design pattern called the Factory Pattern or the Factory Method Pattern and it exists independently of anything to do with tests. I basically think of a factory as code that spits out an object of some sort.

I included the Factory term here because factories are very relevant to how I write tests. Rather than manually spinning up a bunch of arbitrary data using fixtures, I prefer to define factories that can hide away irrelevant details, allowing me to focus only on the parts I care about. For example, if I want to create a restaurant for testing purposes, I might not want to have to specify the restaurant’s phone, address, hours, or any of those other details, even though they may be required attributes. Most of the time I just want to specify the restaurant’s name and nothing else. Factories can make this job very easy and convenient.

Fixture
A way of setting up test data. In Rails, fixtures are defined using YAML files. I prefer factories over fixtures for a number of reasons. One of the main reasons is that I find fixtures to be too “distant” from the tests, making it somewhat mysterious what data is available for any particular test.
Flapping test
A test that sometimes fails and sometimes passes, seemingly at random. Also known as a “flickering” test. They’re often a symptom of one test case “leaking” into subsequent test cases and affecting their behavior.
Edge Case
A path through a UI that isn’t exercised frequently or that is unlikely to be exercised.
End-to-End Test
A test that exercises all the layers of an application stack. It could be said that an end-to-end test is more inclusive than an integration test and that an integration test only has to combine two or more layers of an application stack to be considered an integration test. In practice I’ve found that people use the terms “integration test” and “end-to-end test” fairly interchangeably and if you want total clarity, you have to ask what exactly the person means.
Environment Parity
Environment parity is a broader version of the dev/prod parity term from the Twelve-Factor App principles. The idea is that every difference between two environments is an opportunity for a bug to be present in one environment that is not present in the other. For this reason it’s helpful to keep the test, development and production environments as similar to each other as possible.
Happy Path
A normal/default/valid path through a feature. (I like to think of the opposite as the “sad path”.)
Integration Test
A test that combines two or more layers of an application stack, or that combines two or more systems. See also: end-to-end test.
Mock
A “fake” object. A mock is useful when you want to test a certain object, but don’t want to deal with the hassle of bringing that object’s dependencies into the picture. The dependencies can instead be mocked.
Regression Testing
A type of testing meant to ensure that previously-working functionality has not been broken by newly-developed functionality.
Selenium
A tool for automating browsers. Also, a chemical element.
Stub
A term that may or may not mean the exact same thing as mock. Mock and stub are not terms that seem to have crisp industry consensus.
System Under Test
The system that’s being tested.
Test Case
I don’t think I can improve on Wikipedia‘s definition: “A specification of the inputs, execution conditions, testing procedure, and expected results that define a single test to be executed to achieve a particular software testing objective”.
Test Coverage
The percentage of an application covered by automated tests.
Test Environment
An independent instance of an application used specifically for the purpose of testing.
Test Suite
A collection of tests. An application might have one or many (or zero!) test suites.
Test-Driven Development
The practice of developing features by first writing tests that specify the behavior of the desired functionality, then writing the code to make those tests pass.
Unit Test
A fine-grained test that tests a very specific piece of functionality in isolation. Note: what Rails developers often refer to “unit tests” are not technically unit tests since those tests involve the database.

Taming legacy Ruby code using the “Sprout Method” technique (example 1)

Note: after I wrote this post I came up with a better example, which can be found here.

There are a number of reasons why it can often be difficult to add test coverage to legacy projects.

One barrier is the entanglement of dependencies that stands in the way of instantiating the objects you want to test.

Another barrier is the “wall of code” characteristic often present in legacy code: long, unstructured classes and methods with obfuscated variable names and mysterious behavior. Testing big things is a lot harder than testing small things.

In this example I’m going to demonstrate how to take a long, unstructured method, put tests on it, and then improve its design.

Legacy code example

Here’s a piece of crappy simulated legacy code:

require 'date'

class CreditCard
  def initialize(number, expiration_date, brand)
    @number = number
    @expiration_date = expiration_date
    @brand = brand
  end

  def valid?
    month, year = @expiration_date.split('/').map(&:to_i)
    year += 2000
    if DateTime.now.to_date < Date.new(year, month)
      return false
    else
      if @brand == 'American Express'
        @number.gsub(/\s+/, '').length == 15
      else
        @number.gsub(/\s+/, '').length == 16
      end
    end
  end
end

Why is this code crappy? For one, the `valid?` method is too long to easily be understood at a glance. Secondly, it has a bug. If you give the class an expired card it will still tell you the card is valid.

> cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
 => #<CreditCard:0x007fb8d9825890 @number="3843111122223333", @expiration_date="02/17", @brand="Visa"> 
> cc.valid?
 => true

If we want to fix this code there are a couple different approaches we could take.

Two Possible Approaches

  1. Try to debug the code by tediously running it over and over on the console
  2. Write some tests

The second approach has the benefit of being not only less annoying but also permanent. If we write a test to enforce that the expiration date bug is not present, we can reasonably expect that our test will protect against this bug forever. If we just debug the code in the console, we get no such guarantee.

But it’s not always easy to just go ahead and start writing tests.

Obstacles to Writing Tests

There’s a certain obstacle to testing that’s often present in legacy code. That is, when you have a line or file that’s a bazillion lines long, it’s hard to isolate the piece of code you’re interested in testing. There are often dependencies tangled up in the code. You’re afraid to try to untangle the dependencies because you don’t want to change code you don’t understand. But in order to understand the code, you need to untangle the dependencies. It’s a tough catch-22 type situation.

The Solution

The solution I like to apply to this problem is Michael Feathers’ sprout method technique which he describes in Working Effectively with Legacy Code.

What we’re about to do is:

  1. Write a test around the buggy area—expiration date validation—and watch it fail
  2. Extract the expiration date code into its own method so we can isolate the incorrect behavior
  3. Fix the bug and watch our test pass
  4. Let’s start by writing the first test.

    Writing the First Test

    I’ll write a test that says, “When the credit card’s expiration date is in the past, expect that card not to be valid.”

    require 'rspec'
    require './credit_card'
    
    describe CreditCard do
      describe '#valid?' do
        context 'expired' do
          it 'is not valid' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).not_to be_valid
          end
        end
      end
    end

    When I run this test it fails, as expected.

    Next I’ll break up this code a little bit by extracting a chunk of it into its own method.

    Extracting Our First Method

    The highlighted chunk below looks like a pretty good candidate to be separated.

    def valid?
      month, year = @expiration_date.split('/').map(&:to_i)
      year += 2000
      if DateTime.now.to_date < Date.new(year, month)
        return false
      else
        if @brand == 'American Express'
          @number.gsub(/\s+/, '').length == 15
        else
          @number.gsub(/\s+/, '').length == 16
        end
      end
    end

    I’m going to extract it into a `number_is_right_length?` method.

    require 'date'
    
    class CreditCard
      def initialize(number, expiration_date, brand)
        @number = number
        @expiration_date = expiration_date
        @brand = brand
      end
    
      def valid?
        month, year = @expiration_date.split('/').map(&:to_i)
        year += 2000
        if DateTime.now.to_date < Date.new(year, month)
          return false
        else
          number_is_right_length?
        end
      end
    
      def number_is_right_length?
        if @brand == 'American Express'
          @number.gsub(/\s+/, '').length == 15
        else
          @number.gsub(/\s+/, '').length == 16
        end
      end
    end

    Now that I have this functionality in its own method, I’ll write a test for just that part of the code, in isolation.

    require 'rspec'
    require './credit_card'
    
    describe CreditCard do
      describe '#valid?' do
        context 'expired' do
          it 'is not valid' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).not_to be_valid
          end
        end
      end
    
      describe '#number_is_right_length?' do
        context 'no spaces' do
          context 'right length' do
            it 'returns true' do
              cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
              expect(cc.number_is_right_length?).to be true
            end
          end
        end
      end
    end

    If I run this test it will pass.

    There’s something I don’t like about this, though. The expiration date of the card doesn’t have anything to do with the validity of the length of the card number. The extra setup data is distracting. A future maintainer might look at the test and assume that the expiration date is necessary.

    Removing Extraneous Setup Data

    It would be nice if we could test just the number length by itself. It would be nice if our test could look like this:

    require 'rspec'
    require './credit_card'
    
    describe CreditCard do
      describe '#valid?' do
        context 'expired' do
          it 'is not valid' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).not_to be_valid
          end
        end
      end
    
      describe '#number_is_right_length?' do
        context 'no spaces' do
          context 'right length' do
            it 'returns true' do
              valid = CreditCard.number_is_right_length?('Visa', '3843111122223333')
              expect(valid).to be true
            end
          end
        end
      end
    end

    In order to make our new `number_is_right_length?` method more testable in isolation, I’m going to change it from an instance method to a class method. Instead of using the `@brand` and `@number` instance variables, I’m going to use dependency injection so the `number_is_right_length?` doesn’t rely on a `CreditCard` instance having been created first.

    require 'date'
    
    class CreditCard
      def initialize(number, expiration_date, brand)
        @number = number
        @expiration_date = expiration_date
        @brand = brand
      end
    
      def valid?
        month, year = @expiration_date.split('/').map(&:to_i)
        year += 2000
        if DateTime.now.to_date < Date.new(year, month)
          return false
        else
          self.class.number_is_right_length?(@brand, @number)
        end
      end
    
      def self.number_is_right_length?(brand, number)
        if brand == 'American Express'
          number.gsub(/\s+/, '').length == 15
        else
          number.gsub(/\s+/, '').length == 16
        end
      end
    end

    Extracting Expiration-Related Code

    Now I’m going to extract the expiration-related code into its own method. That code is highlighted below.

    def valid?
      month, year = @expiration_date.split('/').map(&:to_i)
      year += 2000
      if DateTime.now.to_date < Date.new(year, month)
        return false
      else
        self.class.number_is_right_length?(@brand, @number)
      end
    end

    That chunk of code will go out of the `valid?` method and into a new `expired?` method.

    require 'date'
    
    class CreditCard
      def initialize(number, expiration_date, brand)
        @number = number
        @expiration_date = expiration_date
        @brand = brand
      end
    
      def valid?
        if expired?
          false
        else
          self.class.number_is_right_length?(@brand, @number)
        end
      end
    
      def expired?
        month, year = @expiration_date.split('/').map(&:to_i)
        year += 2000
        DateTime.now.to_date < Date.new(year, month)
      end
    
      def self.number_is_right_length?(brand, number)
        if brand == 'American Express'
          number.gsub(/\s+/, '').length == 15
        else
          number.gsub(/\s+/, '').length == 16
        end
      end
    end

    Similar to how I did with the `number_is_right_length?` method, I’ll now write a test for just the `expired?` method in isolation. The main reasons for extracting code into a smaller method are a) smaller methods are easier to understand, b) smaller methods can more easily be given descriptive and appropriate names, further aiding understanding, and c) it’s more manageable to write tests for small methods with few dependencies than large methods with many dependencies.

    Finding and Fixing the Bug

    This test will say, “When the expiration date is in the past, this method should return false.”

    require 'rspec'
    require './credit_card'
    
    describe CreditCard do
      describe '#valid?' do
        context 'expired' do
          it 'is not valid' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).not_to be_valid
          end
        end
      end
    
      describe '#number_is_right_length?' do
        context 'no spaces' do
          context 'right length' do
            it 'returns true' do
              valid = CreditCard.number_is_right_length?('Visa', '3843111122223333')
              expect(valid).to be true
            end
          end
        end
      end
    
      describe '#expired?' do
        context 'expired' do
          it 'returns true' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).to be_expired
          end
        end
      end
    end

    If I run this test I notice that it does not pass. I’ll write another test for the opposite scenario: when the expiration date is in the future, the method should return true.

    require 'rspec'
    require './credit_card'
    
    describe CreditCard do
      describe '#valid?' do
        context 'expired' do
          it 'is not valid' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).not_to be_valid
          end
        end
      end
    
      describe '#number_is_right_length?' do
        context 'no spaces' do
          context 'right length' do
            it 'returns true' do
              valid = CreditCard.number_is_right_length?('Visa', '3843111122223333')
              expect(valid).to be true
            end
          end
        end
      end
    
      describe '#expired?' do
        context 'expired' do
          it 'returns true' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).to be_expired
          end
        end
    
        context 'not expired' do
          it 'returns false' do
            cc = CreditCard.new('3843111122223333', '02/30', 'Visa')
            expect(cc).not_to be_expired
          end
        end
      end
    end

    This test also does not pass. So the method returns true when it should return false and false when it should return true. It’s now clear to me where the mistake lies: I put `<` when I should have put `>`!

    Switching from `<` to `>` fixes the bug and makes both tests pass.

    require 'date'
    
    class CreditCard
      def initialize(number, expiration_date, brand)
        @number = number
        @expiration_date = expiration_date
        @brand = brand
      end
    
      def valid?
        if expired?
          false
        else
          self.class.number_is_right_length?(@brand, @number)
        end
      end
    
      def expired?
        month, year = @expiration_date.split('/').map(&:to_i)
        year += 2000
        DateTime.now.to_date > Date.new(year, month)
      end
    
      def self.number_is_right_length?(brand, number)
        if brand == 'American Express'
          number.gsub(/\s+/, '').length == 15
        else
          number.gsub(/\s+/, '').length == 16
        end
      end
    end

    But again, I’m not a huge fan of the fact that I have to bring irrelevant setup data into the picture. Card number and brand have no bearing on whether a date has passed or not.

    I can give this method the same treatment as `number_is_right_length?` by making it a class method and adding an `expiration_date` parameter.

    require 'date'
    
    class CreditCard
      def initialize(number, expiration_date, brand)
        @number = number
        @expiration_date = expiration_date
        @brand = brand
      end
    
      def valid?
        if self.class.expired?(@expiration_date)
          false
        else
          self.class.number_is_right_length?(@brand, @number)
        end
      end
    
      def self.expired?(expiration_date)
        month, year = expiration_date.split('/').map(&:to_i)
        year += 2000
        DateTime.now.to_date > Date.new(year, month)
      end
    
      def self.number_is_right_length?(brand, number)
        if brand == 'American Express'
          number.gsub(/\s+/, '').length == 15
        else
          number.gsub(/\s+/, '').length == 16
        end
      end
    end

    Now I can change my `expired?` tests to only deal with date and nothing else.

    require 'rspec'
    require './credit_card'
    
    describe CreditCard do
      describe '#valid?' do
        context 'expired' do
          it 'is not valid' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).not_to be_valid
          end
        end
      end
    
      describe '#number_is_right_length?' do
        context 'no spaces' do
          context 'right length' do
            it 'returns true' do
              valid = CreditCard.number_is_right_length?('Visa', '3843111122223333')
              expect(valid).to be true
            end
          end
        end
      end
    
      describe '#expired?' do
        context 'expired' do
          it 'returns true' do
            expect(CreditCard.expired?('02/17')).to be true
          end
        end
    
        context 'not expired' do
          it 'returns false' do
            expect(CreditCard.expired?('02/30')).to be false
          end
        end
      end
    end

    Future-Proofing

    Lastly, this most recent test has an unacceptable flaw. I’m checking that the card is not expired if the date is February 2030. As of 2018, February 2030 is in the future, but what happens if this program survives until March 2030? My test suite will break even though the code still works.

    So instead of making the dates hard-coded, let’s make them relative to the current date.

    require 'rspec'
    require './credit_card'
    
    describe CreditCard do
      describe '#valid?' do
        context 'expired' do
          it 'is not valid' do
            cc = CreditCard.new('3843111122223333', '02/17', 'Visa')
            expect(cc).not_to be_valid
          end
        end
      end
    
      describe '#number_is_right_length?' do
        context 'no spaces' do
          context 'right length' do
            it 'returns true' do
              valid = CreditCard.number_is_right_length?('Visa', '3843111122223333')
              expect(valid).to be true
            end
          end
        end
      end
    
      describe '#expired?' do
        let(:this_year) { Date.today.strftime('%y').to_i }
    
        context 'expired' do
          it 'returns true' do
            expect(CreditCard.expired?("02/#{this_year - 1}")).to be true
          end
        end
    
        context 'not expired' do
          it 'returns false' do
            expect(CreditCard.expired?("02/#{this_year + 1}")).to be false
          end
        end
      end
    end

    The Current State of the Code

    Here’s what our class looks like right now:

    require 'date'
     
    class CreditCard
      def initialize(number, expiration_date, brand)
        @number = number
        @expiration_date = expiration_date
        @brand = brand
      end
     
      def valid?
        if self.class.expired?(@expiration_date)
          false
        else
          self.class.number_is_right_length?(@brand, @number)
        end
      end
     
      def self.expired?(expiration_date)
        month, year = expiration_date.split('/').map(&:to_i)
        year += 2000
        DateTime.now.to_date > Date.new(year, month)
      end
     
      def self.number_is_right_length?(brand, number)
        if brand == 'American Express'
          number.gsub(/\s+/, '').length == 15
        else
          number.gsub(/\s+/, '').length == 16
        end
      end
    end

    This code actually doesn’t look super great. It’s arguably not even better than what we started with. Here are some things we could do to improve it:

    • Change the `if/else` in `valid?` to use a guard clause
    • DRY up the length checks in `expired?`
    • Add test coverage for everything that’s not covered yet—which is a lot
    • Put the “magic numbers” of 15 and 16 into constants

    I also am not sure I’m a huge fan of the class methods. What would it look like if we changed those back to instance methods?

    The Fully Refactored Version

    Here’s what I came up with after further refactoring:

    require 'date'
    
    class CreditCard
      NUMBER_LENGTH = 16
      NUMBER_LENGTH_AMEX = 15
    
      def initialize(number, expiration_date, brand)
        @number = number
        @expiration_date = expiration_date
        @brand = brand
      end
    
      def valid?
        number_is_right_length? && !expired?
      end
    
      def expired?
        DateTime.now.to_date > Date.new(*expiration_year_and_month)
      end
    
      def expiration_year_and_month
        month, year = @expiration_date.split('/').map(&:to_i)
        [year + 2000, month]
      end
    
      def number_is_right_length?
        stripped_number.length == correct_card_length
      end
    
      def correct_card_length
        if @brand == 'American Express'
          NUMBER_LENGTH_AMEX
        else
          NUMBER_LENGTH
        end
      end
    
      def stripped_number
        @number.gsub(/\s+/, '')
      end
    end

    In addition to the bullet points above, I made some methods smaller by pulling certain functionality out into new ones.

    Here’s the final test suite:

    require 'rspec'
    require './credit_card'
    
    describe CreditCard do
      let(:this_year) { Date.today.strftime('%y').to_i }
      let(:future_date) { "02/#{this_year - 1}" }
      let(:past_date) { "02/#{this_year + 1}" }
    
      describe '#valid?' do
        context 'expired' do
          it 'is not valid' do
            cc = CreditCard.new('1111111111111111', '02/17', 'Visa')
            expect(cc).not_to be_valid
          end
        end
      end
    
      describe '#number_is_right_length?' do
        context 'no spaces' do
          context 'right length' do
            it 'returns true' do
              cc = CreditCard.new('1111111111111111', future_date, 'Visa')
              expect(cc.number_is_right_length?).to be true
            end
          end
        end
    
        context 'American Express' do
          it 'needs to be 15 numbers long' do
            cc = CreditCard.new('111111111111111', future_date, 'American Express')
            expect(cc.number_is_right_length?).to be true
          end
        end
    
        context 'non-American-Express' do
          it 'needs to be 16 numbers long' do
            cc = CreditCard.new('1111111111111111', future_date, 'Visa')
            expect(cc.number_is_right_length?).to be true
          end
        end
      end
    
      describe '#expired?' do
        context 'expired' do
          it 'returns true' do
            cc = CreditCard.new('1111111111111111', future_date, 'Visa')
            expect(cc).to be_expired
          end
        end
    
        context 'not expired' do
          it 'returns false' do
            cc = CreditCard.new('1111111111111111', past_date, 'Visa')
            expect(cc).not_to be_expired
          end
        end
      end
    end

Testing Anti-Pattern: Setup Data Leak

In general it’s better to have tests than to not have tests. Most people agree on this.

But just like poor-quality code, poor-quality tests get hard to maintain over time.

One anti-pattern I see in testing, an anti-pattern that makes tests harder and harder to maintain as the tests grow, is what I’ll call “setup data leak”. Here’s what it looks like.

Let’s imagine we have a `User` class. Maybe the user class can do things related to accounts, roles and posts.

In order to test accounts, roles and posts, we have to create some account, role and post data.

A Bad Test

describe User do
  let(:user) { create(:user) }
  let(:account) { create(:account, user: user) }
  let(:role) { create(:role, user: user) }
  let(:post) { create(:post, user: user) }

  describe 'account-related stuff' do
    # tests go here
  end

  describe 'role-related stuff' do
    # tests go here
  end

  describe 'post-related stuff' do
    # tests go here
  end
end

Why is this test bad? Because it defines all the test data in one place. Why is that bad? Two reasons.

The “Single Glob of Data” Phenomenon

Let’s imagine that this test file grows over time and instead of setting up 4 things at the top, we set up 15. How could anyone be sure which of those 15 things is needed for any particular test? Maybe some tests only need 3 of those 15 things. Maybe other tests need all 15. Maybe still others need 8 of them.

Since we don’t know what’s needed for any particular test, we basically have to assume that everything is needed for every test until we prove otherwise on a case-by-case basis. So our 15 things aren’t 15 things anymore but one single horrifying glob of data.

Mystery Guests

Let’s again imagine that this test file grows over time. Deep within the bowels of one particular test you see `role`. What’s `role`? You look to the immediate vicinity and find nothing. Eventually you see that `role` is defined at the very top of the file even though it’s only used in one place.

A mystery guest is a value that appears in a test case but is defined far away from the test case.

If a value is truly used everywhere then there’s no reason not to define it at the very top of the test. But if it’s only used in one place, there’s no reason not to define it right next to the place it’s used.

A Better Test

describe User do
  let(:user) { create(:user) }

  describe 'account-related stuff' do
    let(:account) { create(:account, user: user) }
    # tests go here
  end

  describe 'role-related stuff' do
    let(:role) { create(:role, user: user) }
    # tests go here
  end

  describe 'post-related stuff' do
    let(:post) { create(:post, user: user) }
    # tests go here
  end
end

This test is better than the first one because there’s no “setup data leak” issue. All the setup data is defined close to where it’s needed. To put it another way, all the setup data is defined in the narrowest scope possible.

Did you find this article useful? See all my Ruby/Rails testing articles.