I recently saw a post on Reddit where the OP asked how to test a method which involved puts and gets. The example the OP posted looked like the following (which I’ve edited very slightly for clarity):

class Example
  def ask_for_number
    puts "Input an integer 5 or above"
    loop do
      input = gets.to_i
      return true if input >= 5
      puts "Invalid. Try again:"
    end
  end
end

What makes the ask_for_number method challenging to test is a dependency. Most methods can be tested by saying, “When I pass in argument X, I expect return value Y.” This one isn’t so straightforward though. This is more like “When the user sees output X and then enters value V, expect subsequent output O.”

Instead of accepting arguments, this method gets its value from user input. And instead of necessarily returning a value, this method sometimes simply outputs more text.

How can we give this method the values it needs, and how can we observe the way the method behaves when we give it these values?

A solution using dependency injection (thanks to Myron Marston)

Originally, I had written my own solution to this problem, but then Myron Marson, co-author of Effective Testing with RSpec 3, supplied an answer of his own which was a lot better than mine. Here it is.

I comment on this solution some more below, but you can see in the initialize method that the input/output dependencies are being injected into the class. By default, we use $stdin/$stdout, and under test, we use something else for easier testability.

class Example
  def initialize(input: $stdin, output: $stdout)
    @input = input
    @output = output
  end

  def ask_for_number
    @output.puts "Input an integer 5 or above"
    loop do
      input = @input.gets.to_i
      return true if input >= 5
      @output.puts "Invalid. Try again:"
    end
  end
end

require 'stringio'

RSpec.describe Example do
  context 'with input greater than 5' do
    it 'asks for input only once' do
      output = ask_for_number_with_input(6)

      expect(output).to eq "Input an integer 5 or above\n"
    end
  end

  context 'with input equal to 5' do
    it 'asks for input only once' do
      output = ask_for_number_with_input(5)

      expect(output).to eq "Input an integer 5 or above\n"
    end
  end

  context 'with input less than 5' do
    it 'asks repeatedly, until a number 5 or greater is provided' do
      output = ask_for_number_with_input(2, 3, 6)

      expect(output).to eq <<~OUTPUT
        Input an integer 5 or above
        Invalid. Try again:
        Invalid. Try again:
      OUTPUT
    end
  end

  def ask_for_number_with_input(*input_numbers)
    input = StringIO.new(input_numbers.join("\n") + "\n")
    output = StringIO.new

    example = Example.new(input: input, output: output)
    expect(example.ask_for_number).to be true

    output.string
  end
end

Under test, an instance of StringIO can be used instead of $stdout, thus making the messages sent to @output visible and testable. That’s how we can “see inside” the Example class and test what at first glance appears to be a difficult-to-test piece of code.

One of the most common questions I see from beginners to Rails testing is what mocks and stubs are and when to use them. If you’re confused about mocks and stubs, you’re not alone. In my experience very few people understand them. In this post I’ll attempt to help clarify the matter, particularly in the context of Rails/RSpec.

I want to preface my explanation with a disclaimer. This post is partly derived from personal experience but it’s mostly derived from me just reading a bunch of posts online and reading a few books to try to understand mocks and stubs better. It’s entirely possible that what follows is not 100% accurate. But I’m risking the inaccuracy because I think the internet needs a clear, simple explanation of mocks and stubs and so far I haven’t been able to find one.

Here’s what we’re going to cover in this post:

The problem with testing terminology

The field of automated testing has about a million different terms and there’s not a complete consensus on what they all mean. For example, are end-to-end tests and acceptance tests the same thing? Some people would say yes, some would say no, and there’s no central authority to say who’s right and who’s wrong. That’s a problem with testing terminology in general.

A problem with mocks and stubs in particular is that programmers are often sloppy with the language. People say mock when they mean stub and vice versa.

The result of these two issues is that many explanations of mocks and stubs are very very very confusing.

Test doubles

I had a lightbulb moment when I read in Gerard Meszaros’ xUnit Test Patterns that mocks and stubs are each special types of test doubles.

To me this was a valuable piece of truth. Mocks and stubs are both types of test doubles. I can understand that. That’s a piece of knowledge that’s not likely to be invalidated by something else I’ll read later. My understanding has permanently advanced a little bit.

What’s a test double? The book xUnit Test Patterns (which I understand actually coined the term “test double”) likens a test double to a Hollywood stunt double. From the book’s online explanation of test doubles:

When the movie industry wants to film something that is potentially risky or dangerous for the leading actor to carry out, they hire a “stunt double” to take the place of the actor in the scene. The stunt double is a highly trained individual who is capable of meeting the specific requirements of the scene. They may not be able to act, but they know how to fall from great heights, crash a car, or whatever the scene calls for. How closely the stunt double needs to resemble the actor depends on the nature of the scene. Usually, things can be arranged such that someone who vaguely resembles the actor in stature can take their place.

The example I use later in this post is the example of a payment gateway. When we’re testing some code that interacts with a payment gateway, we of course don’t want our test code to actually hit e.g. the production Stripe API and charge people’s credit cards. We want to use a test double in place of the real payment gateway. The production Stripe API is like Tom Cruise in Mission Impossible, the payment gateway test double is of course like the stunt double.

Stub explanation

The best explanation of mocks and stubs I’ve been able to find online is this post by a guy named Michal Lipski.

I took his explanation, mixed it in my brain with other stuff I’ve read, and came up with this explanation:

A Test Stub is a fake object that’s used in place of a real object for the purpose of getting the program to behave the way we need it to in order to test it. A big part of a Test Stub’s job is to return pre-specified hard-coded values in response to method calls.

You can visit the xUnit Patterns Test Stub page for a more detailed and precise explanation. My explanation might not be 100% on the mark. I’m going for clarity over complete precision.

When would you want to use a test stub? We’ll see in my example code shortly.

Mock explanation

A Mock Object is a fake object that’s used in place of a real object for the purpose of listening to the methods called on the mock object. The main job of a Mock Object is to ensure that the right methods get called on it.

Again, for a more precise (but harder to understand) explanation, you can check out the xUnit Patterns Mock Object page.

We’ll also see a mock object use case in my example code.

The difference between mocks and stubs

As I understand it, and to paint with a very broad brush, Test Stubs help with inputs and Mock Objects help with outputs. A Test Stub is a fake thing you stick in there to trick your program into working properly under test. A Mock Object is a fake thing you stick in there to spy on your program in the cases where you’re not able to test something directly.

Again, I’m going for conciseness and clarity over 100% accuracy here. Now let’s take a look at a concrete example.

Example application code

Below is an example Ruby program I wrote. I tried to write it to meet the following conditions:

• It’s as small and simple as possible
• It would actually benefit from the use of one mock and one stub

My program involves three classes:

• Payment: meant to simulate an ActiveRecord model
• PaymentGateway: simulates a third-party payment gateway (e.g. Stripe)
• Logger: logs payments

The code snippet below includes the program’s three classes as well as a single test case for the Payment class.

class Payment
  attr_accessor :total_cents

  def initialize(payment_gateway, logger)
    @payment_gateway = payment_gateway
    @logger = logger
  end

  def save
    response = @payment_gateway.charge(total_cents)
    @logger.record_payment(response[:payment_id])
  end
end

class PaymentGateway
  def charge(total_cents)
    puts "THIS HITS THE PRODUCTION API AND ALTERS PRODUCTION DATA. THAT'S BAD!"

    { payment_id: rand(1000) }
  end
end

class Logger
  def record_payment(payment_id)
    puts "Payment id: #{payment_id}"
  end
end

describe Payment do
  it 'records the payment' do
    payment_gateway = PaymentGateway.new
    logger = Logger.new

    payment = Payment.new(payment_gateway, logger)
    payment.total_cents = 1800
    payment.save
  end
end

Our test has two problems.

One is that it hits the real production API and alters production data. (My code doesn’t really do this; you can pretend that my puts statement hits a real payment gateway and charges somebody’s credit card.)

The other problem is that the test doesn’t verify that the payment gets logged. We could comment out the @logger.record_payment(response[:payment_id]) line and the test would still pass.

This is what we see when we run the test:

THIS HITS THE PRODUCTION API AND ALTERS PRODUCTION DATA. THAT'S BAD!
Payment id: 302
.

Finished in 0.00255 seconds (files took 0.09594 seconds to load)
1 example, 0 failures

Let’s first address the problem of altering production data. We can do this by replacing PaymentGateway with a special kind of test double, a stub.

Stub example

Below I’ve replaced payment_gateway = PaymentGateway.new with payment_gateway = double(). I’m also telling my new Test Double object (that is, my Test Stub) that it should expect to receive a charge method call, and when it does, return a payment id of 1234.

class Payment
  attr_accessor :total_cents

  def initialize(payment_gateway, logger)
    @payment_gateway = payment_gateway
    @logger = logger
  end

  def save
    response = @payment_gateway.charge(total_cents)
    @logger.record_payment(response[:payment_id])
  end
end

class PaymentGateway
  def charge(total_cents)
    puts "THIS HITS THE PRODUCTION API AND ALTERS PRODUCTION DATA. THAT'S BAD!"

    { payment_id: rand(1000) }
  end
end

class Logger
  def record_payment(payment_id)
    puts "Payment id: #{payment_id}"
  end
end

describe Payment do
  it 'records the payment' do
    payment_gateway = double()
    allow(payment_gateway).to receive(:charge).and_return(payment_id: 1234)

    logger = Logger.new

    payment = Payment.new(payment_gateway, logger)
    payment.total_cents = 1800
    payment.save
  end
end

If we run this test now, we can see that we no longer get the jarring “THIS HITS THE PRODUCTION API” message. This is because our test no longer calls the charge method on an instance of PaymentGateway, it calls the charge method on a test double.

Our payment_gateway variable is no longer actually an instance of PaymentGateway, it’s an instance of RSpec::Mocks::Double.

Payment id: 1234
.

Finished in 0.00877 seconds (files took 0.09877 seconds to load)
1 example, 0 failures

That takes care of the hitting-production problem. What about verifying the logging of the payment? This is a job for a different kind of test double, a mock object (or just mock).

Mock example

Now let’s replace Logger.new with logger = double(). Notice how RSpec doesn’t make a distinction between mocks and stubs. They’re all just Test Doubles. If we want to use a Test Double as a mock or as a stub, RSpec leaves that up to us and doesn’t care.

We’re also telling our new Mock Object that it needs (not just can, but has to, and it will raise an exception if not) receive a record_payment method call with the value 1234.

class Payment
  attr_accessor :total_cents

  def initialize(payment_gateway, logger)
    @payment_gateway = payment_gateway
    @logger = logger
  end

  def save
    response = @payment_gateway.charge(total_cents)
    @logger.record_payment(response[:payment_id])
  end
end

class PaymentGateway
  def charge(total_cents)
    puts "THIS HITS THE PRODUCTION API AND ALTERS PRODUCTION DATA. THAT'S BAD!"

    { payment_id: rand(1000) }
  end
end

class Logger
  def record_payment(payment_id)
    puts "Payment id: #{payment_id}"
  end
end

describe Payment do
  it 'records the payment' do
    payment_gateway = double()
    allow(payment_gateway).to receive(:charge).and_return(payment_id: 1234)

    logger = double()
    expect(logger).to receive(:record_payment).with(1234)

    payment = Payment.new(payment_gateway, logger)
    payment.total_cents = 1800
    payment.save
  end
end

Now that we have the line that says expect(logger).to receive(:record_payment).with(1234), our test is asserting that the payment gets logged. We can verify this by commenting out the @logger.record_payment(response[:payment_id]) and running our test again. We get the following error:

Failures:

  1) Payment records the payment
     Failure/Error: expect(logger).to receive(:record_payment).with(1234)
     
       (Double (anonymous)).record_payment(1234)
           expected: 1 time with arguments: (1234)
           received: 0 times

Why I don’t often use mocks or stubs in Rails

Having said all this, I personally hardly ever use mocks or stubs in Rails. I can count on one hand all the times I’ve used mocks or stubs over my eight years so far of doing Rails.

The main reason is that I just don’t often have the problems that test doubles solve. In Rails we don’t really do true unit tests. For better or worse, I’ve never encountered a Rails developer, myself included, who truly wants to test an object completely in isolation from all other objects. Instead we write model tests that hit the database and have full access to every single other object in the application. Whether this is good or bad can be debated but one thing seems clear to me: this style of testing eliminates the need for test doubles the vast majority of the time.

Another reason is that I find that tests written using test doubles are often basically just a restatement of the implementation of whatever’s being tested. The code says @logger.record_payment and the test says expect(logger).to receive(:record_payment). Okay, what did we really accomplish? We’re not testing the result, we’re testing the implementation. That’s probably better than nothing but if possible I’d rather just test the result, and quite often there’s nothing stopping me from testing the result instead of the implementation.

Lastly, I personally haven’t found myself working on a lot of projects that use some external resource like a payment gateway API that would make test doubles useful. If I were to work on such a project I imagine I certainly would make use of test doubles, I just haven’t worked on that type of project very much.

I used to prefer seeing my system specs run in the browser but lately I’ve been preferring to run them headlessly. It’s a little faster that way and I find it a little less disruptive.

Sometimes I still find myself wanting to see a certain test in the browser though. Seeing the test run in the browser can make diagnosis of any problems much easier.

The dream

What would be ideal is if I could do something like the following. To run a spec headlessly, I could do this:

$ rspec spec/system/create_customer_spec.rb

To see the same spec run in the browser, I could do this:

$ SHOW_BROWSER=true rspec spec/system/create_customer_spec.rb

Let’s take a look at how to turn that dream into reality.

The implementation

The desired functionality can be implemented by adding the following to spec/rails_helper.rb:

# spec/rails_helper.rb

RSpec.configure do |config|
  config.before(:each, type: :system) do
    driven_by ENV['SHOW_BROWSER'] ? :selenium_chrome : :selenium_chrome_headless
  end
end

Now, for all system specs, Capybara will use the :selenium_chrome_headless driver normally and the :selenium_chrome driver when SHOW_BROWSER=true is specified.

Thanks to Reddit user jrochkind for helping me improve this solution.