A VCR + WebMock “hello world” tutorial

by Jason Swett,

Introduction

VCR and WebMock are tools that help deal with challenges related to tests that make network requests. In this post I’ll explain what VCR and WebMock are and then show a “hello world”-level example of using the two tools.

Why VCR and WebMock exist

Determinism

One of the principles of testing is that tests should be deterministic. The passing or failing of a test should be determined by the content of the application code and nothing else. All the tests in the test suite should pass regardless of what order they were run in, what time of day they were run, or any other factor.

For this reason we have to run tests in their own encapsulated world. If we want determinism, we can’t let tests talk to the network, because the network (and things in the network) are susceptible to change and breakage.

Imagine an app that talks to a third-party API. Imagine that the tests hit that API each time they’re run. Now imagine that on one of the test runs, the third-party API happens to go down for a moment, causing our tests to fail. Our test failure is illogical because the tests are telling us our code is broken, but it’s not our code that’s broken, it’s the outside world that’s broken.

If we’re not to talk to the network, we need a way to simulate our interactions with the network so that our application can still behave normally. This is where tools like VCR and WebMock come in.

Production data

We also don’t want tests to alter actual production data. It would obviously be bad if we for example wrote a test for deleting users and then that test deleted real production users. So another benefit of tools like VCR and WebMock is that they save us from having to touch real production data.

The difference between VCR and WebMock

VCR is a tool that will record your application’s HTTP interactions and play them back later. Very little code is necessary. VCR tricks your application into thinking it’s receiving responses from the network when really the application is just receiving prerecorded VCR data.

WebMock, on the other hand, has no feature for recording and replaying HTTP interactions in the way that VCR does, although HTTP interactions can still be faked. Unlike VCR’s record/playback features, WebMock’s network stubbing is more code-based and fine-grained. In this tutorial we’re going to take a very basic look at WebMock and VCR to show a “hello world” level usage.

What we’re going to do

This post will serve as a simple illustration of how to use VCR and WebMock to meet the need that these tools were designed to meet: running tests that hit the network without actually hitting the network.

If you’d like to follow along with this tutorial, I’d suggest first setting up a Rails app for testing according to the instructions in this post.

The scenario

We’re going to write a small search feature that hits a third-party API. We’re also going to write a test that exercises that search feature and therefore hits the third-party API as well.

WebMock

Once we have our test in place we’ll install and configure WebMock. This will disallow any network requests. As a result, our test will stop working.

VCR

Lastly, we’ll install and configure VCR. VCR knows how to talk with WebMock. Because of this, VCR and WebMock can come to an agreement together that it’s okay for our test to hit the network under certain controlled conditions. VCR will record the HTTP interactions that occur during the test and then, on any subsequent runs of the test, VCR will use the recorded interactions rather than making fresh HTTP interactions for each run.

The feature

The feature we’re going to write for this tutorial is one that searches the NPI registry, a government database of healthcare providers. The user can type a provider’s first and last name, hit Search, and then see any matches.

Below is the controller code.

# app/controllers/npi_searches_controller.rb

ENDPOINT_URL = "https://npiregistry.cms.hhs.gov/api"
TARGET_VERSION = "2.1"

class NPISearchesController < ApplicationController
  def new
    @results = []
    return unless params[:first_name].present? || params[:last_name].present?

    query_string = {
      first_name: params[:first_name],
      last_name: params[:last_name],
      version: TARGET_VERSION,
      address_purpose: ""
    }.to_query

    uri = URI("#{ENDPOINT_URL}/?#{query_string}")
    response = Net::HTTP.get_response(uri)
    @results = JSON.parse(response.body)["results"]
  end
end

Here’s the template that goes along with this controller action.

<!-- app/views/npi_searches/new.html.erb -->

<h1>New NPI Search</h1>

<%= form_tag new_npi_search_path, method: :get do %>
  <%= text_field_tag :first_name, params[:first_name], class: "p-1" %>
  <%= text_field_tag :last_name, params[:last_name], class: "p-1" %>
  <%= submit_tag "Search", class: "p-1" %>
<% end %>

<% @results.each do |result| %>
  <div>
    <%= result["basic"]["name_prefix"] %>
    <%= result["basic"]["first_name"] %>
    <%= result["basic"]["last_name"] %>
    <%= result["number"] %>
  </div>
<% end %>

The test

Our test will be a short system spec that types “joel” and “fuhrman” into the first and last name fields respectively, clicks Search, then asserts that Joel Fuhrman’s NPI code (a unique identifier for a healthcare provider) shows up on the page.

# spec/system/npi_search_spec.rb

require "rails_helper"

RSpec.describe "NPI search", type: :system do
  it "shows the physician's NPI number" do
    visit new_npi_search_path
    fill_in "first_name", with: "joel"
    fill_in "last_name", with: "fuhrman"
    click_on "Search"

    # 1386765287 is the NPI code for Dr. Joel Fuhrman
    expect(page).to have_content("1386765287")
  end
end

If we run this test at this point, it passes.

Installing and Configuring WebMock

We don’t want our tests to be able to just make network requests willy-nilly. We can install WebMock so that no HTTP requests can be made without our noticing.

First we add the webmock gem to our Gemfile.

# Gemfile

group :development, :test do
  gem "webmock"
end

Second, we can create a new file at spec/support/webmock.rb.

# spec/support/webmock.rb

# This line makes it so WebMock and RSpec know how to talk to each other.
require "webmock/rspec"

# This line disables HTTP requests, with the exception of HTTP requests
# to localhost.
WebMock.disable_net_connect!(allow_localhost: true)

Remember that files in spec/support won’t get loaded unless you have the line in spec/rails_helper.rb uncommented that loads these files.

# spec/rails_helper.rb

# Make sure to uncomment this line
Dir[Rails.root.join('spec', 'support', '**', '*.rb')].sort.each { |f| require f }

Seeing the test fail

If we run our test again now that WebMock is installed, it will fail, saying “Real HTTP connections are disabled”. We also get some instructions on how to stub this request if we like. We’re not going to do that, though, because we’re going to use VCR instead.

Failures:

  1) NPI search shows the physician's NPI number
   Failure/Error: response = Net::HTTP.get_response(uri)

   WebMock::NetConnectNotAllowedError:
     Real HTTP connections are disabled. Unregistered request: GET https://npiregistry.cms.hhs.gov/api/?address_purpose=&first_name=joel&last_name=fuhrman&version=2.1 with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Host'=>'npiregistry.cms.hhs.gov', 'User-Agent'=>'Ruby'}

     You can stub this request with the following snippet:

     stub_request(:get, "https://npiregistry.cms.hhs.gov/api/?address_purpose=&first_name=joel&last_name=fuhrman&version=2.1").
       with(
         headers: {
           'Accept'=>'*/*',
           'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
           'Host'=>'npiregistry.cms.hhs.gov',
           'User-Agent'=>'Ruby'
           }).
         to_return(status: 200, body: "", headers: {})

       ============================================================

Installing and Configuring VCR

First we’ll add the vcr gem to our Gemfile.

# Gemfile

group :development, :test do
  gem 'vcr'
end

Next we’ll add the following config file. I’ve added annotations so you can understand what each line is.

# spec/support/vcr.rb

VCR.configure do |c|
  # This is the directory where VCR will store its "cassettes", i.e. its
  # recorded HTTP interactions.
  c.cassette_library_dir = "spec/cassettes"

  # This line makes it so VCR and WebMock know how to talk to each other.
  c.hook_into :webmock

  # This line makes VCR ignore requests to localhost. This is necessary
  # even if WebMock's allow_localhost is set to true.
  c.ignore_localhost = true

  # ChromeDriver will make requests to chromedriver.storage.googleapis.com
  # to (I believe) check for updates. These requests will just show up as
  # noise in our cassettes unless we tell VCR to ignore these requests.
  c.ignore_hosts "chromedriver.storage.googleapis.com"
end

Adding VCR to our test

Now we can add VCR to our test by adding a block to it that starts with VCR.use_cassette "npi_search" do. The npi_search part is just arbitrary and tells VCR what to call our cassette.

# spec/system/npi_search_spec.rb

require "rails_helper"

RSpec.describe "NPI search", type: :system do
  it "shows the physician's NPI number" do
    VCR.use_cassette "npi_search" do # <---------------- add this
      visit new_npi_search_path
      fill_in "first_name", with: "joel"
      fill_in "last_name", with: "fuhrman"
      click_on "Search"

      expect(page).to have_content("1386765287")
    end
  end
end

Last time we ran this test it failed because WebMock was blocking its HTTP request. If we run the test now, it will pass, because VCR and WebMock together are allowing the HTTP request to happen.

If we look in the spec/cassettes directory after running this test, we’ll see that there’s a new file there called npi_search.yml. Its contents look like the following.

---
http_interactions:
- request:
    method: get
    uri: https://npiregistry.cms.hhs.gov/api/?address_purpose=&first_name=joel&last_name=fuhrman&version=2.1
    body:
      encoding: US-ASCII
      string: ''
    headers:
      Accept-Encoding:
      - gzip;q=1.0,deflate;q=0.6,identity;q=0.3
      Accept:
      - "*/*"
      User-Agent:
      - Ruby
      Host:
      - npiregistry.cms.hhs.gov
  response:
    status:
      code: 200
      message: OK
    headers:
      Date:
      - Sat, 17 Apr 2021 11:13:41 GMT
      Content-Type:
      - application/json
      Strict-Transport-Security:
      - max-age=31536000; includeSubDomains
      Set-Cookie:
      - TS017b4e40=01acfeb9489bd3c233ef0e8a55b458849e619bdc886c02193c4772ba662379fa1f8493887950c06233f28bbbaac373afba8b58b00f;
        Path=/; Domain=.npiregistry.cms.hhs.gov
      Transfer-Encoding:
      - chunked
    body:
      encoding: UTF-8
      string: '{"result_count":1, "results":[{"enumeration_type": "NPI-1", "number":
        1386765287, "last_updated_epoch": 1183852800, "created_epoch": 1175472000,
        "basic": {"name_prefix": "DR.", "first_name": "JOEL", "last_name": "FUHRMAN",
        "middle_name": "H", "credential": "MD", "sole_proprietor": "YES", "gender":
        "M", "enumeration_date": "2007-04-02", "last_updated": "2007-07-08", "status":
        "A", "name": "FUHRMAN JOEL"}, "other_names": [], "addresses": [{"country_code":
        "US", "country_name": "United States", "address_purpose": "LOCATION", "address_type":
        "DOM", "address_1": "4 WALTER E FORAN BLVD", "address_2": "SUITE 409", "city":
        "FLEMINGTON", "state": "NJ", "postal_code": "088224664", "telephone_number":
        "908-237-0200", "fax_number": "908-237-0210"}, {"country_code": "US", "country_name":
        "United States", "address_purpose": "MAILING", "address_type": "DOM", "address_1":
        "4 WALTER E FORAN BLVD", "address_2": "SUITE 409", "city": "FLEMINGTON", "state":
        "NJ", "postal_code": "088224664", "telephone_number": "908-237-0200", "fax_number":
        "908-237-0210"}], "taxonomies": [{"code": "207Q00000X", "desc": "Family Medicine",
        "primary": true, "state": "NJ", "license": "25MA05588600"}], "identifiers":
        []}]}'
  recorded_at: Sat, 17 Apr 2021 11:13:41 GMT
recorded_with: VCR 6.0.0

Each time this test is run, VCR will ask, “Is there a cassette called npi_search?” If not, VCR will allow the HTTP request to go out, and a new cassette will be recorded for that HTTP request. If there is an existing cassette called npi_search, VCR will block the HTTP request and just use the recorded cassette in its place.

Takeaways

  • Tests should be deterministic. The passing or failing of a test should be determined by the content of the application code and nothing else.
  • We don’t want tests to be able to alter production data.
  • WebMock can police our app to stop it from making external network requests.
  • VCR can record our tests’ network interactions for playback later.

2 thoughts on “A VCR + WebMock “hello world” tutorial

Leave a Reply

Your email address will not be published. Required fields are marked *