Category Archives: Programming

What are the different kinds of Rails tests and when should I use each?

2 Replies

When starting out with Rails testing, it’s hard to know where to start.

First, there’s the decision of which framework to use. Then, if you’ve chosen RSpec (which most people do), you’re presented with a bewildering set of possible test types to use.

In this post I’ll show you what types of tests there are. I’ll show you which ones you should use and which ones you can ignore. Since most commercial Rails projects use RSpec, I’m going to focus on the eight types of tests that the RSpec library offers. (Although if I were to use Minitest, my strategy regarding test types would be pretty much the same.)

The eight types of RSpec specs

The RSpec library offers a lot of different spec types.

  • Model specs
  • System specs/feature specs*
  • Request specs/controller specs*
  • Helper specs
  • View specs
  • Routing specs
  • Mailer specs
  • Job specs

There are two lines with asterisks. These are cases where the RSpec team decreed one spec type obsolete and replaced it with a new type. I’m only including those ones for completeness.

So the up-to-date list is really the following.

  • Model specs
  • System specs
  • Request specs
  • Helper specs
  • View specs
  • Routing specs
  • Mailer specs
  • Job specs

Here’s when I use each.

  • Model specs – always
  • System specs – always
  • Request specs – rarely
  • Helper specs – rarely
  • View specs – never
  • Routing specs – never
  • Mailer specs – never
  • Job specs – never

Let’s talk about each of these spec types in detail. I’ll explain why I use the ones I use and why I ignore the ones I ignore.

Spec types I always use

Believe it or not, the overwhelming majority of the Rails tests I write make use of just two of the eight different spec types offered by RSpec. You might think that this would leave large gaps in my test coverage but it doesn’t. My test coverage is consistently above 95%.

System specs

System specs are “high-level” tests that simulate a user’s keystrokes and mouse clicks. System specs literally open up a browser window (although perhaps an invisible browser window if the tests are run “headlessly”) and use certain tools to manipulate the browser to exercise your application through simulated user input.

The reason I find system specs so valuable is that they test my whole stack, not just a slice of it, and they test my application in the same exact way that a real user will be using it. System specs are the only type of test that give me confidence my whole application really works.

I write so many system specs that I’ve developed a repeatable formula for adding system specs to any new CRUD feature.

Model specs

Even though system specs are indispensable, they’re not without drawbacks. System specs are somewhat “heavy”. They’re often more work to write and more expensive to run than other types of tests. For this reason I like to cover my features with a small number of coarse-grained system specs and a comparatively large number of fine-grained model specs.

As the name implies, model specs are for testing models. I tend to only bring model specs into the picture once a model has reached a certain level of “maturity”. At the beginning of a model’s life, it might have all its needs covered by built-in Rails functionality and not need any methods of its own. Some people write tests for things like associations and validations but I don’t because I find those types of tests to be pointless.

I use model specs to test my models’ methods. When I do so, I tend to use a test-first approach and write a failing test before I add a new line of code so that I’m sure every bit of code in my model is covered by a test.

Spec types I rarely use

Request specs

Request specs are more or less a way to test controller actions in isolation. I tend not to use request specs much because in most cases they would be redundant to my system specs. If I have system specs covering all my features, then of course a broken controller would fail one or more of my tests, making tests specifically for my controllers unnecessary.

I also try to keep my controllers sufficiently simple as to not call for tests of their own.

There are just three scenarios in which I do use request specs. First: If I’m working on a legacy project with fat controllers, sometimes I’ll use request specs to help me harness and refactor all that controller code. Second: If I’m working on an API-only Rails app, then system specs are physically impossible and I drop down to request specs instead. Lastly, if it’s just too awkward or expensive to use a system spec in a certain case then I’ll use a request spec instead. I write more about my reasoning here.

Helper specs

The reason I rarely write helper specs is simple: I rarely write helpers.

Spec types I never use

View specs and routing specs

I find view specs and routing specs to be redundant to system specs. If something is wrong with one of my views or routes, it’s highly likely that one of my system specs will catch the problem.

Mailer specs and job specs

I don’t write mailer specs or job specs because I try very hard to make all my mailers and background jobs one-liners (or close). I don’t think mailers and background jobs should do things, I think they should only call things. This is because mailers and background jobs are mechanical devices, not code organization devices.

To test my mailers and background jobs, I put their code into a PORO model and write tests for that PORO.

Takeaways

RSpec offers a lot of different spec types but you can typically meet 98% of your needs with just system specs and model specs.

If you’re a total beginner, I’d suggest starting with system specs.

What level of test coverage should I shoot for?

1 Reply

“What level of test coverage should I shoot for?” is one of the questions most commonly asked by beginners to Rails testing.

My answer is that you shouldn’t shoot for a particular level of test coverage. I recommend that instead you make testing a habitual part of your development workflow. A healthy level of test coverage will flow from there.

I also want to address why people ask this question. I think people ask this because they want some way of knowing whether they’re testing “enough” or doing testing “right”. Test coverage is one way of measuring this but I think there are better, more meaningful ways.

I think that if you’re feeling the kinds of pains that missing tests leave in their absence, then you need more tests. If you’re not feeling those kinds of pains, then you’re good.

Pains that tell you your test coverage might be insufficient

Too many bugs

This is the obvious one. All software has bugs, but if you feel like the rate of new bugs appearing in production is unacceptably high, it may be a symptom of too little test coverage.

Too much manual testing

This is another fairly obvious one. The only alternative to using automated tests, aside from not testing at all, is to test manually.

Some level of manual testing is completely appropriate. Automated tests can never replace, for example, exploratory testing done by a human. But humans should only carry out the testing that can’t be done better by a computer. Otherwise testing is much more expensive and time-consuming than it needs to be.

Infrequent deployments

Infrequent deployments can arise as a symptom of too few tests for a couple different reasons.

One possible reason is that the need for manual testing bottlenecks the deployment timing. If it takes two days for manual testers to do a full regression test on the application, you can of course only deploy a fully-tested version of your application once every two days at maximum. (And this is assuming the test suite passes every time, which is not typically the case.)

Another possible reason for infrequent deployments is the following logic: things go wrong every time we deploy, therefore things will go wrong less often if we deploy less often, so let’s deploy less often. Unfortunately this decision means that problems pile up and get introduced to production all at once on each deployment instead of getting sprinkled lightly over time.

With the presence of a good test suite, deployments can happen many times a day instead of just once every few weeks or months.

Inability to refactor or make big changes

When a particular change has a small footprint, manual testing is usually good enough (although of course sometimes changes that seem like they’d have small footprints cause surprising regressions in distant areas).

When a change has a large footprint, like a Rails version upgrade or a broad refactoring, it’s basically impossible to gain sufficient confidence of the safety of the change without having a solid automated test suite. So on codebases without good test coverage, these types of improvements tend not to happen.

Poor code quality

As I’ve written elsewhere, it’s not possible to have clean, understandable code without having tests.

The reason is that refactoring is required in order to have good code and automated tests are required in order to do sufficient refactoring.

Diminished ability to hire and retain talent

Lastly, it can be hard to attract and retain high-quality developers if you lack tests and you’re suffering from the ailments that result from having poor test coverage.

If a job candidate asks detailed questions about your development practices or the state of your codebase, he or she might develop a negative perception of your organization relative to the other organizations where he or she is interviewing. All other things being equal, a sophisticated and experienced engineer is probably more likely to pick some other organization that does write tests over yours which doesn’t.

Even if you manage to get good people on your team, you might have trouble keeping them. It’s painful to live with all the consequences of not having tests. Your smartest people are likely to be the most sensitive to these pains, and they may well seek somewhere else to work where the development experience is more pleasant.

The takeaway

I don’t think test coverage is a particularly meaningful way to tell whether you’re testing enough. Instead, assess the degree to which you’re suffering from the above symptoms of not having enough tests. Your degree of suffering is probably proportionate to your need for more tests.

If you do this, then “good” coverage numbers are likely to follow. Last time I checked my main codebase at work my test coverage level was 96.47%.

Which test framework should I learn, RSpec or Minitest?

5 Replies

A common Rails testing question is which testing framework to use. RSpec and Minitest are the two that most people are deciding between. To many beginners it’s not clear which is the better choice.

We could weigh the technical pros and cons of each framework. Many people find things to love and hate about both RSpec and Minitest. You can find some passionate flame wars online if you look.

But before we get into all that, there are some realities to consider that overshadow the relative technical merits of the two frameworks. There are two particular facts we should think about.

Fact #1: usually, someone else decides for you

Most of us don’t have much choice as to whether to use RSpec or Minitest at work.

At some point we’ll get a job. At that job they’ll either use RSpec there or Minitest (or something else or nothing at all). Whatever they use at work, that’s what we’ll be using. Our personal preferences are moot.

Fact #2: usually, they’ve chosen RSpec

For better or worse, it’s my experience and the experience of most Rails developers I’ve talked with that most commercial projects use RSpec. (Note how I said most commerical projects. Most commercial projects use RSpec and most OSS Ruby projects, in my experience, use Minitest. I do not know why this is the way it is.)

Out of curiosity I did a (totally unscientific) poll regarding which test framework they use at work. Take it with a grain of salt, but here are the results.

Even if my numbers are off by quite a bit, RSpec is still the more popular framework.

What does this mean?

My take is that this means if your goal is to get a Rails job, learning RSpec over Minitest will give you a higher probability that your skills match the tech stack that’s used at any particular company.

Some people may object to this way of looking at it. They might argue that if you always you go with whatever’s most popular instead of what’s the best technical choice, you may end up using a Windows laptop or switching from Rails to Node.js.

This argument is flawed though. We’re free to make our own choices on the big things but we can’t dictate what comes along with those choices. We can choose to use Rails instead of a different framework, but we can’t reasonably say that we’re only going to work on Rails projects that use, for example, Minitest and MySQL and Angular and no other combination of technologies. We have to compromise a little or face extremely limited job options.

Also, it doesn’t matter much

Having said all that, I actually don’t believe your choice of which test framework to learn matters!

RSpec and Minitest differ syntactically but they don’t really have meaningful conceptual differences. The principles of testing are the same regardless of which test framework you’re using, or even which language you’re using for that matter.

You’re very unlikely to become an expert in Minitest and then get turned down for a job because they use RSpec there, or vice versa. Employers typically realize that if someone is skilled with testing, they’ll be able to pick up any test framework relatively easily.

So try both

In a sense it might sound depressing that the answer to the RSpec/Minitest question is a) we don’t have a choice and b) it doesn’t matter anyway. I actually find these facts freeing.

If the choice between RSpec and Minitest doesn’t matter that much then we’re free to evaluate both according to our own independent judgment and taste and not worry about whether we’re making the “right” choice. Whatever we choose, we’re likely to develop skills that will apply to any job, whether they use Minitest or RSpec there.

So my advice is to try both frameworks and see which one you like better. Neither one is objectively superior to the other.

But if you just want me to pick for you, I say RSpec

My very simplistic logic is that RSpec is what you’ll most likely be forced to use at work, so that’s what you might as well learn.

But again, I encourage you to try both and decide for yourself. This is ultimately not a very important decision. Learning testing principles is much more important than learning testing frameworks.

How I set up a Rails application for testing

2 Replies

Below is how I set up a fresh Rails application for testing. I’ll describe it in three parts:

  1. An application template that can add all the necessary gems and configuration
  2. My setup process (commands I run to create a new Rails app)
  3. A breakdown of the gems I use

Let’s start with the application template.

My application template

First, if you don’t know, it’s possible to create a file called an application template that you can use to create a Rails application with certain code or configuration included. This is useful if you create a lot of new Rails applications with parts in common.

Here’s an application template I created that will do two things: 1) install a handful of testing-related gems and 2) add a config file that will tell RSpec not to generate certain types of files. A more detailed explanation can be found below the code.

gem_group :development, :test do
  gem 'rspec-rails'
  gem 'factory_bot_rails'
  gem 'capybara'
  gem 'webdrivers'
  gem 'faker'
end

initializer 'generators.rb', <<-CODE
  Rails.application.config.generators do |g|
    g.test_framework :rspec,
      fixtures:         false,
      view_specs:       false,
      helper_specs:     false,
      routing_specs:    false,
      request_specs:    false,
      controller_specs: false
  end
CODE

The first chunk of code will add a certain set of gems to my Gemfile. A more detailed explanation of these gems is below.

The second chunk of code creates a file at config/initializers/generators.rb. The code in the file says “when a scaffold is generated, don’t generate files for fixtures, view specs, helper specs, routing specs, request specs or controller specs”. There are certain kinds of tests I tend not to write and I don’t want to clutter up my codebase with a bunch of empty files. That’s not to say I never write any of these types of tests, just sufficiently rarely that it makes more sense for me to create files manually in those cases than for me to allow files to get generated every single time I generate a scaffold.

The setup process

When I run rails new, I always use the -T flag for “skip test files” because I always use RSpec instead of the Minitest that Rails comes with by default.

Also, incidentally, I always use PostgreSQL. This choice of course has little to do with testing but I’m including it for completeness.

In this particular case I’m also using the -m flag so I can pass in my application template. Application templates can be specified using either a local file path or a URL. In this case I’m using a URL so that you can just copy and paste my full rails new command as-is if you want to.

$ rails new my_project -T -d postgresql \
  -m https://raw.githubusercontent.com/jasonswett/testing_application_template/master/application_template.rb

Once I’ve created my project, I add it to version control. (I could have configured my application template to do this step manually, but I wanted to explicitly show it as a separate step, partially to keep the application template clean and easily understandable.)

$ git add .
$ git commit -a -m'Initial commit'

The gems

Here’s an explanation of each gem I chose to add to my project.

rspec-rails

RSpec is one of the two most popular test frameworks for Rails, the other being Minitest.

The rspec-rails gem is the version of the RSpec gem that’s specifically fitted to Rails.

factory_bot_rails

Factory Bot is a tool for generating test data. Most Rails projects that use RSpec also use Factory Bot.

Like rspec-rails, factory_bot_rails is a Rails-specific version of a more general gem, factory_bot.

capybara

Capybara is a tool for writing acceptance tests, i.e. tests that interact with the browser and simulate clicks and keystrokes.

The underlying tool that allows us to simulate user input in the browser is called Selenium. Capybara allows us to control Selenium using Ruby.

webdrivers

In order for Selenium to work with a browser, Selenium needs drivers. There are drivers for Chrome, drivers for Edge, etc. Unfortunately it can be somewhat tedious to keep the drivers up to date. The webdrivers gem helps with this.

faker

By default, Factory Bot (the tool for generating test data) will give us factories that look something like this:

FactoryBot.define do
  factory :customer do
    first_name { "MyString" }
    last_name { "MyString" }
    email { "MyString" }
  end
end

This is fine for just one record but becomes a problem if we have multiple records plus a unique constraint. If in this example we require each customer to have a unique email address, then we’ll get a database error when we create two customer records because the email address of MyString will be a duplicate.

One possible solution to this problem is to replace the instances of "MyString" with something like SecureRandom.hex. I don’t like this, though, because I often find it helpful if my test values resemble the kinds of values they’re standing in for. With Faker, I can do something like this:

FactoryBot.define do
  factory :customer do
    first_name { Faker::Name.first_name }
    last_name { Faker::Name.last_name }
    email { Faker::Internet.email }
  end
end

This can make test problems easier to troubleshoot than when test values are simply random strings like c1f83cef2d1f74f77b88c9740cfb3c1e.

Honorable mention

I also often end up adding the VCR and WebMock gems when I need to test functionality that makes external network requests. But in general I don’t believe in adding code or libraries speculatively. I only add something once I’m sure I need it. So I typically don’t include VCR or WebMock in a project from the very beginning.

Next steps

After I initialize my Rails app, I usually create a walking skeleton by deploying my application to a production and staging environment and adding one small feature, for example the ability to sign in. Building the sign-in feature will prompt me to write my first tests. By working in this way I front-load all the difficult and mysterious work of the project’s early life so that from that point on, my development work is mostly just incremental.

If you’re brand new to Rails testing and would like to see an example of how I would actually write a test once I have the above application set up, I might recommend my Rails testing “hello world” post.

How I make sure I really understand a feature before building it

Leave a reply

The very real danger of misinterpreting a feature requirement

When I was a young and inexperienced developer I often failed to build a feature the right way the first time.

My memory is that, often, I would show what I built to my stakeholder (stakeholder meaning boss or freelance client) and the stakeholder would say that what I built didn’t really resemble what we had talked about.

I would then go away, embarrassed and surprised, and make another attempt. Sometimes this process would be repeated multiple times before we finally got on the same page. It sucked. It was obviously a bad experience for both myself and my stakeholder. Yet, unfortunately, this was the way things normally went.

The glory of getting it right the first time

I don’t have this problem anymore. Today, when I build a feature, I typically do get it right the first time.

It’s true that the feature still sometimes has to be tweaked after the first pass, but this is usually not because I misunderstood my stakeholder, but because neither myself nor my stakeholder could imagine the ideal version of the feature without seeing a rough draft first. Today I almost never have the “this isn’t what we talked about” problem.

So what changed? How did I go from being so terrible at understanding feature requirements to being so good at it? I’ll tell you, but first let me explain the reasons why it’s so hard to understand a feature’s requirements.

Why it’s so hard to understand a feature’s requirements the first time

It’s hard to communicate things

Conveying things in high fidelity from one person’s mind to another is notoriously hard. It’s basically impossible to say something of any complexity to another person and have them understand exactly what you mean.

Imagine you’ve never been to Paris before. Then you schedule a trip to Paris. Some people tell you in detail what kind of stuff is in Paris. You read some books about the place. Then, when you actually get to Paris, much of the experience is nothing like you imagined.

And Paris actually exists! Think of how much harder it gets when the thing being communicated is just an idea in someone’s mind. This brings me to the next reason why it’s hard to understand a feature’s requirements the first time.

Sometimes the stakeholder doesn’t even understand what the stakeholder means

Just because someone describes a feature requirement to you doesn’t mean that that feature idea is complete, self-consistent, or the right thing to build. When I was less experienced I tended to think of the requirements as The Requirements. I didn’t always realize that the “requirements” were often just half-baked ideas that needed a lot of polishing before they were really ready to be implemented.

Even if a particular feature does make sense and could conceivably be built, there’s often an issue of scoping ambiguity. When we say we’re going to build Feature X, where exactly are the contours of Feature X? What’s inside the scope and Feature X and what’s outside it? These questions need to be answered in detail before the feature can be successfully built. Otherwise the development of the feature tends to drag on, getting kicked from sprint to sprint, with both developers and stakeholders getting increasingly frustrated as the feature fails to reach a state of doneness.

How to understand a feature’s requirements the first time

The secret to understanding a feature’s requirements is to do two things:

  1. Perform usability tests to flesh out the feature and verify the feature’s logical consistency and suitability for the task
  2. Make a detailed written/graphical agreement with the stakeholder regarding what the feature is

I’ll explain each of these things individually.

Usability testing

At some point early in my career I got tired of building the wrong thing over and over. So I decided to try to figure out how to stop making that mistake. I started researching “user interface design” and found a book called User Interface Design by Soren Lauesen (which remains one of the best software books I’ve ever read). This was when I first learned about usability testing.

Usability testing is a topic big enough to fill a book (or a lot of books, actually), so I can only scratch the surface here, but here’s the basic idea.

There are two important truths in usability testing. The first is that for all but the simplest features, it’s impossible to come up with a usable design on the first try. (By “usable design”, I mean a design that’s complete enough that the user can carry out their business and that’s sufficiently unconfusing that the user doesn’t throw their hands up in frustration along the way.)

The second truth is that the only way to tell whether a design is sufficiently usable is to test it.

If you buy these two axioms, here’s how to actually perform usability testing, or at least how I do it.

The first step is to talk with the stakeholder to gain an understanding of the feature’s requirements. I don’t have any special methodology for this one, although I’ll give two tips. First, at the risk of stating the obvious, it’s a good idea to write down the things the stakeholder says. Second, it’s a good idea to repeat your understanding of the requirements to the stakeholder and ask if your understanding is consistent with what they’re thinking.

The next step is to translate the requirements into a visual UI design. If I’m working in-person, I usually start really lo-fi and just draw wireframes with pen and paper. If I’m working remotely, I’ll use Balsamiq. The point is that I know my early attempts will probably be off the mark, so I want to use methods that will allow me a quick turnaround time.

Once I have the first draft my design prototypes ready I’ll get together with either an actual user of the system I’m working on or someone who’s representative of a user. Sometimes that person and the stakeholder are the same person and sometimes they’re not. Sometimes no such person is available and I have to get some random person to serve in the role. Sometimes I just have to use the stakeholder in that role, even if it’s not ideal. In any case, it’s best if the test user is someone who’s not involved with the design work. Anyone who’s too familiar with the feature already will fail to stumble over the design’s confusing parts and the usability testing process will be less fruitful.

Once I’ve found my test user I’ll have a usability testing meeting. Ideally, the meeting involves a facilitator, the test user, and a record keeper. I myself typically serve in the facilitator role. Before the meeting, I as facilitator will have come up with a handful of test scenarios for us to go through. For example, a test scenario might say, “Wanda Smith enters the hotel and wants to book a room for two nights. Find the available options in the system and complete the transaction.” For each test scenario, I’ll give the test user the appropriate starting design (which is a piece of paper with a single web page’s wireframes on it) and instruct the test user to “click” on things with their pen and to fill out forms with the pen just like they would on a web page. When the test user “submits” a form, I take away that piece of paper and give them the appropriate subsequent page.

I want to emphasize that never in this process do I ask the test user what they “think” of the design. When asked what they “think” of a design, people will typically just say that it looks good. This is of course not helpful. Our purpose is to put the design through a series of pass/fail tests that will not allow any serious defects to hide.

It’s also important not to give the test user any hints during this process. If the design is sufficiently unclear that the test user gets stuck, we need to let them get stuck. The whole point of the usability testing process is to uncover the weaknesses of the design that let the user get stuck. When this happens, it’s the record keeper’s job to make a note of where exactly the test user got stuck so that the defect can be addressed in the next round of design prototypes. Once the record keeper has made their note of the defect, the test user can be given a hint and allowed to continue, if possible.

When I started doing usability testing, I found to my surprise that my first attempt at the design was almost always unusable. The test user almost always got hopelessly stuck during the first meeting. It would usually take two or three rounds of prototyping before I arrived at a usable design.

Once the usability testing process is complete, the designs created during the usability testing process can be translated into written requirements.

Written requirements

If a stakeholder and a developer don’t have a shared vision of exactly what a feature is going to be when it’s complete, then they’re inviting trouble down the road when the software that gets built doesn’t match what’s expected. And it’s really hard to have a shared vision of exactly what a feature is going to be if there aren’t highly detailed written and/or graphical requirements.

The beauty of usability testing is that it kills ambiguity and logical inconsistency. If the ideas for the feature start off vague, they must be made concrete in order to be translated into design prototypes. If the ideas for the feature start of illogical, the logic failures will surface as defects and not survive the usability testing process.

Because usability testing goes so far toward clarifying requirements, often the designs that result from the usability testing process can just be translated into words and used along with the designs themselves as the feature’s specs. It’s not quite that simple though.

I’ve seen a lot of features fail to get completed within anything close to the time expected, including features I’ve worked on myself. Often the reason is that the feature was too big, or too vague, or both.

When translating designs to specs (“user stories” if you’re using agile development), it’s important not to make the user stories too big. I like to keep all my user stories small enough that they can be started, completed, tested, deployed and verified all in the same day. I’ve sometimes seen the mistake of teams working in one-week sprints where the stories inside the sprint are each expected to take one week. This is bad because it leaves no margin for error. And of course, stories that are expected to take a week often take two weeks or more. So these stories keep getting pushed from sprint to sprint, for several sprints in a row. When each story is only a day’s worth of work, this is not so much of a risk, even if the story takes several times as long as expected.

It’s also important for each user story to be crisply defined. My rule of thumb is that some outside person should be able to read the story’s description and understand exactly what steps they would need to perform in the application in order to verify that the story is done. The question of “How do we know when this feature is done?” should be abundantly clear.

In addition to each story having a clear definition of done, it’s also helpful if the team has a definition of done for stories in general. For example, is it considered done when it’s feature-complete or only when it’s deployed and verified to be working? (I prefer the latter definition myself.) Having an agreement like this helps prevent any ambiguity or disagreement as to whether any particular feature is really actually done.

Takeaways

Building the wrong thing is a common problem, but luckily, it’s a very fixable one. The key is to perform usability testing in order to sharply define the feature and then to document the feature’s requirements clearly and precisely so there’s little room for misunderstanding.

Not only will this way of working eliminate huge amounts of waste and frustration in the development process, it’s also a lot more fun.

The difference between let, let! and instance variables in RSpec

17 Replies

The purpose of let and the differences between let and instance variables

RSpec’s let helper method is a way of defining values that are used in tests. Below is a typical example.

require 'rspec'

RSpec.describe User do
  let(:user) { User.new }

  it 'does not have an id when first instantiated' do
    expect(user.id).to be nil
  end
end

Another common way of setting values is to use instance variables in a before block like in the following example.

require 'rspec'

RSpec.describe User do
  before { @user = User.new }

  it 'does not have an id when first instantiated' do
    expect(@user.id).to be nil
  end
end

There are some differences between the let approach and the instance variable approach, with one in particular that’s quite significant.

Differences between let and instance variables

First, there’s the stylistic difference. The syntax is of course a little different between the two approaches. Instance variables are of course prefixed with @. Some people might prefer one syntax over the other. I personally find the let syntax ever so slightly tidier.

There are also a couple mechanical differences. Because of how instance variables work in Ruby, you can use an undefined instance variable and Ruby won’t complain. This presents a slight danger. You could for example accidentally pass some undefined instance variable to a method, meaning you’d really be passing nil as the argument. This means you might be testing something other than the behavior you meant to test. This danger is admittedly remote though. Nonetheless, the let helper defined not an instance variable but a new method (specifically, a memoized method—we’ll see more on this shortly), meaning that if you typo your method’s name, Ruby will most certainly complain, which is of course good.

The other mechanical difference is that let can create values that get evaluated lazily. I personally find this to be a dangerous and bad idea, which I’ll explain below, but it is a capability that the helper offers.

Perhaps the most important difference between let and instance variables is that instance variables, when set in a before block, can leak from one file to another. If for example an instance variable called @customer is set in “File A”, then “File B” can reference @customer and get the value that was set in File A. Obviously this is bad because we want our tests to be completely deterministic and independent of one another.

How let works and the difference between let and let!

How let works

I used to assume that let simply defines a new variable for me to use. Upon closer inspection, I learned that let is a method that returns a method. More specifically, let returns a memoized method, a method that only gets run once.

Since that’s perhaps kind of mind-bending, let’s take a closer look at what exactly this means.

An example method

Consider this method that 1) prints something and then 2) returns a value.

def my_name
  puts 'thinking about what my name is...'
  'Jason Swett'
end

puts my_name

When we run puts my_name, we see the string that gets printed (puts 'thinking about what my name is...') followed by the value that gets returned by the method (Jason Swett).

$ ruby my_name.rb
thinking about what my name is...
Jason Swett

Now let’s take a look at some let syntax that will create the same method.

require 'rspec'

describe 'my_name' do
  let(:my_name) do
    puts 'thinking about what my name is...'
    'Jason Swett'
  end

  it 'returns my name' do
    puts my_name
  end
end

When we run this test file and invoke the my_name method, the same exact thing happens: the method `puts`es some text and returns my name.

$ rspec my_name_spec.rb
thinking about what my name is...
Jason Swett
.

Finished in 0.00193 seconds (files took 0.08757 seconds to load)
1 example, 0 failures

Just to make it blatantly obvious and to prove that my_name is indeed a method call and not a variable reference, here’s a version of this file with parentheses after the method call.

require 'rspec'

describe 'my_name' do
  let(:my_name) do
    puts 'thinking about what my name is...'
    'Jason Swett'
  end

  it 'returns my name' do
    puts my_name() # this explicitly shows that my_name() is a method call
  end
end

Memoization

Here’s a version of the test that calls my_name twice. Even though the method gets called twice, it only actually gets evaluated once.

require 'rspec'

describe 'my_name' do
  let(:my_name) do
    puts 'thinking about what my name is...'
    'Jason Swett'
  end

  it 'returns my name' do
    puts my_name
    puts my_name
  end
end

If we run this test, we can see that the return value of my_name gets printed twice and the thinking about what my name is... part only gets printed once.

$ rspec my_name_spec.rb
thinking about what my name is...
Jason Swett
Jason Swett
.

Finished in 0.002 seconds (files took 0.08838 seconds to load)
1 example, 0 failures

The lazy evaluation of let vs. the immediate evaluation of let!

When we use let, the code inside our block gets evaluated lazily. In other words, none of the code inside the block gets evaluated until we actually call the method created by our let block.

Take a look at the following example.

require 'rspec'

describe 'let' do
  let(:message) do
    puts 'let block is running'
    'VALUE'
  end

  it 'does stuff' do
    puts 'start of example'
    puts message
    puts 'end of example'
  end
end

When we run this, we’ll see start of example first because the code inside our let block doesn’t get evaluated until we call the message method.

$ rspec let_example_spec.rb
start of example
let block is running
VALUE
end of example
.

Finished in 0.00233 seconds (files took 0.09836 seconds to load)
1 example, 0 failures

The “bang” version of let, let!, evaluates the contents of our block immediately, without waiting for the method to get called.

require 'rspec'

describe 'let!' do
  let!(:message) do
    puts 'let block is running'
    'VALUE'
  end

  it 'does stuff' do
    puts 'start of example'
    puts message
    puts 'end of example'
  end
end

When we run this version, we see let block is running appearing before start of example.

$ rspec let_example_spec.rb 
let block is running
start of example
VALUE
end of example
.

Finished in 0.00224 seconds (files took 0.09131 seconds to load)
1 example, 0 failures

I always use let! instead of let. I’ve never encountered a situation where the lazily-evaluated version would be helpful but I have encountered situations where the lazily-evaluated version would be subtly confusing (e.g. a let block is saving a record to the database but it’s not abundantly clear exactly at what point in the execution sequence the record gets saved). Perhaps there’s some performance benefit to allowing the lazy evaluation but in most cases it’s probably negligible. Confusion is often more expensive than slowness anyway.

Takeaways

  • The biggest advantage to using let over instance variables is that instance variables can leak from test to test, which isn’t true of let.
  • The difference between let and let! is that the former is lazily evaluated while the latter is immediately evaluated.
  • I always use the let! version because I find the execution path to be more easily understandable.

When to refactor

Leave a reply

As a system grows, constant refactoring is needed in order to keep the code understandable.

There are four possible times to perform any piece of refactoring.

  1. During a change
  2. Independently of changes
  3. After a change
  4. Before a change

Some of these times to perform refactorings are good and some are less good. Let’s evaluate each of these times.

Evaluating the various times to refactor

Refactoring during a change

Of the four possible times to refactor, refactoring during a change is the worst time to do it. The reason is because when you look back at the changes you made, you have no easy way to see what’s a behavior change and what’s a refactoring. If you introduce a bug during your work, you won’t know if the bug came from the behavior change or a mistake in your refactoring. It also makes pull requests harder to review.

Mixing refactoring with behavior changes also creates problems around version control. If you discover later that your commit introduced a bug and you need to roll back the commit, you’ll be forced to roll back both the refactoring and the behavior change, even if only one of the two is problematic and the other one is fine. Mixing refactoring with behavior changes also makes it hard to pinpoint bugs using git bisect.

Independently of changes

Performing refactoring independently of changes is an okay time to do it but it’s not my favorite. Bad code only costs something if and when that code needs to be worked with. Having bad code sitting in your codebase is kind of like having a dull saw sitting in your garage. Sharp saws are better than dull saws, but dull saws don’t cause problems until you actually have to use them. Better in my opinion to sharpen a saw either right before or right after you use it, since that way you can be more sure your sharpening is worthwhile because you’re sharpening a saw that you know gets used.

After a change

Refactoring code right after a change is a great time to do it. For anything but the smallest changes, it’s basically impossible to write good code on the first try. Sometimes a change triggers small, local refactorings. Sometimes a change alters the story of the codebase enough that a broad refactoring scattered across the whole application is called for.

One downside to refactoring after a change is that it can be hard to tell exactly what needs refactoring. If you’re refactoring something you wrote five minutes ago, then you’re not a very good simulation of a future maintainer who has to look at your code for the first time and try to understand it. For this reason I don’t refactor my code to an obsessive degree when I’m performing refactorings right after a change because I know that my idea of what’s “best” is likely to be wrong at this point in time. I’ll probably have a better idea of how to clearly express the code after I’ve had some time to forget what it does so that the gaps between what the code does and how well the code explains what it does are more obvious.

Before a change

Refactoring before a change is also a great time to do it. As mentioned above, it’s often better to refactor a piece of code you don’t understand than a piece of code you do understand, because if you don’t understand the code, then you have a stronger need for the code to clearly explain to you what it does. If you understand a piece of code throughly then the code might seem clear enough even when it’s not.

Another reason that refactoring before a change is a good idea is that you know for sure that your refactoring is going to be helpful. In fact, you may well discover that the change you want to make is basically impossible without some refactoring first.

A metaphor to remember this by

The Boy Scout Rule, “leave the campground cleaner than you found it”, is a popular rule among programmers, but I think the Boy Scout Rule too easily misinterpreted. In what way exactly are you supposed to leave the campground cleaner than you found it?

I prefer a kitchen metaphor. I think “clean the kitchen before you make dinner” and “clean the kitchen after you make dinner” make pretty good metaphors for refactoring before and after a change. Obviously it’s more annoying and time-consuming to try to cook dinner in a kitchen full of dirty dishes than a clean one. And if you clean the kitchen after you make dinner, it just makes it that much faster and easier to make dinner next time because there will be less cleaning.

Takeaways

Don’t mix refactoring with behavior changes. The best time to do refactoring is either right before or right after a change.

Understanding Ruby blocks

6 Replies

Blocks are a fundamental concept in Ruby. Many common Ruby methods use blocks. Blocks are also an integral part of many domain-specific languages (DSLs) in libraries like RSpec, Factory Bot, and Rails itself.

In this post we’ll discuss what a block is. Then we’ll take a look at four different native Ruby methods that take blocks (times, each, map and tap) in order to better understand what use cases blocks are good for.

Lastly, we’ll see how to define our own custom method that takes a block.

What a block is

Virtually all languages have a way for functions to take arguments. You pass data into a function and then the function does something with that data.

A block takes that idea to a new level. A block is a way of passing behavior rather than data to a method. The examples that follow will illustrate exactly what is meant by this.

Native Ruby methods that take blocks

Here are four native Ruby methods that take blocks. For each one I’ll give a description of the method, show an example of the method being used, and then show the output that that example would generate.

Remember that blocks are a way to pass behavior rather than data into methods. In each description, I’ll use the phrase “Behavior X” to describe the behavior that might be passed to the method.

Method: times

Description: “However many times I specify, repeat Behavior X.”

Example: three times, print the text “hello”. (Behavior X is printing “hello”.)

3.times do
  puts "hello"
end

Output:

hello
hello
hello

Method: each

“Take this array. For each element in the array, execute Behavior X.”

Example: iterate over an array containing three elements and print each element. (Behavior X is printing the element.)

[1, 2, 3].each do |n|
  puts n
end

Output:

1
2
3

Method: map

“Take this array. For each element in the array, execute Behavior X, append the return value of X to a new array, and then after all the iterations are complete, return the newly-created array.”

Example: iterate over an array and square each element. (Behavior X is squaring the element.)

squares = [1, 2, 3].map do |n|
  n * n
end

puts squares.join(",")

Output:

1,4,9

Method: tap

“See this value? Perform Behavior X and then return that value.”

Example: initialize a file, write some content to it, then return the original file. (Behavior X is writing to the file.)

require "tempfile"

file = Tempfile.new.tap do |f|
  f.write("hello world")
  f.rewind
end

puts file.read

Output:

hello world

Now let’s look at how we can write our own method that can take a block.

Custom methods that take blocks

An HTML generator

Here’s a method which we can give an HTML tag as well as a piece of behavior. The method will execute our behavior. Before and after the behavior will be the opening and closing HTML tags.

inside_tag("p") do
  puts "Hello"
  puts "How are you?"
end

The output of this code looks like this.

<p>
Hello
How are you?
</p>

In this example, the “Behavior X” that we’re passing to our method is printing the text “Hello” and then “How are you?”.

The method definition

Here’s what the definition of such a method might look like.

def inside_tag(tag, &block)
  puts "<#{tag}>"  # output the opening tag
  block.call       # call the block that we were passed
  puts "</#{tag}>" # output the closing tag
end

Adding an argument to the block

Blocks can get more interesting when add arguments.

In the below example, the inside_tag block now passes an instance of Tag back to the block, allowing the behavior in the block to call tag.content rather than just puts. This allows our content to be indented.

class Tag
  def content(value)
    puts "  #{value}"
  end
end

def inside_tag(tag, &block)
  puts "<#{tag}>"
  block.call(Tag.new)
  puts "</#{tag}>"
end

inside_tag("p") do |tag|
  tag.content "Hello"
  tag.content "How are you?"
end

The above code gives the following output.

<p>
  Hello
  How are you?
</p>

Passing an object back to a block is a common DSL technique used in libraries like RSpec, Factory Bot, and Rails itself.

The technical details of blocks

There are a lot of technical details to learn about blocks. There are some interesting questions you could ask about blocks, including the following:

These are all good questions worth knowing the answer to, and you can click the links above to find out. But understanding these details is not necessary in order to understand the high-level gist of blocks.

Takeaway

A block is a way of passing behavior rather than data to a method. Not only do native Ruby methods make liberal use of blocks, but so do many popular Ruby libraries. Custom methods that take blocks can also sometimes be a good way to add expressiveness to your own applications.

How to Dockerize a Sinatra application

10 Replies

Why we’re doing this

Docker is difficult

In my experience, Dockerizing a Rails application for the first time is pretty hard. Actually, doing anything with Docker seems pretty hard. The documentation isn’t that good. Clear examples are hard to find.

Dockerizing Rails is too ambitious as a first goal

Whenever I do anything for the first time, I want to do the simplest, easiest possible version of that thing before I try anything more complicated. I also never want to try to learn more than one thing at once.

If I try to Dockerize a Rails application without any prior Docker experience, then I’m trying to learn the particulars of Dockerizing a Rails application while also learning the general principles of Docker at the same time. This isn’t a great way to go.

Dockerizing a Sinatra application gives us practice

Dockerizing a Sinatra application lets us learn some of the principles of Docker, and lets us get a small Docker win under our belt, without having to confront all the complications of Dockerizing a Rails application. (Sinatra is a very simple Ruby web application framework.)

After we Dockerize our Sinatra application we’ll have a little more confidence and a little more understanding than we did before. This confidence and understanding will be useful when we go to try to Dockerize a Rails application (which will be a future post).

By the way, if you’ve never worked with Sinatra before, don’t worry. No prior Sinatra experience is necessary.

What we’re going to do

Here’s what we’re going to do:

  1. Create a Sinatra application
  2. Run the Sinatra application to make sure it works
  3. Dockerize the Sinatra application
  4. Run the Sinatra application using Docker
  5. Shotgun a beer in celebration (optional)

I’m assuming you’re on a Mac and that you already have Docker installed. If you don’t want to copy/paste everything, I have a repo of all the files here.

(Side note: I must give credit to Marko Anastasov’s Dockerize a Sinatra Microservice post, from which this post draws heavily.)

Let’s get started.

Creating the Sinatra application

Our Sinatra “application” will have just one file. The application will have just one endpoint. Create a file called hello.rb with the following content.

# hello.rb

require 'sinatra'

get '/' do
  'It works!'
end

We’ll also need to create a Gemfile that says Sinatra is a dependency.

# Gemfile

source 'https://rubygems.org'

gem 'sinatra'

Lastly for the Sinatra application, we’ll need to add the rackup file, config.ru.

# config.ru

require './hello'

run Sinatra::Application

After we run bundle install to install the Sinatra gem, we can run the Sinatra application by running ruby hello.rb.

$ bundle install
$ ruby hello.rb

Sinatra apps run on port 4567 by default, so let’s open up http://localhost:4567 in a browser.

$ open http://localhost:4567

If everything works properly, you should see the following.

Dockerizing the Sinatra application

Dockerizing the Sinatra application will involve two steps. First, we’ll create a Dockerfile will tells Docker how to package up the application. Next we’ll use our Dockerfile to build a Docker image of our Sinatra application.

Creating the Dockerfile

Here’s what our Dockerfile looks like. You can put this file right at the root of the project alongside the Sinatra application files.

# Dockerfile

FROM ruby:2.7.4

WORKDIR /code
COPY . /code
RUN bundle install

EXPOSE 4567

CMD ["bundle", "exec", "rackup", "--host", "0.0.0.0", "-p", "4567"]

Since it might not be clear what each part of this file does, here’s an annotated version.

# Dockerfile

# Include the Ruby base image (https://hub.docker.com/_/ruby)
# in the image for this application, version 2.7.4.
FROM ruby:2.7.4

# Put all this application's files in a directory called /code.
# This directory name is arbitrary and could be anything.
WORKDIR /code
COPY . /code

# Run this command. RUN can be used to run anything. In our
# case we're using it to install our dependencies.
RUN bundle install

# Tell Docker to listen on port 4567.
EXPOSE 4567

# Tell Docker that when we run "docker run", we want it to
# run the following command:
# $ bundle exec rackup --host 0.0.0.0 -p 4567.
CMD ["bundle", "exec", "rackup", "--host", "0.0.0.0", "-p", "4567"]

Building the Docker image

All we need to do to build the Docker image is to run the following command.

I’m choosing to tag this image as hello, although that’s an arbitrary choice that doesn’t connect with anything inside our Sinatra application. We could have tagged it with anything.

The . part of the command tells docker build that we’re targeting the current directory. In order to work, this command needs to be run at the project root.

$ docker build --tag hello .

Once the docker build command successfully completes, you should be able to run docker images and see the hello image listed.

Running the Docker image

To run the Docker image, we’ll run docker run. The -p 4567:4567 portion says “take whatever’s on port 4567 on the container and expose it on port 4567 on the host machine”.

$ docker run -p 4567:4567 hello

If we visit http://localhost:4567, we should see the Sinatra application being served.

$ open http://localhost:4567

Conclusion

Congratulations. You now have a Dockerized Ruby application!

With this experience behind you, you’ll be better equipped to Dockerize a Rails application the next time you try to take on that task.

Why I don’t use Shoulda matchers

7 Replies

One of the testing questions I commonly get is about Shoulda matchers. People ask if I use Shoulda matchers and if Shoulda matchers are a good idea.

I’ll share my thoughts on this. First I’ll explain what Shoulda is, then I’ll explain why I don’t use it.

What Shoulda is

If you’re unfamiliar with Shoulda matchers, the premise, from the GitHub description, is: “Shoulda Matchers provides RSpec- and Minitest-compatible one-liners to test common Rails functionality that, if written by hand, would be much longer, more complex, and error-prone.”

A few examples of specific Shoulda matchers are validates_presence_of (expects that a model attribute has a presence validator), have_many (expects that a has_many association exists), and redirect_to (expects that a redirection takes place).

I like the idea of a library that can clean up a lot of my repetitive test code. Unfortunately, Shoulda matchers only apply to the kinds of tests I would never write.

Test behavior, not implementation

To me it doesn’t make much sense to, for example, write a test that only checks for the presence of an Active Record association and doesn’t do anything else.

If I have an association, presumably that association exists in order to enable some piece of behavior, or else it would be pointless for the association to exist. For example, if a User class has_many :posts, then that association only makes sense if there’s some Post-related behavior.

So there are two possibilities in light of testing that the User class has_many :posts. One is that I write a test for both the association itself and the behavior enabled by the association, in which case the test for the association is redundant and adds no value. The other possibility is that I write a test only for the post association, but not for the post behavior, which wouldn’t make much sense because why wouldn’t I write a test for the post behavior?

To me it only makes sense in this example to write tests for the post behavior and write no tests directly for the association. The logic of this decision can be proved by imagining what would happen if the has_many :posts line were removed. Any tests for the post behavior would start failing because the behavior would be broken without the association line present.

Takeaway

Don’t test low-level implementations. It’s pointless. Test behavior instead.

Since Shoulda is only good for testing low-level implementations, I don’t recommend using it.