Most beginner programmers (and even many experienced programmers) take a slow, painful, wasteful approach to programming.
The wasteful way
The way many programmers code is to spend a bunch of time writing code without checking to see that it works, then finally run the program once they’ve accumulated many lines of code. The program inevitably fails.
Next, the programmer sits and puzzles over what might have gone wrong. Since the programmer wrote a lot of code without checking it, there’s a lot of stuff that could possibly be the culprit, and therefore the debugging process is slow and painful.
The debugging process is usually not a systematic one but rather a guessing game. “Maybe it’s this. Nope it’s not that. Maybe it’s this other thing. Nope, it’s not that either. Hmm.” As the clock ticks, frustration mounts, and maybe a little desperation sets in. It’s not fast and it’s not fun.
The smarter way: feedback loops
Instead of working in the slow, painful, wasteful way described above, you can work in feedback loops. As I described in my other post about feedback loops, the feedback loop process goes like this:
Decide what you want to accomplish
Devise a manual test you can perform to see if #1 is done
Perform the test from step 2
Write a line of code
Repeat test from step 2 until it passes
Repeat from step 1 with a new goal
When you use the feedback loop method, it’s hard to run too far astray. If you only write a little bit of code at a time and you keep everything working at all times, then you’re guaranteed to always have a program that’s either fully working or very close to fully working.
Feedback loops and automated testing
Automated testing is just the practice of coding using feedback loops, but with the testing step automated.
Here’s how the feedback loop would go with automated tests involved. The automated test parts are included in bold.
Decide what you want to accomplish
Devise a manual test you can perform to see if #1 is done (write a test)
Perform the test from step 2 (run the test)
Write a line of code
Repeat test from step 2 until it passes (run the test again)
Repeat from step 1 with a new goal
Obviously there’s also a lot of technical knowledge that’s needed in order to write automated tests. For example, there are test frameworks that enable automated testing, there are libraries that help your tests interact with a browser, and there are libraries that help with generating test data. But more important than any particular tool are the principles behind automated testing.
Perhaps the most important idea behind automated testing is the feedback loop. And luckily for you if you’re a beginner, you can learn how to program in feedback loops without having to learn anything to do with automated testing yet. And once you do, writing automated tests will feel much more natural.
One of the most common questions about testing, including what to write tests for, is what NOT to write tests for.
When people ask me what to write tests for, my honest but maybe not very helpful answer is “basically everything”. But I don’t test literally absolutely everything. There are some cases when I choose to skip tests.
My criteria for skipping a test
My habit of writing tests for everything leads me to write tests for my code by default. For most of the code I write, I actually finder it harder to write the code without tests than with tests.
But sometimes I get lazy or my instincts tell me that a test would be a waste of time. When I’m feeling like this, I ask myself three questions:
If this code were to misbehave, would it fail silently?
If this code were to misbehave, how bad would the consequences be?
If this code were to misbehave, how frequently would it fail?
How costly would it be to write a test?
If the code I’m working on would fail in a very obvious way, and the consequences are minor, and the failure would only happen once a year, and the test would be very costly to write, then that’s a case where I would probably skip the test.
On the other hand, if the code would fail silently, OR if the consequences would be bad, OR if the failure would be frequent, OR if it’s super easy to write the test, then I would just write the test.
You could think of these questions as a boolean expression where all the items get OR’d together:
“Should I write a test?” formula (boolean OR)
Might fail silently?
Consequences might be bad?
Might fail frequently?
Test is easy to write?
If any one of the items is true, then the whole boolean expression is true and I go ahead and write the test. Otherwise I skip the test without guilt or worry.
Behavior vs. implementation
There are also entire types of tests I avoid.
I don’t test for things like the presence of associations, the presence of methods, etc. Such tests are pointless. Rather, I test the behavior that these things enable. Testing the behavior is the only way you can really be sure anything works.
Takeaways
I’ll skip a test if and only if the feature won’t fail silently, the consequences won’t be bad, the failure won’t occur frequently, and the test is expensive to write.
I don’t test implementation details, but rather I test the behaviors that the implementations enable.
When you look at a Factory Bot factory definition, the syntax might look somewhat mysterious. Here’s an example of such a factory.
FactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
The goal of this tutorial is to demystify this syntax. The way we’ll do this is to write our own implementation of Factory Bot from scratch. Or, more precisely, we’ll write an implementation of something that behaves indistinguishably from Factory Bot for a few narrow use cases.
Concepts we’ll learn about
Blocks
Factory Bot syntax makes heavy use of blocks. In this post we’ll learn a little bit about how blocks work.
Message sending
We’ll learn the nuanced distinction between calling a method on an object and sending a message to an object.
method_missing
We’ll learn how to use Ruby’s method_missing feature so that we can define methods for objects dynamically.
Our objective
Our goal with this post will be to write some code that makes the factory below actually work. Notice that this factory is indistinguishable from a Factory Bot factory except for the fact that it starts with MyFactoryBot rather than FactoryBot.
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
In addition to this factory code, we’ll also need some code that exercises the factory.
Exercising the factory
Here’s some code that will exercise our factory to make sure it actually works. Just like how our factory definition mirrors an actual Factory Bot factory definition, the below code mirrors how we would use a Factory Bot factory.
After we get our factory working properly, the above code should produce the following output.
First name: John
Last name: Smith
Let’s get started.
How to follow along
If you’d like to code along with this tutorial, I have a sample project that you can use.
The code in this tutorial depends on a certain Rails project, so I’ve created a GitHub repo at https://github.com/jasonswett/my_factory_bot where there’s a Dockerized Rails app, the same exact Rails app that I used to write this tutorial. You’ll find instructions to set up the project on the GitHub page.
The approach
We’ll be writing our “Factory Bot clone” code using a (silly) development methodology that I call “error-driven development”. Error-driven development works like this: you write a piece of code and try to run it. If you get an error when you run the code, you write just enough code to fix that particular error, and nothing more. You repeat this process until you have the result you want.
The reason I sometimes like to code this way is that it prevents me from writing any code that hasn’t been (manually) tested. Surprisingly enough, this “methodology” actually works pretty well.
Building the factory
The first thing to do is to create a file called my_factory_bot.rb and put it at the Rails project root.
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
Then we’ll run the code like this:
$ rails run my_factory_bot.rb
The first thing we’ll see is an error saying that MyFactoryBot is not defined.
uninitialized constant MyFactoryBot (NameError)
This is of course true. We haven’t yet defined something called MyFactoryBot. So, in the spirit of practicing “error-driven development”, let’s write enough code to make this particular error go away and nothing more.
class MyFactoryBot
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
Now, if we run the code again (using the same rails run command from above), we get a different error.
undefined method `define' for MyFactoryBot:Class (NoMethodError)
This is also true. The MyFactoryBot class doesn’t have a method called define. So let’s define it.
class MyFactoryBot
def self.define
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
Now we get a new error.
undefined method `create' for MyFactoryBot:Class (NoMethodError)
This of course comes from the user = MyFactoryBot.create(:user) line. Let’s define a create method in order to make this error go away. Since we’re passing in an argument, :user, when we call create, we’ll need to specify a parameter for the create method. I’m calling the parameter model_sym since it’s a symbol that corresponds to the model that the factory is targeting.
class MyFactoryBot
def self.define
end
def self.create(model_sym)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
Now we get an error for the next line.
undefined method `first_name' for nil:NilClass (NoMethodError)
We’ll deal with this error, but not just yet, because this one will be a little bit tricky, and there are some certain other things that will make sense to do first. Let’s temporarily comment out the lines that call first_name and last_name.
class MyFactoryBot
def self.define
end
def self.create(model_sym)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
#puts "First name: #{user.first_name}"
#puts "Last name: #{user.last_name}"
Now, if we run the file again, we get no errors.
Making it so the factory block gets called
Right now, the block inside of MyFactoryBot.define isn’t getting used at all. Let’s add block.call to the defined method so that the block gets called.
class MyFactoryBot
def self.define(&block)
block.call
end
def self.create(model_sym)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
#puts "First name: #{user.first_name}"
#puts "Last name: #{user.last_name}"
Now when we run the file we get the following error.
undefined method `factory' for main:Object (NoMethodError)
It makes sense that we would get an error that says “undefined method factory“. We of course haven’t defined any method called factory
The receiver for the “factory” method
Notice how the error message says undefined method `factory' for main:Object. What’s the main:Object part all about?
Sending a message vs. calling a method
In Ruby you’ll often hear people talk about “sending a message to an object” rather than “calling a method on an object”. The distinction between these things is subtle but significant.
The a variable is an instance of the Array class. When we do a.to_s, we’re sending the to_smessage to the a object. The a object will happily respond to the to_s message and return a stringified version of the array: "[nil, nil, nil, nil, nil]"
The a object does not respond to (for example) the to_i method. If we send to_i to a, we get an error:
undefined method `to_i' for [nil, nil, nil, nil, nil]:Array
Notice the format of the last part of the error message. It’s value of receiving object:class of receiving object.
Understanding main:Object
In Ruby, every message that gets passed has a receiver, even when it doesn’t seem like there would be. When we do e.g. a.to_s, the receiver is obvious: it’s a. What about when we just call e.g. puts?
When we send a message that doesn’t explicitly have a receiver object, the receiver is a special object called main. That’s why when we call factory we get an error message that says undefined local variable or method `factory' for main:Object. The interpreter sends the factory message to main because we’re not explicitly specifying any other receiver object.
Changing the receiver of the factory message
If we want our program to work, we’re going to have to change the receiver of factory from main to something else. If the receiver were just main, then our factory method would have to just be defined out in the open, not as part of any object. If the factory method is not defined as part of any object, then it can’t easily share any data with any object, and we’ll have a pretty tough time.
We can change the receiver of the factory message by using a Ruby method called instance_exec.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.create(model_sym)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
#puts "First name: #{user.first_name}"
#puts "Last name: #{user.last_name}"
The instance_exec method will execute our block in the context of self. We can see that now, when we run our file, our error message has changed. The receiver object is no longer main. It’s now MyFactoryBot.
undefined method `factory' for MyFactoryBot:Class (NoMethodError)
Let’s add the factory method to MyFactoryBot.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym)
end
def self.create(model_sym)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
#puts "First name: #{user.first_name}"
#puts "Last name: #{user.last_name}"
Now we don’t get any errors.
We know we’ll also need to invoke the block inside of the factory call, so let’s use another instance_exec inside of the factory method to do that.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
instance_exec(&block)
end
def self.create(model_sym)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
#puts "First name: #{user.first_name}"
#puts "Last name: #{user.last_name}"
Now it’s complaining, unsurprisingly, that there’s no method called first_name.
undefined method `first_name' for MyFactoryBot:Class (NoMethodError)
Adding the first_name and last_name methods
We of course need to define methods called first_name and last_name somewhere. We could conceivably add them on the MyFactoryBot class, but it would probably work out better to have a separate instance for each factory that we define, since of course a real application will have way more than just one factory.
Let’s make it so that the factory method creates an instance of a new class called MyFactory and then invokes the block on MyFactory.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
factory = MyFactory.new
factory.instance_exec(&block)
end
def self.create(model_sym)
end
end
class MyFactory
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
#puts "First name: #{user.first_name}"
#puts "Last name: #{user.last_name}"
We of course still haven’t actually defined a method called first_name, so we still get an error about that, although the receiver is not MyFactory rather than MyFactoryBot.
undefined method `first_name' for #<MyFactory:0x0000ffff9a955ae8> (NoMethodError)
Let’s define a first_name and last_name method on MyFactory in order to make this error message go away.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
factory = MyFactory.new
factory.instance_exec(&block)
end
def self.create(model_sym)
end
end
class MyFactory
def first_name
end
def last_name
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
#puts "First name: #{user.first_name}"
#puts "Last name: #{user.last_name}"
Now we get no errors.
Providing values for first_name and last_name
Let’s now come back and uncomment the last two lines of the file, the lines that output values for first_name and last_name.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
factory = MyFactory.new
factory.instance_exec(&block)
end
def self.create(model_sym)
end
end
class MyFactory
def first_name
end
def last_name
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
We get an error.
undefined method `first_name' for nil:NilClass (NoMethodError)
Remember that every message in Ruby has a receiver. Apparently in this case the receiver for first_name when we do user.first_name is nil. In other words, user is nil. That’s obviously not going to work out. It does make sense, though, because MyFactoryBot.create(:user) has no return value.
Let’s try making it so that MyFactoryBot#create returns our instance of MyFactory.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
@factory = MyFactory.new
@factory.instance_exec(&block)
end
def self.create(model_sym)
@factory
end
end
class MyFactory
def first_name
end
def last_name
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
Now there are no errors but, but there are also no values present in the output we see.
First name:
Last name:
Let’s have a closer look and see exactly what user is.
This was maybe a good intermediate step but we of course want user to be an instance of our Active Record User class, just like how the real Factory Bot does. Let’s change MyFactoryBot#create so that it returns an instance of User.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
@factory = MyFactory.new
@factory.instance_exec(&block)
end
def self.create(model_sym)
@factory.user
end
end
class MyFactory
attr_reader :user
def initialize
@user = User.new
end
def first_name
end
def last_name
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts user.class.name
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
This gives no errors but there are still no values present.
In order for the first_name and last_name methods to return values, let’s make it so each one calls the block that it’s given.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
@factory = MyFactory.new
@factory.instance_exec(&block)
end
def self.create(model_sym)
@factory.user
end
end
class MyFactory
attr_reader :user
def initialize
@user = User.new
end
def first_name(&block)
@user.first_name = block.call
end
def last_name(&block)
@user.last_name = block.call
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts user.class.name
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
Now we see “John” and “Smith” as our first and last name values.
User
First name: John
Last name: Smith
Generalizing the factory
What if we wanted to add this? It wouldn’t work.
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
email { "john.smith@example.com" }
end
end
Obviously we can’t just have hard-coded first_name and last_name methods in our factory. We need to make it so our factory can respond to any messages that are sent to it (provided of course that those messages correspond to actual attributes on our Active Record models).
Let’s take the first step toward generalizing our methods. Instead of @user.first_name = block.call, we’ll do @user.send("first_name=", block.call), which is equivalent.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
@factory = MyFactory.new
@factory.instance_exec(&block)
end
def self.create(model_sym)
@factory.user
end
end
class MyFactory
attr_reader :user
def initialize
@user = User.new
end
def first_name(&block)
@user.send("first_name=", block.call)
end
def last_name(&block)
@user.send("last_name=", block.call)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
user = MyFactoryBot.create(:user)
puts user.class.name
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
We can go even further than this. Rather than having methods called first_name and last_name, we can use Ruby’s method_missing to dynamically respond to any message that gets sent.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
@factory = MyFactory.new
@factory.instance_exec(&block)
end
def self.create(model_sym)
@factory.user
end
end
class MyFactory
attr_reader :user
def initialize
@user = User.new
end
def method_missing(attr, *args, &block)
# If the message that's sent is e.g. first_name, then
# the value of attr will be :first_name, and the value
# of "#{attr}=" will be "first_name=".
@user.send("#{attr}=", block.call)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
email { "john.smith@example.com" }
end
end
user = MyFactoryBot.create(:user)
puts user.class.name
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
puts "Email: #{user.email}"
If we run our file again, we can see that our method_missing code handles not only first_name and last_name but email as well.
First name: John
Last name: Smith
Email: john.smith@example.com
More generalization
What if we want to have more factories besides just one for User? I happen to have a model in my Rails app called Website. What if I wanted to have a factory for that?
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
MyFactoryBot.define do
factory :website do
name { "Google" }
url { "www.google.com" }
end
end
Right now, it wouldn’t work because I have the User class hard-coded in my factory.
undefined method `name=' for #<User:0x0000ffff8f950d28> (NoMethodError)
Let’s make it so that rather than hard-coding User, we set the class dynamically. Let’s also make it so that the argument you pass to the factory method (e.g. :user or :website) retries the appropriate factory. We can accomplish this by putting our factories into a hash.
class MyFactoryBot
def self.define(&block)
instance_exec(&block)
end
def self.factory(model_sym, &block)
@factories ||= {}
@factories[model_sym] = MyFactory.new(model_sym)
@factories[model_sym].instance_exec(&block)
end
def self.create(model_sym)
@factories[model_sym].record
end
end
class MyFactory
attr_reader :record
def initialize(model_sym)
@record = model_sym.to_s.classify.constantize.new
end
def method_missing(attr, *args, &block)
@record.send("#{attr}=", block.call)
end
end
MyFactoryBot.define do
factory :user do
first_name { "John" }
last_name { "Smith" }
end
end
MyFactoryBot.define do
factory :website do
name { "Google" }
url { "www.google.com" }
end
end
user = MyFactoryBot.create(:user)
puts user.class.name
puts "First name: #{user.first_name}"
puts "Last name: #{user.last_name}"
puts
website = MyFactoryBot.create(:website)
puts website.class.name
puts "Name: #{website.name}"
puts "URL: #{website.url}"
Now, when we run our file, both factories work.
First name: John
Last name: Smith
Email: john.smith@example.com
Name: Google
URL: www.google.com
Takeaways
Factory Bot syntax (and other DSLs) aren’t magic. They’re just Ruby.
Blocks can be a powerful way to make your code more expressive and understandable.
In Ruby, it’s often more useful to talk about “sending messages to objects” rather than “calling methods on objects”.
Every message sent in Ruby has a receiver. When a receiver is not explicitly specified, the receiver is a special object called main.
The method_missing method can allow your objects to respond to messages dynamically.
One of the biggest mistakes I see in Rails testing (especially model testing) is testing implementation instead of behavior.
Let’s say I run a factory that makes cars. I want to make sure that each car I make actually works. How should I go about making sure each car works?
Here are two possible methods.
Testing implementation
One way I could try to make sure each car works is to check to see if it has all the right stuff. I could check for the presence of an engine, an ignition, a brake system, wheels, and everything else that’s needed in order to get from point A to point B. If the car has all this stuff, then it’s all set.
Testing behavior
Another way I could try to make sure the car works is to start it up and try to drive it somewhere.
The first way is bad and the second way is good. Here’s why.
Why testing behavior is better
If I “verify” that my car works by checking for the presence of various parts, then I haven’t really actually verified anything. I haven’t demonstrated that the system under test (the car) actually meets spec (can drive).
If I test the car by actually driving it, then the questions of whether the car has various components become moot. If for example the car can travel down the road, we don’t need to ask if the car has wheels. If it didn’t have wheels it wouldn’t be moving.
All of our “implementation” questions can be translated into more meaningful “behavior” questions.
Does it have an ignition? -> Can it start up?
Does it have an engine and wheels? -> Can it drive?
Does it have brakes? -> Can it stop?
Lastly, behavior tests are better than implementation tests because behavior tests are more loosely coupled to the implementation. I ask “Can it start up?” instead of “Does it have an engine?” then I’m free to, for example, change my car factory from a gasoline-powered car factory to an electric car factory without having to change the set of tests that I perform. In other words, behavior tests enable refactoring.
Different granularities of behavior testing
I want to be sure that my argument isn’t misconstrued as a discussion between unit tests and integration tests. The implementation tests being performed weren’t actually verifying the proper behavior of the various components (engine, ignition, etc.) in isolation. The implementation tests were merely verifying the presence of components.
In fact, performing some “unit” tests and some “integration” tests in my car factory would probably be a good idea. It would be smart to make sure each part works individually before I combine them all. I’d just want to make sure my unit and integration tests were testing behavior, not implementation.
Verifying that my car can drive is a test at the highest granularity. At a lower level of granularity, I might want to verify that the engine works, even before the engine is connected to an actual car. To test my engine, I wouldn’t want to check to see whether the engine has six cylinders, each with a piston inside. Instead I would want to give my engine some fuel and actually see if it runs.
So please don’t confuse the implementation vs. behavior question with the unit test vs. integration test question. We sometimes want to perform unit tests and we sometimes want to perform integration tests, but we pretty much always want to test behavior instead of implementation.
How this relates to Rails
I sometimes come across examples of Rails model tests that I consider pointless. These tests verify the presence of validations, associations, callbacks and other implementation details. These tests are pointless because they don’t actually test anything. Having these tests is like checking for the presence of your engine and brakes rather than actually just driving your car.
It’s also pointless to write system specs that e.g. check for the presence of various form fields and buttons, for the same exact reason.
When I wrote about nested factories in Factory Bot, I got a couple of questions to the effect of “why wouldn’t you use traits for that?”
In responses to these questions, I’ll lay out my method for deciding when to use traits versus nested factories.
“Is” versus “has”
My method is pretty simple: If the factory I’m considering has something, I use a trait. If the factory is something, I use a nested factory. Let’s look at a concrete example.
“Has” example (trait)
In the following example I want to create a special kind of user record, a user that has a phone number. The user is still conceptually a regular old user. The only difference is that this user happens to have a value for its phone_number attribute.
FactoryBot.define do
factory :user do
username { Faker::Internet.username }
password { Faker::Internet.password }
trait :with_phone_number do
phone_number { Faker::PhoneNumber.phone_number }
end
end
end
When I want to create a user that has a phone number, I do so by doing FactoryBot.create(:user, :with_phone_number).
“Is” example (nested factory)
In the following example I want to create a special kind of user record, a user that is a physician. Conceptually, a physician user is not just a regular old user. This type of user is different in kind. A physician user has different capabilities from a regular user and is used in different ways from a regular user.
FactoryBot.define do
factory :user do
username { Faker::Internet.username }
password { Faker::Internet.password }
factory :physician_user do
role { 'physician' }
end
end
end
When I want to create a physician user, I do so by doing FactoryBot.create(:physician_user). Note the contrast between this and FactoryBot.create(:user, :with_phone_number).
Takeaway
When deciding whether to use a trait or a nested factory, consider whether the record has something or if it is something. If it has something, use a trait. If it is something, use a nested factory.
Sometimes you want to be able to create records that are 95% the same as the “default” but have one or two small differences. Nested factories can be useful for this purpose.
Physician user example
Let’s use the following as an example. Let’s say that in the majority of your application’s tests that involve a User, you want just a regular old user. Maybe you have 30 such tests.
But for a handful of tests you want to use a special kind of user, a user that indicates that the person using the application is a physician.
In this scenario, the technical difference between a “physician user” and a regular user is that a physician user has a role attribute that’s set to 'physician'. Let’s say there are 6 tests that use a physician user, quite a small proportion of the total.
Options for addressing this challenge
You have the following options for addressing this challenge.
First, you could set the role to “physician” individually on each of your 6 physician user tests. Unfortunately this would be bad for all the reasons that duplication is bad. If the nature of your physician user were ever to change, you’d have to know that you’d have to make the change in all 6 of these places. This solution isn’t great.
Alternatively, you could alter the user factory to always have role set to “physician”. Under this scenario, every user would always be a physician user, even when it’s not necessary. Even if this would function properly from a technical perspective, it would be misleading to any maintainers of the test suite. For any test, it’s a good idea to create the minimum amount of setup data and no more. An outside observer has no way of knowing which parts of a test’s setup data are necessary and which aren’t, and so must assume it’s all potentially necessary. We wouldn’t want to send our future maintainers an untruthful message.
A third option is to create an alternate version of the user factory that inherits from the original, and use that version in our physician tests. In this scenario we’re not duplicating our setup steps, and we’re also not modifying our “default” factory. The name for this in Factory Bot is nested factories.
How nested factories work
Consider the following User factory.
FactoryBot.define do
factory :user do
username { Faker::Internet.username }
password { Faker::Internet.password }
end
end
In addition to the above, we can define a nested factory called :physician_user which will have all the same attributes as our regular users, but with the extra attribute of role with the value of 'physician'.
FactoryBot.define do
factory :user do
username { Faker::Internet.username }
password { Faker::Internet.password }
factory :physician_user do
role { 'physician' }
end
end
end
To create a physician user we would simply invoke the following:
FactoryBot.create(:physician_user)
Unlike root-level factory names which map to Active Record models, nested factory names are fully arbitrary.
Takeaways
If you have setup data needs that are common across tests, it’s usually not a good idea to duplicate the setup. Duplication in test code usually has the same negative consequences as duplication in application code.
Each test should have as much setup data as is necessary for that test and no more. Otherwise a maintainer will have a hard time telling what’s necessary and what’s not.
Factory Bot’s nested factories can help us keep our test code DRY while continuing to adhere to certain good testing principles.
If you’ve used Factory Bot at all, you’ve seen syntax like this:
FactoryBot.define do
factory :user do
first_name { 'John' }
last_name { 'Smith' }
email { 'john.smith@example.com' }
end
end
When I was first getting started with Rails and I wasn’t very familiar with Ruby, I would look at these files and understand what’s going on conceptually but I would have no idea what’s going on syntactically.
Why I didn’t understand this code
There were three barriers to my understanding at that early stage of my Ruby experience:
Ruby’s optional parentheses feature, although it can help code be very expressive, can also make it hard for beginners to tell what’s what.
I didn’t understand Ruby blocks yet.
I didn’t know about dynamically-defined methods.
Let’s look at a modified example that will make some of the syntax clearer.
A modified factory definition example
Below is a version of the above factory definition that’s functionally equivalent but with two syntactical changes.
The first change is that I’ve included parentheses for all method calls. The second is that I’ve changed all block usages to do syntax instead of the shorthand {} syntax.
FactoryBot.define() do # define is a method
factory(:user) do
first_name do
'John'
end
last_name do
'Smith'
end
email do
'john.smith@example.com'
end
end
end
You can see here that define and factory are each just methods. Each of the two methods takes a block. (If you’re not very comfortable with blocks yet, check out my post on understanding Ruby blocks.)
first_name, last_name and email are also methods that take blocks, speaking loosely. Before we can talk about those we need to talk about methods versus messages.
Methods and messages
When you call a method on an object, it can be said that you’re sending a message to that object. For example, when you call "5".to_i, you’re sending the message to_i to the object "5", which is of course a String.
In the above case, the message to_i also happens to be a method that’s defined on String. This doesn’t need to be the case though. We could send any message at all to String, String just might not necessarily respond to that particular message. That’s why we have the respond_to? method, to see what messages an object responds to.
The messages an object will respond to need not be limited to the methods that are defined on that object. An object author can use the method_missing method to allow an object to respond to any message that’s sent to it, and respond in any way that the object author chooses.
Factory definitions and messages
What’s likely happening with first_name, last_name and email is that Factory Bot is using method_missing to allow arbitrary messages to be sent, and if the message (e.g. the message of first_name) matches an attribute on the model that the factory is for, then Factory Bot uses the block passed with the message to set the value of that attribute.
Takeaways
Factory Bot’s factory definitions are made out of methods and blocks.
Adding parentheses to any piece of DSL code can often make the code clearer.
Ruby objects can be passed arbitrary messages, and objects can be designed to respond to those messages.
When I see questions from beginners regarding learning testing, sometimes they seem to conflate testing with test-driven development (TDD). People will say “I have such-and-such question about TDD” but really it’s just a question about testing, nothing to do with TDD specifically.
Other people sometimes ask questions about whether TDD is “better” than writing tests after.
In this post I’ll try to clarify what’s TDD and what’s not. I’ll also explain whether I think it makes sense for testing beginners to try to practice TDD.
Testing != TDD
First of all, at the risk of stating the obvious, testing and TDD aren’t the same thing. TDD is a specific kind of testing practice where you write the tests before you write the code that makes the test pass. (If you want to go deeper into TDD, I highly recommend Kent Beck’s Test Driven Development: By Example.)
Learning vs. incorporating
Another mistake beginners sometimes make is to conflate learning testing with incorporating testing as a habitual part of their development workflow. They feel like they need to start adopting testing practices into their workflow from day one, and if they fail to do that, then they’ve failed at learning testing.
I think it’s more productive to separate the jobs of learning testing and applying testing. It’s not like skiing, where you learn it and do it at the same time. It’s more like basketball, where you practice free throws in your driveway and build some skills that way before you try to play a real game in front of an audience. You’ll get farther in the beginning if you separate the practice from the application of what you’ve learned to production tests. When you get comfortable enough, you can take off the training wheels and get all your practice from writing production tests.
TDD is beneficial but optional
TDD is super helpful in certain scenarios but it’s not something you absolutely need to learn when you’re first learning testing. I think it’s completely appropriate to first learn the fundamentals of testing in general, and then start to learn TDD once you’ve developed a decent level of comfort with testing.
I don’t always practice TDD
I’m not an advocate of practicing TDD 100% of the time in Rails, even for experienced testers. The reason is that when I’m building a new feature, I often have little idea what shape that feature will take, and the most realistic way for me to hammer it into shape is to just start building it. Once I’ve built some of the feature, then I’ll start adding tests. So, a portion of the time, I write my tests after writing my application code.
The place where I find TDD most useful is for model code. I practice TDD in my models a high percentage of the time. Once I’ve put the broad strokes of a feature in place, I’ll usually use TDD to work out the fine-grained aspects of it.
Takeaways
Testing and test-driven development aren’t the same thing.
When you’re first learning testing, it can be helpful to separate learning testing from applying testing.
You don’t need to learn TDD when you’re starting out.
I don’t always practice TDD or even advocate practicing TDD 100% of the time. I myself practice TDD maybe 60% of the time.
One of the most common questions asked by developers new to Rails testing is “How do I add tests to an existing Rails project?”
The answer largely depends on your experience level with testing. Here are my answers based on whether you have little testing experience or if you’re already decently comfortable with testing.
If you have little testing experience
If you have little testing experience, I would suggest getting some practice on a fresh Rails app before trying to introduce testing to the existing Rails project you want to add tests to.
Adding tests to an existing project is a distinct skill from writing tests for new projects. Adding tests to an existing project can be difficult even for very experienced testers, for reasons described below.
At the same time, you probably don’t want to wait a year to learn testing before you start enjoying the benefits of testing on your existing Rails app. What I would suggest is to first start a fresh throwaway Rails app for the purpose of learning testing. Then, once you’ve gotten a little experience there, see if you can apply something to your existing Rails app. Then, if things get too hard in the existing app, switch back to the throwaway app so you can strengthen your skills more. Continue switching back and forth until you don’t need to anymore.
If you’re already comfortable with testing
Here’s how I suggest adding tests to an existing Rails project: 1) develop a shared vision with your team, 2) start with what’s easiest, then 3) expand your test coverage.
Develop a shared vision
Going from no tests to decent test coverage is unfortunately not as simple as just deciding one day that from now on we’re going to write tests.
The team maintaining the codebase needs to decide certain things, like what testing tools they’re going to use and what testing approach they’re going to use.
In other words, if the team wants to go from point A to point B, they have to decide exactly where point B is and how they intend to try to get there.
Start with what’s easiest
When adding tests to a codebase that has few or no tests, it might seem logical to start by adding tests where tests would be most valuable. Or it might seem logical to require all new changes to have tests. Unfortunately, both these ideas have problems.
The features in an application that are most valuable are also likely to be among the most non-trivial. This means that tests for these features will probably be relatively hard to write due to the large amount of setup data needed. Code written without testability in mind can also be difficult to test due to entangled dependencies.
Requiring all new changes to have tests also has problems. New changes aren’t usually independent of existing code. They’re usually quite tangled up. This brings us back to the same problem we’d have adding tests to our most important features: the setup and dependencies make adding tests difficult, sometimes prohibitively so.
What I would do instead is start with what’s easiest. I would look for the simplest CRUD interfaces in the app and add some tests there, even if those particular tests didn’t seem to add much value. The idea isn’t to add valuable tests right from the start but to establish a beachhead that can be expanded upon.
Expand
Once you have a handful of tests for trivial features, you can add tests for increasingly complicated features. This will give you a much better shot at ending up with good test coverage than trying to start with the most valuable features or trying to add tests for all new changes.
The mechanical details
If your existing Rails application doesn’t have any testing infrastructure, I would suggest taking a look at my how I set up a Rails application post. (Remember that it’s possible to apply an application template to an existing project.)
As you add tests to your project starting with the most trivial features, I would suggest starting with system specs as opposed to model specs or any other type of specs. The reason is that system specs are often more straightforward to conceive of and understand. If you’d like a formula you can apply to add system specs to almost any CRUD feature, you can find that here.
One of the most common questions for Rails developers new to testing is “What are all the Rails testing tools and how do I use them?”
I’ll explain what the major tools are but I want to preface it by saying that the most important thing to learn to be a successful tester is testing principles, not testing tools. If you think of testing like a taco, the tools are the tortilla and the principles are the stuff inside the taco. The tortilla is essential but it’s really only a vehicle.
The following are the tools I use for my testing.
RSpec
RSpec is a test framework. A test framework is what gives us a structure for writing our tests as well as the ability to run our tests.
There are other test frameworks but RSpec is the most popular one for commercial Rails projects. The second most popular test framework is Minitest.
Test frameworks differ syntactically but the testing principles and practices are going to be pretty much the same no matter what framework you’re using. (If you’re not sure whether you should learn RSpec or Minitest, I write about that here.)
Factory Bot
One of the challenges of Rails testing is generating test data. For example, if you’re writing a test that logs a user in and then takes some action, you’re going to have to create a user in the database at the beginning of the test. Many tests require much more complicated test data setup.
There are two common ways of generating test data in Rails tests: fixtures and factories.
Fixtures
Fixtures typically take the form of one or more YAML files with some hard-coded data. The data is translated into database records one time, before any of the tests are run, and then deleted afterward. (This happens in a separate test database instance of course.)
Factories
With factories, database data is generated specifically for each test. Instead of loading all the data once at the beginning and deleting it at the end, data is inserted before each test case and then deleted before the next test case starts. (More precisely, the data isn’t deleted, but rather the test is run inside a database transaction and the data is never committed in the first place, but that’s a mechanical detail that’s not important right now.)
Relative merits of fixtures and factories
I tend to prefer factories because I like having my data generation right inside my test, close to where the test is happening. With fixtures the data setup is too distant from where the test happens.
In my experience, for whatever reason, most people who use RSpec use factories and most people who use Minitest use fixtures. If you’d like to learn more about factories and fixtures, I write more about it here.
Capybara
Some Rails tests only exercise Ruby code. Other tests actually open up a browser and simulate user clicks and keystrokes.
Simulating user input this way requires us to use some sort of tool to manipulate the browser. Capybara is a library that uses Ruby to wrap a driver (usually the Selenium driver), letting us simulate clicks and keystrokes using convenient Ruby methods.
For more examples of how to use Capybara, go here.
VCR and WebMock
One principle of testing is that tests should be deterministic, meaning they run the same way every time no matter what.
When an application’s behavior depends on external services (e.g. a third-party API like Stripe) it makes it harder to have deterministic tests. The tests can be made to fail by an internet connection failure or a temporary outage of the external service.
Tools like VCR and WebMock can help smooth out these challenges. VCR can let us run our tests against the real external service, but capture all the service’s responses in local files so that subsequent test runs don’t talk to the external service but rather just go off of the saved responses. That way, even if the internet connection fails or the service goes down, the tests still work.
WebMock is a tool that serves a similar purpose, although I usually use it in a more limited way. I don’t consider my test suite to be deterministic unless it doesn’t talk to the network at all, so I use WebMock to enforce that my test suite isn’t making any network requests.
Tools I don’t use
Cucumber is a somewhat popular tool when it comes to acceptance testing. It’s my view that Cucumber adds an extra layer of complexity and indirection without adding any value. Here are some details on why I don’t recommend Cucumber.
I also don’t use Shoulda matchers. Shoulda matchers make it easy and convenient to write certain kinds of tests, but the kinds of tests Shoulda helps you write are not a good kind of test to write in the first place. Shoulda helps you write tests that test your code’s implementation rather than its behavior. Here are more details on why I don’t recommend Shoulda.
Takeaways
Rails testing tools take some time to learn, but the important part (and perhaps more difficult part) is learning testing principles.