Testing implementation vs. behavior in Rails

by Jason Swett,

One of the biggest mistakes I see in Rails testing (especially model testing) is testing implementation instead of behavior.

Let’s say I run a factory that makes cars. I want to make sure that each car I make actually works. How should I go about making sure each car works?

Here are two possible methods.

Testing implementation

One way I could try to make sure each car works is to check to see if it has all the right stuff. I could check for the presence of an engine, an ignition, a brake system, wheels, and everything else that’s needed in order to get from point A to point B. If the car has all this stuff, then it’s all set.

Testing behavior

Another way I could try to make sure the car works is to start it up and try to drive it somewhere.

The first way is bad and the second way is good. Here’s why.

Why testing behavior is better

If I “verify” that my car works by checking for the presence of various parts, then I haven’t really actually verified anything. I haven’t demonstrated that the system under test (the car) actually meets spec (can drive).

If I test the car by actually driving it, then the questions of whether the car has various components become moot. If for example the car can travel down the road, we don’t need to ask if the car has wheels. If it didn’t have wheels it wouldn’t be moving.

All of our “implementation” questions can be translated into more meaningful “behavior” questions.

  • Does it have an ignition? -> Can it start up?
  • Does it have an engine and wheels? -> Can it drive?
  • Does it have brakes? -> Can it stop?

Lastly, behavior tests are better than implementation tests because behavior tests are more loosely coupled to the implementation. I ask “Can it start up?” instead of “Does it have an engine?” then I’m free to, for example, change my car factory from a gasoline-powered car factory to an electric car factory without having to change the set of tests that I perform. In other words, behavior tests enable refactoring.

Different granularities of behavior testing

I want to be sure that my argument isn’t misconstrued as a discussion between unit tests and integration tests. The implementation tests being performed weren’t actually verifying the proper behavior of the various components (engine, ignition, etc.) in isolation. The implementation tests were merely verifying the presence of components.

In fact, performing some “unit” tests and some “integration” tests in my car factory would probably be a good idea. It would be smart to make sure each part works individually before I combine them all. I’d just want to make sure my unit and integration tests were testing behavior, not implementation.

Verifying that my car can drive is a test at the highest granularity. At a lower level of granularity, I might want to verify that the engine works, even before the engine is connected to an actual car. To test my engine, I wouldn’t want to check to see whether the engine has six cylinders, each with a piston inside. Instead I would want to give my engine some fuel and actually see if it runs.

So please don’t confuse the implementation vs. behavior question with the unit test vs. integration test question. We sometimes want to perform unit tests and we sometimes want to perform integration tests, but we pretty much always want to test behavior instead of implementation.

How this relates to Rails

I sometimes come across examples of Rails model tests that I consider pointless. These tests verify the presence of validations, associations, callbacks and other implementation details. These tests are pointless because they don’t actually test anything. Having these tests is like checking for the presence of your engine and brakes rather than actually just driving your car.

It’s also pointless to write system specs that e.g. check for the presence of various form fields and buttons, for the same exact reason.

Takeaway

Test behavior, not implementation.

3 thoughts on “Testing implementation vs. behavior in Rails

  1. Pingback: Testing implementation vs. behavior in Rails - Sebastian Buza's Blog

  2. Greg Sabia Tucker

    I know that these comments are moderated, so this one is unlikely to ever be published. My point is that your analogy for the automotive industry is strange, because the automotive industry categorically DOES NOT rely on testing behavior, even with a small amount of unit testing. The automotive industry almost wholly relies on unit testing!

    There is a very good reason why they use unit testing which you have completely overlooked in your method for testing behavior: What happens when something doesn’t work? The equivalent using your method would be: “The car broke down, why? The engine doesn’t run.” You have no tests as to where or why. In real life, a codebase, or an automotive industry, needs a somewhat robust foundation of unit tests to stand on. Testing behavior is in the “Nice to have” category, and I don’t mean to suggest it’s pointless, because I definitely do my fair share of behavior testing in my codebases. But again, testing behavior should be subsequent of a rubust unit-tested foundation. “The car broke down, why? Cylinder #8 injector voltage below threshold; Catalyst system Bank 2 system too lean; etc. etc.”. All things point to my injector in Cylinder #8 failing or failed.

    Testing behavior is good, but unit testing should be the higher priority.

    Love your blog, otherwise! :]

    Reply
  3. Ralph

    For anyone reading this in the future, this is untrue. I worked in Quality Control (AI & VP) at Toyota on summers before I became a software dev.

    Toyota at least absolutely does both forms of testing. To suggest that cars aren’t driven before they reach the consumer is laughable.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *