One of the testing questions I commonly get is about Shoulda matchers. People ask if I use Shoulda matchers and if Shoulda matchers are a good idea.
I’ll share my thoughts on this. First I’ll explain what Shoulda is, then I’ll explain why I don’t use it.
What Shoulda is
If you’re unfamiliar with Shoulda matchers, the premise, from the GitHub description, is: “Shoulda Matchers provides RSpec- and Minitest-compatible one-liners to test common Rails functionality that, if written by hand, would be much longer, more complex, and error-prone.”
A few examples of specific Shoulda matchers are validates_presence_of (expects that a model attribute has a presence validator), have_many (expects that a has_many association exists), and redirect_to (expects that a redirection takes place).
I like the idea of a library that can clean up a lot of my repetitive test code. Unfortunately, Shoulda matchers only apply to the kinds of tests I would never write.
Test behavior, not implementation
To me it doesn’t make much sense to, for example, write a test that only checks for the presence of an Active Record association and doesn’t do anything else.
If I have an association, presumably that association exists in order to enable some piece of behavior, or else it would be pointless for the association to exist. For example, if a User class has_many :posts, then that association only makes sense if there’s some Post-related behavior.
So there are two possibilities in light of testing that the User class has_many :posts. One is that I write a test for both the association itself and the behavior enabled by the association, in which case the test for the association is redundant and adds no value. The other possibility is that I write a test only for the post association, but not for the post behavior, which wouldn’t make much sense because why wouldn’t I write a test for the post behavior?
To me it only makes sense in this example to write tests for the post behavior and write no tests directly for the association. The logic of this decision can be proved by imagining what would happen if the has_many :posts line were removed. Any tests for the post behavior would start failing because the behavior would be broken without the association line present.
Takeaway
Don’t test low-level implementations. It’s pointless. Test behavior instead.
Since Shoulda is only good for testing low-level implementations, I don’t recommend using it.
You never explained why shoulda matchers are used only for testing associations? I thought there are as general as any other matches, just more succinct.
I don’t use Shoulda matchers only for testing associations, I use Shoulda matchers only for testing validations. I don’t use Shoulda matchers for associations.
You say it doesn’t make sense to write a test that checks only for the presence of an ActiveRecord association, but why would you write a test for behavior that is already covered in Rails core? Do you extend this concept to every feature in every gem you include?
If your argument is that it’s pointless to write tests for validations, then I reluctantly admit that that’s actually a pretty good point. Writing a test for the presence of a validation isn’t as illogical as writing a test for the presence of e.g. a has_many association, but it’s still of dubious value.
I see no difference in testing associations vs. validations unless you’re testing a custom validation, conditional validation, or an association with conditions etc.
You may want to write a test that the validation message appears
I think the whole shoulda-matchers library is pointless because it’s just testing Rails DSL/macros.
As developers, we trust that Rails works.
Should I be testing the entire middleware stack in Rails too? Rack? All the gems in my gemfile?
Fair enough. I have a slightly different take on “we trust that Rails works” though. To me it has nothing to do with trust. I write tests that verify that my code works, and if Rails somehow broke, then my features would break too. And if Rails breaks in a way that doesn’t test my features, then I have no reason to care. No trust needed.
I actually looked at the implementation of the validation matchers after I posted this, and internally it does check the behavior rather than whether you typed “validates_presence_of” or whatever. So I retract my statement! 🙂
I looked at association matchers and the point stands. It’s just using reflections to check that the association exists.