A client of mine recently asked: “How do you determine, when first opening a project, which areas of testing need attention?”
There are three methods I use for determining what areas of a test suite need attention: 1) looking at the test suite code for “test smells”, 2) asking the team “where it hurts”, and 3) actually working directly with the project.
Looking at the code for “test smells”
Just as there are code smells, there are test smells. The book xUnit Test Patterns: Refactoring Test Code (which I highly recommend) has a list of test smells in the back. Some of these smells are detectible by looking at the code. Some of them can only be found experientially. Here are some that can be discovered by looking at the test code without running it.
Developers Not Writing Tests
This one is obvious. If there are very few tests in the test suite (or if the test suite doesn’t exist at all), then this is a clear opportunity for improvement.
Obscure Test
An Obscure Test is one where it’s hard to understand the meaning of a test just by looking at it, usually because the meaning is obscured by the presence of so many low-level details. These are relatively easy to spot.
Test Code Duplication
Duplication is also pretty easy to spot. It’s also often relatively cheap and straightforward to fix.
Asking the team “where it hurts”
Often a team is well aware of where their testing deficiencies lie. Common problems include:
- Not enough tests
- Not enough testing skills on the team
- Slow tests
- Flaky tests
- Application code that’s too hard to test
These issues can be easily uncovered simply by asking.
Actually working directly with the project
This is often the most effective way of telling which areas of the test suite need the most attention, although it’s also the slowest.
One challenge in identifying areas of a test suite that need attention is that not all tests have equal value. The test that tests the contact page is not as valuable as the test that tests the checkout page. So effort spent improving the contact page test might result in a much better test but the exercise still would have been a waste of time.
Another challenge is that not all code is equally testable. Let’s say I have an application with zero tests and no testing infrastructure. Logic might seem to dictate that I should write a test for the checkout page before I write a test for the contact page because the checkout page is much more economically valuable than the contact page.
But if my application lacks testing infrastructure than the task of writing a test for the checkout page might be a prohibitively large undertaking because it would involve not only writing complex tests but also setting up the underlying test infrastructure. Maybe I’d be better off writing a test for the contact page first just to get a beachhead in place, then gradually work my way up to having tests for the checkout page.
These are things that might be hard to tell just by looking at the code. So when I start work on a new project, I’ll often use all three of these methods—looking at the code, talking with the team, working with the code—to determine which areas of the test suite need attention.