As a manager you love metrics. Code coverage is a good metric. But a very limited one. Don’t be naive : a 100% code coverage is impossible and not enough to ensure quality. You must do more.
Code Coverage And How To Get It
Code coverage helps you determine how many methods/classes in your software are covered by unit tests. It’s a way to know if some part of your code is totally untested.
When developing with Eclipse you can use the MoreUnit plugin for example to help you see rough class coverage like that :
Classes that have unit tests appear with a green ribbon on their icon. But it won’t tell you if you all your methods are covered. For this you have another view in Eclipse with MoreUnit to show you what’s missing.
On your integration server (remember Hudson/Jenkins ?) plugins like PMD will help you compute more detailed stats of your code. But don’t be fooled by statistics.
Why You Can’t Have A 100%
Remember the episode 3 ? You can’t test everything. So you can’t have all your methods covered. Some of them are doing database access, others are using Eclipse RCP so you can’t load the environment, etc.
And that’s not the goal anyway !
Even if the code is complex, tests should not be. This is why writing tests is harder than code sometimes.
Good Tests Not Good Coverage
It’s a question of test quality.
And here we are : in the infinite loop. Tests should have great quality. But sometimes they are just assertTrue(true); statements.
So do we need tests for tests ? Not really. But good practices and rigor is necessary to have useful tests.
You have other tools too : code and test reviews, peer programming (I know : it’s the worst budget nightmare of all managers, but believe me it pays off ).
How To Write Good Tests
Here are a few tips to make cleaner, more useful unit tests :
- Use very explicit variable and method names
- Use very concrete tests (with hardwired dates, nothing computed, but real made up dates for example). Your test data should be manualy checkable.
- Test expected exceptions (with the @Test(expected=MyExceptionToTest.class)
- Use multiple methods to test the same method. Same as in the code : divide your classes in small pieces easy to read and maintain.
- Use comments and use them more than in your code. In your code comments should be only when something is really not explicit. But in unit tests you must detail the reasons of a test. Remember tests are specifications.
- Set up an environment with a known In and Out hypothesis
- Call the method to test
- Compare the exit with the excpeted out
- If different : crash !
An Example At Air France
Some programmer had to maintain unit tests in a very complex decision making environment. This colleague encountered the problem that one of the unit test crashed. But he didn’t know the method in itself and even the test in itself.
The test was badly written (according to him), not obvious an buggy. This is the opposite of real important unit test. You could have a hundred tests per class but if not maintained and fixed they are useless.
TODO
- Try to read some tests if you are not a programmer. Because if you can’t read them, there is a problem.
For the developers out there
How good are your tests ? Can you read others tests ? If not, ask them to review their tests together and clarify them.
