As a manager you love metrics. Code coverage is a good metric. But a very limited one. Don’t be naive : a 100% code coverage is impossible and not enough to ensure quality. You must do more.
Code Coverage And How To Get It
Code coverage helps you determine how many methods/classes in your software are covered by unit tests. It’s a way to know if some part of your code is totally untested.
When developing with Eclipse you can use the MoreUnit plugin for example to help you see rough class coverage like that :
Classes that have unit tests appear with a green ribbon on their icon. But it won’t tell you if you all your methods are covered. For this you have another view in Eclipse with MoreUnit to show you what’s missing.
On your integration server (remember Hudson/Jenkins ?) plugins like PMD will help you compute more detailed stats of your code. But don’t be fooled by statistics.
Why You Can’t Have A 100%
Remember the episode 3 ? You can’t test everything. So you can’t have all your methods covered. Some of them are doing database access, others are using Eclipse RCP so you can’t load the environment, etc.
And that’s not the goal anyway !
Even if the code is complex, tests should not be. This is why writing tests is harder than code sometimes.
Good Tests Not Good Coverage
It’s a question of test quality.
And here we are : in the infinite loop. Tests should have great quality. But sometimes they are just assertTrue(true); statements.
So do we need tests for tests ? Not really. But good practices and rigor is necessary to have useful tests.
You have other tools too : code and test reviews, peer programming (I know : it’s the worst budget nightmare of all managers, but believe me it pays off ).
How To Write Good Tests
Here are a few tips to make cleaner, more useful unit tests :
- Use very explicit variable and method names
- Use very concrete tests (with hardwired dates, nothing computed, but real made up dates for example). Your test data should be manualy checkable.
- Test expected exceptions (with the @Test(expected=MyExceptionToTest.class)
- Use multiple methods to test the same method. Same as in the code : divide your classes in small pieces easy to read and maintain.
- Use comments and use them more than in your code. In your code comments should be only when something is really not explicit. But in unit tests you must detail the reasons of a test. Remember tests are specifications.
- Set up an environment with a known In and Out hypothesis
- Call the method to test
- Compare the exit with the excpeted out
- If different : crash !
An Example At Air France
Some programmer had to maintain unit tests in a very complex decision making environment. This colleague encountered the problem that one of the unit test crashed. But he didn’t know the method in itself and even the test in itself.
The test was badly written (according to him), not obvious an buggy. This is the opposite of real important unit test. You could have a hundred tests per class but if not maintained and fixed they are useless.
TODO
- Try to read some tests if you are not a programmer. Because if you can’t read them, there is a problem.
For the developers out there
How good are your tests ? Can you read others tests ? If not, ask them to review their tests together and clarify them.
Thanks for the interesting series. However I can not agree with this:
Use comments and use them more than in your code. In your code comments should be only when something is really not explicit. But in unit tests you must detail the reasons of a test. Remember tests are specifications.
Using comments in both code and test is discouraged since there is no cheap (automated) way to tell the code and comments are out of sync. This is the same reason why tests are considered to be better specification than paper specs. If you infest your test code with comments explaining things not expressed in code, it is not good specification anymore.
Oliver.
Thanks for you comment Oliver.
I see what you mean. It’s true : the code in itself in the test can change.
But the whole purpose of each test won’t move that much I think.
A single line to explain what you are testing and how is quite easy to maintain. The method you test can change inside, but it’s purpose is still the approximately the same don’t you think ?