‘100% code coverage, unless…’

When asked about my point of view about a guideline concerning code coverage, my answer always is: go for 100% code coverage. 100%, unless… Here is why.

When the guideline for code coverage is 80% (not an uncommon guideline) the devil is in the details. Or in the 20% which is not covered to be exact. When a developer ‘only’ needs to cover 80% of his code, you can expect him (or her) to start with the easiest scenario’s, and work his way towards the more difficult scenario’s until the coverage guideline has been met. This way, the scenario’s which are the most difficult to realize as a test aren’t hit. And, more important, the developer didn’t think about the most difficult scenario’s and how to test them. In that case the remaining 20% is completely unknown*. I know several testers and I can assure you: they don’t like that.

In case developers need to come up with 100% code coverage, or a good reason why they can’t get there (with a reasonable amount of effort), all code paths have been seen and analyzed. When a certain part if the code is not covered you can assert the risk, assess how much effort is needed to get the code coverage up to par, and make a weighed decision to do so, or not. In the last case you have an entry for the ‘unless’ category.

Sure, sometimes the tooling doesn’t allow us to achieve a clean 100% code coverage. Visual Studio 2005 for instance sometimes skips a closing curly bracket, so you only come near the 100% for a specific method. That is one in the ‘unless’ category. The risk of not making the 100% has been asserted, the reason for it has been identified and a decision has been actively made.

Notes:
– You should not only look at the code coverage numbers. The quality of the asserts is pretty important too, as I mentioned earlier.
– When using [wikipedia:Test Driven Development] (strictly), these situations should be rare. Normally you start out with test cases, and then write code to make the test pass. Some even think that every line of code should add to a test case passing and that, when a line of code doesn’t do so, you shouldn’t have written it.

• ‘Completely unknown’ is a bit heavy on the drama, but it adds to the story, don’t you think so? 😉

• Methodology
• Test