Forget Code Coverage

Forget chasing code coverage, in my view it is a somewhat pointless statistic and chasing it can (in some cases) lead to poor quality tests and a lot of wasted time. It can also lead to a false sense of security as to the overall health of your code.

Let’s look at what the statistic on its own can tell us….

Code Coverage 90%

• Can we tell if the system is working? No
• Can we tell if it meets Customers requirements? No
• Are we able to say if the app is well written? No

All this stat tells us is that at least 1 test case exists that executes 90% of the codebase. That test may not even have an assert or any means to fail for a breaking code change.

I fully understand why this metric exists and why some Company’s choose to set minimum levels, as it is impossible to measure code quality with a simple statistic. Quality is subjective.

If you have poor tests, but high coverage it can lead to a false sense of security that the application or codebase is well tested and of high quality, but neither of these can be inferred simply form code coverage alone.

What we really want to know is…

• Does the system meet the requirements?
• Does it work as intended?
• Is the system easy to work on (extend and refactor)?
• Can I rely on the tests to protect the behaviour?

There are no stats available for these, so code coverage is here to stay, but there is another way to help you focus on quality.

Stop chasing coverage and let it follow you. By adopting TDD (and BDD) you are guaranteed high code coverage from day 1, and in addition (if done right using SOLID principles) you also get the knowledge that the codebase is generally well crafted, easier to work on and above all well tested with potential issues identified early in the development cycle (at one of the cheapest points to fix).

A quick note on 100% coverage, unless its quick to achieve, it’s probably not worth wasting too much time chasing that last few percent of the sake of it – unless you can justify it. Do a quick mental cost-benefit analysis and ask is it really worth the extra effort. Depends on the system, sometimes it maybe yes, but more often than not, it will be no – Settle with the knowledge that you have covered as many use cases as possible and your tests and code are well written.

In short – Focus on producing good quality tests for the required use cases (including error handling as appropriate for the system). Try TDD and or BDD and you will find that your code improves and you are no longer looking over your shoulder at the code coverage percentage – It’s just high by default, with a quality test suite underpinning the statistics which is far more valuable.

Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives