My first experiment with achieving 100% line coverage caught two bugs in the last three lines covered. They were both trivial bugs in error handling. They happened because I had roughed out some ideas early in development about error handling, but did not have anything driving me to recognize when development had invalidated early assumptions—until I thought about how to test those error cases.
This summary in the morning paper reminds me of that experience. Adrian quotes from the paper:
“In 23% of the catastrophic failures, while the mistakes in error handling were system specific, they are still easily detectable. More formally, the incorrect error handling in these cases would be exposed by 100% statement coverage testing on the error handling stage… once the problematic basic block in the error handling code is triggered, the failure is guaranteed to be exposed.”
Note also: “They could have been detected by simple code inspection” — I find writing tests after coding to be a form of code review, because I think differently about my code when writing tests for it, bringing “different eyes” to it. Many of the bugs I find while writing tests I find before I execute the test; it’s the discipline of inspection that exposes them. Writing tests is not a substitute for good code review, but it does often find the low-hanging fruit and encourage refactoring into clearer code for which review is more meaningful.
Imported from Google+ — content and formatting may not be reliable