The Upside of Coverage Testing
| Posted on Wednesday, October 01, 2008 |
0
commentsAdd comments |
This is part one of a two part discussion on Coverage Testing. The complement to this discussion is The Downside of Coverage Testing. These discussions are follow-ups from my blog Stable Builds: Using Ant and QA Tools.
Coverage testing tools, such as Emma, provide a developer the means to test all lines, conditional branching, and routine entry of a unit of code. Unit here denotes a chunk of code that is exercised as a unit—a method, a class, a component, or a system. The goal is to obtain 100% coverage of the targeted unit’s code. The benefit to coverage testing is in the assurance that every line of the unit has been tested and has passed that testing. As the complexity of a project increases, especially in regards to the development of an entire system, coverage plays a vital role in managing the development lifecycle by ensuring that integration of new units are thoroughly tested and that their integration do not break the existing build.
To wet my feet, so-to-speak, I decided to do coverage testing of the
Baseline
One must first assess where he stands, get a baseline, when testing, performing maintenance, or instituting a repair. A run of Emma against the existing
First Aid
A coverage tool, and all Quality Assurance tools for that matter, is part of a developer’s first aid kit in code triage. So, I set about trying to bandage the
Setting Splints
To achieve 100% coverage required the addition of some test cases within the
Covering the untested
Coding-After or Bug Control Pills
However you swallow it, coverage testing is helpful in obtaining stable code. Testing of any type is better protection against unplanned bugs, than no testing at all. Coverage testing ensures that code doesn’t lay fallow within an implementation—all code is shallow—and that it has at a minimal been looked at least once beyond the act of hacking it out. Coverage tools are simple to use, once incorporated into a developer’s routine, and become insurance policies allaying concerns of integrating new code with old code. While the
My
Coverage testing tools, such as Emma, provide a developer the means to test all lines, conditional branching, and routine entry of a unit of code. Unit here denotes a chunk of code that is exercised as a unit—a method, a class, a component, or a system. The goal is to obtain 100% coverage of the targeted unit’s code. The benefit to coverage testing is in the assurance that every line of the unit has been tested and has passed that testing. As the complexity of a project increases, especially in regards to the development of an entire system, coverage plays a vital role in managing the development lifecycle by ensuring that integration of new units are thoroughly tested and that their integration do not break the existing build.
To wet my feet, so-to-speak, I decided to do coverage testing of the
Stack implementation from one of my earlier blogs (noted above). The coverage tool being used is Emma, which according to its User Guide “is essential for detecting dead code and verifying which parts of your application are actually exercised by your test suite and interactive use.” While the Stack implementation is a trivial unit for testing, it should suffice in demonstrating the upside of coverage testing.Baseline
One must first assess where he stands, get a baseline, when testing, performing maintenance, or instituting a repair. A run of Emma against the existing
Stack implementation indicates that it does not have complete coverage. While all of the classes in the implementation had been tested, not every method (73%), code block (69%), or line (75%) had been.First Aid
A coverage tool, and all Quality Assurance tools for that matter, is part of a developer’s first aid kit in code triage. So, I set about trying to bandage the
Stack implementation to reach full coverage of its methods, code blocks, and lines. Emma provides a summary level html page that displays the code, highlighting the untested portions. For the Stack implementation, only the EmptyStackException class had complete coverage, while the Stack and ClearStack classes each lacked or had deficient coverage testing of two of their methods. The Stack required further testing of its top() and toSting() methods, while ClearStack had no testing of its isEmpty() and getTop() methods.Setting Splints
To achieve 100% coverage required the addition of some test cases within the
Stack implementation. Since Emma had indicated where to focus my attention, it was an easy task to go about setting splints to mend the broken coverage. Tackling the ClearStack methods, I added the new tests testTopObject(), testEmptyStack(), and testNonEmptyStack. These tests covered reaching the getTop() method and its lines of code, while the other two tests reached the isEmpty() method and tested it for both true and false returns.Covering the untested
Stack methods were just as easy to resolve. I added testIllegalTop() to test for the throw of an EmptyStackException when calling top() on an empty stack. The remaining coverage was provide with the creation of a testToString() test.Coding-After or Bug Control Pills
However you swallow it, coverage testing is helpful in obtaining stable code. Testing of any type is better protection against unplanned bugs, than no testing at all. Coverage testing ensures that code doesn’t lay fallow within an implementation—all code is shallow—and that it has at a minimal been looked at least once beyond the act of hacking it out. Coverage tools are simple to use, once incorporated into a developer’s routine, and become insurance policies allaying concerns of integrating new code with old code. While the
Stack implementation may have been trivial, there was an extensive amount of it that wasn’t being tested. Had this been a critical (life-or-death) system, untested code could very well be a law-suit waiting to happen. Testing may not be able to guarantee that code is 100% error free, it is, however, due diligence on the part of the developer in aspiring to that end.My
Stack implementation with full coverage testing: Stack-Reeves-6.0.630.zip