The Beauty of Code Coverage
A lot of people don't realize that with Visual Studio Team System you can get code coverage results even when using manual tests. I was showing a team how to perform code coverage on manual tests, the results they get and how valuable this is. Their testing processes are very immature so an ideal place to start are manual tests. I can hear the cries from the TDD zealots now about unit testing being the best start. I often agree but am also pragmatic and must take each situation into account. This situation needs manual tests before unit tests. Results over form. Back to the story.
It was a nice experience to just take some code that has existed for some time, create a quick manual test on it, set the testconfig to instrument the assemblies of interest and then perform the test. We passed the test and then I pulled up the code coverage results. It covered one specific aspx page that was the target with about 60% coverage. That's pretty darn good. I then pulled up the code behind on the page flipped the code coverage button to highlight code covered and voila. It was instant gratification to see the room light up with recognition and to then point out that based on the code coverage I could now see two other scenarios we didn't check for. What if there are no results returned from the data access- we didn't test that situation. What if the results return only a single record- we didn't test that path. The code coverage results helped us determine additional paths to test without having to think too hard about it.
I'm a strong proponent that the goal in testing is all about cumulative flow and trends. I don't care if coverage starts out at 1% I just want to see trends for that number going up and with an asymptotic approach toward 100%. The last 20% will be much harder than the first 80% and there are laws of diminishing returns at play. If you have 100% code coverage on all your code then you're a sick person that I would argue is wasting time building tests to a level of the most minimal return. I mean that in the nicest of ways. :-)
I just wanted to share that you really should instrument code coverage on your manual tests and then look at what you missed. Based on that select new areas to then cover by tests (of any sort). And I just wanted to share a bit of a victory in one of my own teams.