debuggable

Hey folks,

the last week I have been dealing with an implementation of code coverage analysis for the upcoming release 1.2 of CakePHP. For those of you who do not yet know what code coverage is, here is a short excerpt from Wikipedia:

Code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that inspects the code directly and is therefore a form of white box testing.

There are several different kinds of criteria to code coverage. The two most important ones are line coverage (or statement coverage as wikipedia puts) it and path coverage. Line coverage basically analyzes if each line of the subject-under-test has been executed during the test.

Path coverage examines if every possible route through the code has been executed. Since I rely on Xdebug for the implementation we are talking line coverage here.

So how is it going to work? Pretty simple actually. Whenever you run a CakePHP test case Cake assembles information in the background about which lines of your subject-under-test are called. Then it will display all code lines from the subject-under-test and mark those that were "covered" (ie executed by the testcase) and which were not. Lines that play no role for the code coverage percentage, such as multiline comments, will be displayed in a different color.

How could an example report look like? Here you go:

I have also implemented a diff-style report. It just lists all the lines that are not covered by your tests and then seven lines before and after each of those. This is what the diff view looks like for the cakephp core router.test.php:

Within the dev team we are currently discussing these two views. If you favor one of these, can recommend another or just want to voice your opinion on the looks of them, please do so by commenting. For your information: Code coverage analysis currently only works for separate test cases only. However, the plan for this week is to get aggregate code coverage working. This means that your group tests are analyzed and the coverage percentage displayed for each of the test subjects involved.

If you ask me this is a very valuable tool when developing tests for your cakephp application. Although we are talking line coverage here and not path coverage (do your tests go through every possible paths through your application?) it is a good indicator of how thorough your tests are.

Please fire away your suggestions!

-- Tim Koschuetzki aka DarkAngelBGE