Contact Us

Upcoming: Code Coverage in CakePHP - Test How Well You Test

Posted on 27/4/08 by Tim Koschützki

Hey folks,

the last week I have been dealing with an implementation of code coverage analysis for the upcoming release 1.2 of CakePHP. For those of you who do not yet know what code coverage is, here is a short excerpt from Wikipedia:

Code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that inspects the code directly and is therefore a form of white box testing.

There are several different kinds of criteria to code coverage. The two most important ones are line coverage (or statement coverage as wikipedia puts) it and path coverage. Line coverage basically analyzes if each line of the subject-under-test has been executed during the test.

Path coverage examines if every possible route through the code has been executed. Since I rely on Xdebug for the implementation we are talking line coverage here.

So how is it going to work? Pretty simple actually. Whenever you run a CakePHP test case Cake assembles information in the background about which lines of your subject-under-test are called. Then it will display all code lines from the subject-under-test and mark those that were "covered" (ie executed by the testcase) and which were not. Lines that play no role for the code coverage percentage, such as multiline comments, will be displayed in a different color.

How could an example report look like? Here you go:

I have also implemented a diff-style report. It just lists all the lines that are not covered by your tests and then seven lines before and after each of those. This is what the diff view looks like for the cakephp core router.test.php:

Within the dev team we are currently discussing these two views. If you favor one of these, can recommend another or just want to voice your opinion on the looks of them, please do so by commenting. For your information: Code coverage analysis currently only works for separate test cases only. However, the plan for this week is to get aggregate code coverage working. This means that your group tests are analyzed and the coverage percentage displayed for each of the test subjects involved.

If you ask me this is a very valuable tool when developing tests for your cakephp application. Although we are talking line coverage here and not path coverage (do your tests go through every possible paths through your application?) it is a good indicator of how thorough your tests are.

Please fire away your suggestions!

-- Tim Koschuetzki aka DarkAngelBGE


You can skip to the end and add a comment.

Neil Crookes said on May 27, 2008:

Nice one Tim, this looks great.

As to which representation of the code coverage, or lack thereof, I prefer - its difficult isn't it, because each best suits different situations.

The first example is good if your subject under test is short or your code coverage is quite bad, whereas the latter is better if your subject under test is long and your code coverage is good.

Perhaps you could present one or the other depending on some criteria. A combination of number of lines and the percentage code coverage?

Tim Koschützki said on May 27, 2008:

Hey Neil can you post that comment in this post:

Then I can delete this post here. Thanks. I will reply to your comment on the other post.

This post is too old. We do not allow comments here anymore in order to fight spam. If you have real feedback or questions for the post, please contact us.