Release early, Release often, CakeTaster
Posted on 16/11/06 by Felix Geisendörfer
Deprecated post
The authors of this post have marked it as deprecated. This means the information displayed is most likely outdated, inaccurate, boring or a combination of all three.
Policy: We never delete deprecated posts, but they are not listed in our categories or show up in the search anymore.
Comments: You can continue to leave comments on this post, but please consult Google or our search first if you want to get an answer ; ).
Ok I just realized that I've been following a bad pattern on two of my little side projects. The pattern of not releasing before everything is as 'perfect' and feature complete as it can be. Those projects are the CakeTaster test suite and also my Svn/Ftp Deployment Task for bake.
I guess the reason for that is that my blog has turned into my #1 resource for marketing myself since I've started it in January. This causes me to feel hesitant about publishing things that are unfinished, might have bad bugs in them and could make myself look like a poor programmer. Well, the truth is, the poorest programmer of all is the ones that doesn't release. Because what good am I doing to the world when tinkering for for month on things people are in bad need for today, just to make myself look like all code I ever write is perfect the first time. None at all. Therefor I just took 20 minutes, cleaned up the current code of the CakeTaster test suite and decided to release it.
Download: You can download the 0.1 experimental release of CakeTaster here.
Install: Installing the testsuite is simply a matter of unzipping the zip file and copying it over your existing application code. But just to be sure you should take a look in the folder before in order to make sure you don't accidentally overwrite an own controller.
Usage: There is only one kind of testing available right now, the functional testing of Controllers. There are already two sample tests inside the zip file, they will end up resting in /app/tests/functional and should serve as a good example of what a ControllerTest looks like. But just in case you are too lazy to download the package, here is one of the tests:
class PostsControllerTest extends ControllerTestCase
{
function test_index()
{
$response = $this->get('/posts/index');
$vars = $this->viewVars;
$this->assertTrue(!empty($vars['posts']), 'Posts array is not empty');
$this->assertTrue(true, 'True === True');
}
}
CakeTaster::runTest();
As you can see $this->get invokes a call to a given url, there is also a post functional as well as a parameter for arguments which I haven't tested so far. The class itself extends the UnitTestCase of SimpleTest and therefor supports all kinds of asserts that SimpleTest does. The get/post function will always contain the view rendered by the Controller, and $this->viewVars will contain, uhm, the view vars ; ).
In order to run the tests you have to browse to /tests. It should give a little interface allowing to filter for test names as well as to expand/collapse certain tests.
And for those of you who want to see what they are getting into before doing so, here comes a little screen shot:
Alright, I would love to hear any kind of feedback on the suite. I know it's not even close to being done but the basics are there. So if you find a bug, want to make a suggestion or something like this, let me know in the comments.
--Felix Geisendörfer aka the_undefined
Did you like this blog post? If so, please consider subscribing to the Blog RSS feed.
You can skip to the end and add a comment.
If your application is less than 5 controllers, and you can test it in entirety in about 10 minutes, then you'll probably be fine without having automated tests - just remember to run through the app completely before deploying a new version.
However, if your app is more like 30 controllers, and you know that after the first deployment any changes will need to be made within a working day (that is, to make the changes, and fully test on a test server, then deploy), having automated tests is really the only way to _know_ that everything is going to roll out smoothly...
We have one application that fits into this last category, and has no test cases (not a Cake application). Thankfully at this stage the changes that need to be made are very small, so are fairly easy to test manually. However, the stress of not having any real assurance that some random bug has not crept in is still quite uncomfortable.
Of course, I am preaching what I haven't yet implemented, so perhaps there is something to be read from that! :)
It depends on the size of your codebase, the purpose of your code and if you are working with other developers or QA. That is not to say that unit tests can be useful even for the smallest of projects but sometimes if you need to bridge a gap of some non-core task then pragmaticism will outweigh overhead. A better goal (and one which I think maybe shared by the author of this blog) is to reduce the overhead of implementing tests such that you can write them up front and run them easily. I have no hard data on this but I suspect that even the smallest of tasks could benefit by the more rigourous process you apply by putting down concrete demands on your functions. Assumptions get tested early and prevent those 5-minute tasks from being tied down on a false assumption.
I'm a big advocate of test driven development. Once you are clear about the interface and the things your app needs to do, start by writing a test case for the first Controllers action. Run the test, and see it fail (because the Controller doesn't exist yet). Then create the controller and try to write the least bit of code required to make the original test pass. Then go back and add more tests. There are severals advantages to this aproaches. You'll not write tests for things you don't need to be tested. The design of your code will become better because you'll be constantly looking at it from 2 perspectives (inside out, outside in). And most importantly, you'll be more likely to refactor while enjoying a better sleep ; ).
When adding the tests for things that already have been written, it might be difficult to set priorities. You'll randomly throw tests at pieces of your code you think should be tested. Those won't be terribly bad, but not anywhere as precise as the ones you get by using TDD.
Alright, I hope some of you guys are going to give this experimental release a little dry run and let me know what you think.
Hm, why do I have to call CakeTaster::runTest(); at the end of the file? Seems a bit illogical, as the test functions should be called automatically by SimpleTest.
Thanks for the crucially important work, Felix. Cannot wait to put it to use! Though, file cake_taster.namespace.php seems to be missing from the archive.
Sorry, my error. The file is in place.
Felix Geisendorfer's Blog: Release early, Release often, CakeTester...
...
Daniel: The call is there because I am for all the test files to also be standalone executables in the command line. So that you can directly call them up and run the test they contain. However, I think I could work around this call by reproducing CakePHP's __destruct support for php4.
Yaroslav: Give it a try, but plz keep in mind it's experimental. I would not rely on it for an important project right now. But your review would help me to improve and further develop the suite to that point.
Hm, wouldn't it be easier to have a separate command line script which executes the tests?
Daniel: This is planned as well. But I still want the stand-alone thing to work as I can imagine situations were it would be pretty neat to have.
Using your suite for several hours, great experience so far. Some changes were made in order to test my controllers: for example, AppControler extends TestableController. TestableController records the URL it is redirecting to rather than doing usual redirection when invoked from a test case. Under similar circumstances, TestableController uses alternative components for Session and Cookie - storing the persistence variables in the test case object rather than user's browser. So it is possible to set up the session environment before invoking the method and also chain-link the controller calls, imitating consecutive actions of a user.
Yaroslav: Thanks for your comment! I've checked out redirecting a little bit, but right now it seems in php4 your only chance to check for it is to overwrite the AppController's redirect function to store the informaiton in the controller. php5 knows a function called headers_list which would do the trick, but there is no php4 version of it. I'll talk with the other devs and see if there is a way to make CakePHP a little bit more test friendly.
Support for asserting session data and cookies is planned as well.
Let me know if you notice anything else that could be of importance so I'll know about it when doing the next iteration of the suite.
Great work and a really nice approach towards your, well, "fans" :D
I have fans? Why didn't anybody tell me earlier? Gotta start working on the merchandising right away!
No seriously, I'm glad you like the suite and I'll try to turn it into an awesome bake utility that will be able to handle all things testing. Really important for me right now are Model testing as well as fixtures. I'll keep you guys up to date.
This looks really nice. I am curious, though - is there a way to access the cake session component inside a test? I can't get it to work. I keep getting "Fatal error: Call to a member function write() on a non-object" errors.
hydra12: How did you try to access it. I haven't tested this, but using $this->Controller->Session should work after a $this->get() call. Let me know if that helps.
Still no luck. Here's my code:
get('/surveys/index/mark/password1/testing_suite');
$vars = $this->viewVars;
//debug($response);
//$this->Surveys->Session->write('logged', true);
//$this->Surveys->Session->write('username','mbuckner');
$this->assertTrue(!empty($vars['mySurveys']), 'mySurveys array is not empty');
$this->assertTrue($this->Surveys->Session->check('username'), 'username Session variable is set.');
}
}
CakeTaster::runTest();
?>
I'm still getting the same error for any attempt to access Session.
Sorry, some of it got cut off . . .
class SurveysControllerTest extends ControllerTestCase
{
function test_createSurvey()
{
$response = $this->get('/surveys/index/mark/password1/testing_suite');
$vars = $this->viewVars;
//debug($response);
//$this->Surveys->Session->write('logged', true);
//$this->Surveys->Session->write('username','mbuckner');
$this->assertTrue(!empty($vars['mySurveys']), 'mySurveys array is not empty');
// $this->assertTrue($this->Surveys->Session->check('username'), 'username Session variable is set.');
}
}
hydra12: When I said $this->Controller->Session I really ment "$this->Controller->Session". I didn't mean $this->{ControllerName}->Session. Try this and let me know how it goes ; ).
[...] After recently releasing my experimental little testsuite called CakeTaster it is time for me to release the next project I've been working on. This one might be even more interesting for most people, as it's already pretty advanced. Nevertheless I consider this to be an experimental release, so plz don't depend on it quite yet ; ). [...]
Hi,
I am new to cake php. i dont know how to create a file for example normal registration form
name, password ,mail id,phone number.i want to enter these values are post it will save to the database .please give me sample coding and procedures it is urgent.
how to write and where we have to change and what are the files are need to create a cake php.
Thanks
anand: No offense, but your request is very off-topic. This is my blog where I, amongst other stuff, write about my experiences with CakePHP. This is no channel for support of any kind besides the fact that I usually answer questions regarding the contents of my post. Please refer to the CakePHP manual, the CakePHP google group and the #cakephp IRC channel at freenode.net.
Thanks
Much needed!!! Thanks for putting the work into getting testing Cake Apps. What is your thinking on fixtures, and refreshing the DB for each test. I'm new to CakePHP, so I've just been doing it heavy handedly in a setUP() method in each test class. Something like this, but I'm sure there is a better way!
function setUp()
{
shell_exec("mysql -u xxxx -pxxxx db_name
argggg, cut me off.
Anyways, that shell_exec just loads up a sql file into the db that looks like this -
DROP TABLE IF EXISTS sites;
CREATE TABLE sites(
id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
domain VARCHAR(50),
created DATETIME DEFAULT NULL,
modified DATETIME DEFAULT NULL
);
Another question. How to force CakePHP into using the testing db setup as defined in /config/database.php when we are running test cases?
I can see how to do it model by model, but that is not what we want in this situation.
matt: Thanks for your interest. Fixtures, DB migration and other goodies are on my todo list (together with turning this into a stable project). However I'm very short on time these days because I'm fortuned with tons of client work (^^). But I also work on a "super secret" web app that might allow me to lighten this weight (it'll be commercial) in future so I can spent more time on things like this.
As for your last question: You can create an AppModel::__construct() function that detects if you are in testing mode (check if some SimpleTest class is defined or something similiar) and then switch DB for all Models. Let me know if that works.
I've spent nearly the whole day in CakeTaster, and yea it has a long way to go, but what you've done so far is really well done, and very useful.
Have you come up with a way to set cookies for the request you are testing.
In rails, you do something like this -
@request.cookies['name'] = CGI::Cookie.new('name', 'cookie value')
then run your test.
Something along the same lines would be cool. I'll dig into it a bit, and see what can be done.
matt: Sorry for the late response. No I've not done anything for cookie testing yet and I'll yet have to play with the new testing suite of Cake 1.2. If I make any significant progress on the entire testing topic I'll mention it in a blog post on here soon.
Nice work! I'm just getting settled in cakephp but one of the first things i was looking for was testing tools. Thanks for releasing before it was perfect. I'm a big fan of agile. :)
I had to make a slight change to get tests working. I'm using cake v.1.1.13. viewVars kept coming up null until I removed the pre-appended underscore around line 132 in libs/controller_test.php. Now the viewVars are coming in nicely and tests are passing.
Could this due to a version difference?
Diona: Yes this is caused by a change in the CakePHP core that was made a while ago.
Hello,
is there any new version of you suite adapted to the 1.2 version?
thanks, regards
danielz
Hi Daniel: No as of right now there is no such version and I'm not working on one. However I'm thinking about bringing the joys of integrational controller testing to the new testsuite in 1.2 if I find some time ; ).
After installation of the caketaster i run the url like http://localhost/application_name/tests and get this Fatal error: Cannot redeclare class caketaster in C:\Program Files\xampp\htdocs\forum\vendors\caketaster\libs\cake_taster.namespace.php on line 4
would you plz help me with this?
Hi Felix,
want to thank you for your great work. Just managed to get your Suite running with cake 1.2.xx beta 5875. Had to tweak your code a little so if anyone needs it, just post here and i will help if i can :-)
greetings
Lars
Hi Lars,
I'd like to try the TestSuite, so if it's possible, please share your tweaks with me.
This post is too old. We do not allow comments here anymore in order to fight spam. If you have real feedback or questions for the post, please contact us.
You know what, I am yet to find a good reason why I should use tests at all. Surely actual usage of the app is better. Or is it just me maybe not appreciating what tests can do?