debuggable

 
Contact Us
 
4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12

CakePHP RemoveCache Shell - Remove Your Cache Files Easily

Posted on 4/6/09 by Tim Koschützki

Hey folks,

it's been a while since the last post here on Debuggable. However, this ends now and apart from this very post here we have something bigger brewing for you as well. So please stay tuned a little longer. :)

Anyways, on to some Caching fun.

The Motivation

1. When you work with large projects you can end up with a ton of cache files (models, db cache, etc.) as well as view caching files. When you try removing them with:

cd /app/tmp/cache/models && rm -f cake_*

and

cd /app/tmp/cache/views && rm -f *

.. you can end up with with the "argument list too long" error easily.

2. Also, executing two commands sucks - we are lazy after all. Now you could pull out some fancy bash fun to pipe file names. Have a look at this:

find . -type f | awk '!/empty/ {print "rm", $0}' | bash

The problem is, when you run this in /app/tmp it will not only remove cache files, but also files in /tmp/sessions, /tmp/logs and so on. If you ask me, the command is complex enough, so no need to add more funny stuff there to take this into account.

(For you peeps who want to see this, I bugged Felix to tell me: find . -type f | awk '!/empty$|^.\/logs|^.\/sessions/ {print "rm", $0}' | bash)

3. Once you are on windows, you do not have a powerful bash to your side.

I thought a simple call to a CakePHP shell can do the trick as well and doesn't force you to waste half a minute to remember and type in the proper bash command.

The Solution

The RemoveCache shell allows you to remove your cae cache files easily. It takes two parameter:

  • A boolean to control if you want to remove standard cache files (models, db cache, etc.)
  • A regex pattern to control which view cache files to remove

Here are some common usage scenarios:

Usage: cake remove_cache <std_cache_boolean> <pattern_to_match_viewcache_files>
Usage: cake remove_cache // removes all cache files
Usage: cake remove_cache 0 // removes only view cache files
Usage: cake remove_cache 0 home // removes only the view cache file for your homepage
Usage: cake remove_cache 0 articles_ // removes all view cache files for your articles controller
Usage: cake remove_cache 1 /letter_z$/ // removes all std cache files and view cache files ending with 'letter_z'

I did not put in a pattern for standard cache files, because most of the time you cannot remember your cache keys anyways and most of the time it doesn't harm if a cache file is invalidated to rebuilt the cache. If someone wants a pattern for that too, because they have long-taking queries, just comment and I will add it.

By default, the shell looks in your standard /app/tmp directory (plus subfolders) to find the cache files. If you have a shared Cake installation or any other fancy setup, please adjust the cache paths in the inititalize() method.

The Code

<?php
/**
 * Remove Cache Shell
 *
 * This shell allows you to remove cache files easily and provides you with a couple configuration options.
 * If run with no command line arguments, RemoveCache removes all your standard cache files (db cache, model cache, etc.)
 * as well as your view caching files.
 *
 *
 * RemoveCache Shell : Removing your Cache
 * Copyright 2009, Debuggable, Ltd. (http://debuggable.com)
 *
 * Licensed under The MIT License
 * Redistributions of files must retain the above copyright notice.
 *
 * @filesource
 * @copyright     Copyright 2009, Debuggable, Ltd. (http://debuggable.com)
 * @link          http://www.cakefoundation.org/projects/info/cakephp CakePHP(tm) Project
 * @license       http://www.opensource.org/licenses/mit-license.php The MIT License
 */

class RemoveCacheShell extends Shell {
/**
 * undocumented function
 *
 * @return void
 * @access public
 */

  function initialize() {
    parent::initialize();

    $this->settings = array(
      'view_cache_path' => APP . 'tmp' . DS . 'cache' . DS . 'views',
      'std_cache_paths' => array(
        APP . 'tmp',
        APP . 'tmp' . DS . 'cache',
        APP . 'tmp' . DS . 'models',
        APP . 'tmp' . DS . 'persistent'
      )
    );
  }
/**
 * undocumented function
 *
 * @return void
 * @access public
 */

  function main() {
    $args = $this->args;

    $stdCache = !isset($args[0]) || $args[0];
    $viewCachePattern = isset($args[1]) ? $args[1] : '.*';

    if ($stdCache) {
      $this->_cleanStdCache();
    }

    $this->_cleanViewCache($viewCachePattern);
  }
/**
 * Cleans the standard cache, ie all model caches, db caches, persistent caches
 * Files need to be prefixed with cake_ to be removed
 *
 * @return void
 * @access public
 */

  function _cleanStdCache() {
    $paths = $this->settings['std_cache_paths'];

    foreach ($paths as $path) {
      $folder = new Folder($path);
      $contents = $folder->read();
      $files = $contents[1];
      foreach ($files as $file) {
        if (!preg_match('/^cake_/', $file)) {
          continue;
        }
        $this->out($path . DS . $file);
        @unlink($path . DS . $file);
      }
    }
  }
/**
 * Cleans all view caching files. Takes a pattern to match files against.
 *
 * @param string $pattern
 * @return void
 * @access public
 */

  function _cleanViewCache($pattern) {
    $path = $this->settings['view_cache_path'];

    if ($pattern{0} != '/') {
      $pattern = '/' . $pattern . '/i';
    }

    $folder = new Folder($path);
    $contents = $folder->read();
    $files = $contents[1];
    foreach ($files as $file) {
      if (!preg_match($pattern, $file)) {
        continue;
      }
      $this->out($path . DS . $file);
      @unlink($path . DS . $file);
    }
  }
/**
 * undocumented function
 *
 * @return void
 * @access public
 */

  function help() {
    $this->out('Debuggable Ltd. Remove Cache Shell - http://debuggable.com');
    $this->hr();
    $this->out('Important: Configure your paths in the shell\'s initialize() function.');
    $this->hr();
    $this->out('This shell allows you to remove cache files easily and provides you with a couple configuration options.');
    $this->out('If run with no command line arguments, RemoveCache removes all your standard cache files (db cache, model cache, etc.) ');
    $this->out('as well as your view caching files.');
    $this->out('');
    $this->out('Set the first parameter to 0 (zero), to not remove standard cache files.');
    $this->out('Set a regex pattern for the second argument, to match viewcache files to delete.');
    $this->hr();
    $this->out("Usage: cake remove_cache <std_cache_boolean> <pattern_to_match_viewcache_files>");
    $this->out("Usage: cake remove_cache \t\t// removes all cache files");
    $this->out("Usage: cake remove_cache 0 \t\t// removes only view cache files");
    $this->out("Usage: cake remove_cache 0 home \t// removes only the view cache file for your homepage");
    $this->out("Usage: cake remove_cache 0 articles_ \t// removes all view cache files for your articles controller");
    $this->out("Usage: cake remove_cache 1 /letter_z$/ \t// removes all std cache files and view cache files ending with 'letter_z'");
    $this->out('');
  }
}
?>

Enjoy! Feedback welcome.

-- Tim Koschuetzki aka DarkAngelBGE

 

The biggest CakeFest to be held in Berlin

Posted on 25/3/09 by Felix Geisendörfer

I will make this short. The 3rd and so far biggest CakeFest will be held in Berlin, home of Debuggable! From July 9-12, people from all over Europe and the rest of the world will travel to Germany in order to celebrate and learn about the best PHP framework there is.

If you have not attended a CakeFest so far, here are some good ideas of what to expect:

The event consists of two parts:

  • July 9-10: CakePHP Workshop - (Lead by the CakePHP core team)
  • July 11-12: CakePHP Conference - (Presented by the core team + community)

The 2-day workshop is packed with a series of tutorials designed to give developers, both new and seasoned, a solid understanding in building reliable CakePHP applications. Veterans and experts can skip over stuff they already know and use this time for 1 on 1 sessions with the non-presenting developers. The Workshop + Conference ticket is 599 EUR (499 EUR if you do not attend the conference).

The conference itself is going to be packed with talks, delivered by both the CakePHP core team as well as interested community members. At just 199 EUR (student discounts to be announced soon) there is no excuse for not attending.

I will talk on JavaScript for (Cake)PHP developers as well as share my experience in project management using GitHub + Lighthouse. Tim is still making up his mind, but will probably talk about Advanced Debugging.

The location for the conference is a few streets down in my neighborhood, so be assured that we will have more than enough opportunity to gather & celebrate in the evenings.

If you have any questions, please feel free to post them here or email them to me at felix@debuggable.com.

Otherwise go ahead and sign up, you will not regret it as the conference will also feature some major announcements exclusively made there.

- -Felix Geisendörfer aka the_undefined

 

Git alias for displaying the GitHub commit url

Posted on 18/3/09 by Felix Geisendörfer

If you often find yourself pointing your team members to commit urls in GitHub, this might be fun for you.

I created git alias called 'hub' that automatically guesses the github repository url for the repository you are currently in:

$ git hub
https://github.com/felixge/my-project

Based on that I created a second alias called 'url', which gives you the url to HEAD commit:

$ git url
https://github.com/felixge/my-project/commit/0bdc57323a1ffec7ffe10bf83147cab5d6838d45

You can however also provide another sha1 you want to link to:

$ git url 22db8914220b717b0954b84365030ae3c9602a17
https://github.com/felixge/my-project/commit/22db8914220b717b0954b84365030ae3c9602a17

If you find those aliases useful, here are my ~/.gitconfig alias definitions for them:

[alias]
  hub =! echo "https://github.com/"`git config remote.origin.url` | sed -E s/[a-z]+@github\.com:// | sed s/\.git$//
  url =!sh -c 'HEAD=`git rev-parse HEAD` && SHA1=`[ "$0" = "sh" ] && echo $HEAD || echo $0` && echo `git hub`"/commit/"${SHA1i}'

(Bash gurus: I am sure you can do the above much more elegantly, wanna give it a try?)

Further I also have this little bash script in ~/bin/tiny:

#!/bin/bash
curl "http://tinyurl.com/api-create.php?url=$1"
echo ""

This allows me to make tiny urls for my links if I need to paste them on long-url unfriendly territory:

$ git url | xargs tiny
http://tinyurl.com/cva44t

Or if you are on OSX, you can use this to open the git url in your browser:

$ git url | xargs open

Now all of the above could probably be done much smarter, but as far as I am concerned it works great ; ).

-- Felix Geisendörfer aka the_undefined

PS: What command line tricks are part of your daily workflow?

 

Muscles on demand - Clean a large git repository the cloud way

Posted on 13/3/09 by Felix Geisendörfer

Hey folks,

don't you hate it when you sometimes have to stop your work because your dev machine is ultra-busy doing some CPU or I/O heavy operations that will take hours?

Even so it doesn't happen to me a lot, I actually ran into such a case last night while trying to fix the Git repository of a project we are working on. The repository itself was not corrupted, but it became so fat that git-index-pack would explode on many of the team members. How did that happen? Well it turns out that over time some of the image directories of the project were committed into the repository by accident. This ended up being an insane 1.7 GB of '.git' objects.

With SVN, this is when you realize you made a poor choice in versioning control software and it is time to start the repository over - loosing all history.

Not so much wit Git. Git has an excellent tool called git-filter-branch that you can use to rewrite an entire repositories history.

In our case we wanted to pretend app/webroot/files had never existed:

git filter-branch --index-filter 'git rm -r --cached app/webroot/files' -- --all

However, our repository is full of commits (3.5k) and as I mentioned has some incredibly huge and ugly blobs in it. This means the operation above is slow like crazy.

So instead of having my poor laptop tortured with it for an hour, I decided to hire some muscle. I knew the operation I planned to do was fairly CPU and I/O heavy. For that reason I fired up an c1.xlarge Ec2 instance featuring 8 cores and 7 GB RAM.

Launching one of Eric Hammond's excellent Ubuntu EC2 AMI's and installing git took < 5 minutes. Add a few minutes to transfer the 1.7GB repository over and I was ready to go.

Having recently read Kevin's excellent article on tmpfs, I just put the repository in /dev/shm. This simply meant that the repository was now fully stored in memory - 30x faster then HDD!

Even with all this power, the whole process still took 15 minutes to complete, but the result was impressive. Instead of 1.7GB the repository was shrunk down to 80MB and little angels were dancing & singing around it. It was beautiful : ).

I pushed the lean and mean clone up to github using 'git push -f' and then switching over the local clones of each team member was just a matter of:

git checkout -b backup-master
git branch -D master
git fetch
git checkout -b master origin/master

Of course the new master branch and its backup wouldn't be very nice to each other as far as merges are concerned, but cherry picking the most recent commits worked great.

As you can see cloud computing is not only for the application level, but it can also be a great tool for your development process as a whole. After all it is incredible what can be accomplished for $0.80 this way ; ).

-- Felix Geisendörfer aka the_undefined

 

How to render fixed length rows of items

Posted on 9/3/09 by Felix Geisendörfer

Hey folks,

if you find yourself in a situation where you need to write a template that splits up a list of items in multiple rows with each row having a fixed amount of items, here is a simple solution:

<div class="items">
  <?php while ($items): ?>
    <div class="row">
      <?php for ($i = 0; ($item = array_shift($items)) && $i < 4; $i++): ?>
        <div class="item">
          <?php echo $item['Item']['title']; ?>
        </div>
      <?php endfor; ?>
    </div>
  <?php endwhile; ?>
</div>

This will put 4 items in each row and a smile on your face for not having to write very ugly code to achieve the same : ).

-- Felix Geisendörfer aka the_undefined

 
4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12