Workflow Automatisation
Posted on 7/8/06 by Felix Geisendörfer
Deprecated post
The authors of this post have marked it as deprecated. This means the information displayed is most likely outdated, inaccurate, boring or a combination of all three.
Policy: We never delete deprecated posts, but they are not listed in our categories or show up in the search anymore.
Comments: You can continue to leave comments on this post, but please consult Google or our search first if you want to get an answer ; ).
I'm currently reading The Pragmatic Programmer (shame on me for not having that done earlier). One of the many advices in the book includes to make more use of the console if possible. Now I'm on Windows - there is no console one would really want to use - so I got myself a fresh copy of cygwin and started playing around with it.
I have to admit, I'm very impressed so far. I've never really gotten into *nix systems, not because I wasn't interested, but because I was already very familiar with Windows and hated not being able to do all the things I already knew how to do on Win32 in Linux.
Anyway, now that my bash scripting skills are slowly building up, I'm very exited about the cool things I could do with it from now on. Especially when it comes to automate the most annoying taks in my coding cycle. This especially includes keeping my development and production enviornments in sync (ftp, mysql, etc.), updating existing projects to the latest cakephp version, validating xhtml/css, running unit/integrational tests, and so on.
What totally amazes me, is how much of that can actually be automated and done better then I usally do it by hand. Let's just take updating CakePHP for example: I usally go to cakephp.org and download a nightly. Then I unpack the nightly, and take all files except the /app folder out of it, and overwrite my current project. Doesn't take too long, but it's really the very same process over and over and over again. And in case something in the /app folder changed, I'll have to manually find out what. Now imagine the same thing to work like this:
/$ project develco
Looking for Project "develco" ... Project found in: /www/develco - develco is now your active Project
/$ update
Checking for newer version Latest CakePHP Version is: 3382 Project develco's Version is: 3275 Do you want to (u)pdate, (s)ee the changelog, or (c)ancel?
/$ u
Applying diff Patch, please wait ... Update complete.
Now the fun part is, that coding this, is actually not that difficult, and you even get a patch from svn instead of lousy file replacement. And things like finding the current CakePHP version from the SVN repository can be done in one line of code:
echo p | svn info https://svn.cakephp.org/repo/branches/1.1.x.x | grep "Last Changed Rev: ([0-9].+)" -oP | grep "[0-9]+" -oP
(the echo p | is used to (p)ermantly accept the uncertified session you get when connecting to cakephp's svn server).
But to be honest, it's not really the time savings I'm after. Writing all of those scripts will probably take up several hours (some of it is already done). But what it will do, is to take away all those annoying tasks I have to do over and over again, which I hope will improve my behaviors. Just take testing for example. It's like cleaning up your desk, you should be doing it, things will get lost / go wrong if you don't, and you'll have a better feeling after you did it. Now cleaning up your desk will always remain a little annoyance, but what you want, is to get as little resistance in between you, and cleaning up your desk. Which means: Make testing as easy an comfortable as possible.
Alright, enough enthusiasm: What do you guys think? Does it makes sense to automate all this tasks I mentioned? What kind of tasks have you automated so far? I'm interested in hearing it all ; ).
--Felix Geisendörfer aka the_undefined
Did you like this blog post? If so, please consider subscribing to the Blog RSS feed.
You can skip to the end and add a comment.
I'm glad that you start getting familar with bash! Since I'm using linux exclusively for more than 3 years now (using Win only for testing my websites) I'm also a big fan of bash and in some cases perl.
Especially for all these tasks you mentioned I think it's not a waste of time to write little scripts. For example sometimes you have to pack new versions of your webapp as a zip-package for your client. That means deleting all temporary files, all uploaded data, and then zipping it. IMHO a perfekt task for a bashscript. I also use it to automate backups of projects which I do not hold in an SVN repository...
I dont think its a waste of time either, often when writing a little (or not so little) script i'll learn something new and for anything more than a few lines of shell script i normally move to python.
I do the usual stuff like deploying my applications, generating documentation, backups etc..
If you are running cygwin you might prefer the rxvt terminal over the standard cmd based shell, some tips here :
http://freemode.net/archives/000121.html
id still love something like multignometerm for use with cygwin but without running X and all that goes with it.
thanks for all the comments:
Reen: In the beginning, I'll not save any time on this I'm sure. But most of the stuff I write can be reused to automate any kind of automatisations I'll find myself in need for in future. So on the long run I'll save time, and stop feeling like a computer myself when repeating the same kind of steps over and over again by hand ; ).
Christoph: Yeah, I know this kind of staff. Another example is when you manage a project in SVN you have all those .svn files that you need to get rid of before uploading the project somewhere. I'm looking forward to getting this crab automated ; ).
Martin: I'll check it out the console. Thanks for the pointer. Regarding other scripting languages: Yes python is probably good, but I don't feel like learning it just for my automatisation needs. Doing bash scripting by try & error has been pretty easy so far. For more complex things I'd probably go with php or maybe learn some ruby which I wanted to do at some point anyway.
To automate tasks I use Ant scripts as I can execute them directly from my IDE (Eclipse). Another advantage is that they are platform-independent, so everyone in the team can use them. And they are less cryptic than bash scripts ;-)
Hey Daniel,
I'm playing around with Ant right now and I'm pretty amazed. It seems to be a lot easier to work then bash scripting and offers great features! Thanks a lot for that little tip : ).
[...] For everybody who has read my previous post on automatisation, there are good news: [...]
Martin: thanks for the tip with rxvt, that's a lot better than cmd.exe.
It's easy to get rid of the .svn files using `svn export'. Try `svn help export'.
Hi Rajesh Kumar: I know svn export and it's nice, but there are situations when you can't use it. For example if your working copy is different from your SVN Head and you don't want to commit your working copy first. Or if for example things like your database connection files are not under source control at all because they contain your private password. Another reason is if you have no network connectivity to the SVN server in which case svn export doesn't work either. But other then that I usally try to use SVN export as well.
Did you ever finish these automation scripts? Care to publish them? or at least send me a copy :P
I was trying to do exactly what you have described and it would be really hand to have a base to work from :)
- Its cheeky to ask, i know, but nothing ventured nothing gained :P
Drayen: Yeah I did. But actually they became a mess to maintain. I should have done them in PHP right away and not in some dos/cywgin/php hybrid ^^. I don't even know if I still have them around, but if so I wouldn't want you to get hurt by them. However I might do something like this again in better quality and will publish it on here.
If you have 'find' installed, try:
> find /path/to/dir/ -name ".svn" -depth -exec rm -vrf {} \;
This post is too old. We do not allow comments here anymore in order to fight spam. If you have real feedback or questions for the post, please contact us.
I'm using cygwin for several years now, mostly for backup with rsync and development in c++. I don't like bash-scripting, so i've done some small but useful scripts with python. In most cases I think that it is a waste of time to write an hour or more on a script just to reduce the time of a task by 50%.