Unlimited Model fields - Expandable Behavior
Posted on 1/6/08 by Felix Geisendörfer
Hey folks,
lets say you have a model called Upload. Your Upload model has some generic fields like this:
The problem
However, you would like to store different types of meta information for different kinds of uploads. Examples:
Images:
- Width
- Height
- Quality (if jpg)
- Camera
- Lens
- Focus
Videos:
- FPS
- Bitrate
PDFs:
- Author
- Description
ZIPs:
- Original Size
- File Count
- Compression Rate
The solution
So what are you going to do? Add 13 fields to your uploads table? Probably not. It is time to normalize things:
Ok nothing fancy so far. CakePHP's associations make it easy to deal with it. However, working with this setup can be a little inconvenient at times. Everytime you fetch a set of records from Upload, you will have to manually extract the meta information from the associated UploadField records:
- Upload: id: 1 name: funny.mov type: video/quicktime bytes: 20480 created: 2008-06-01 14:47:23 UploadField: - id: 1 upload_id: 1 key: fps val: 26 - id: 2 upload_id: 1 key: bitrate val: 376
So everytime you want to access your videos bitrate you will have to search your UploadField records for the 'bitrate' key. How annoying. But worry not, Expandable comes to rescue. With the Expandable behavior activated on your Upload model, your resultset will look like this:
- Upload: id: 1 name: funny.mov type: video/quicktime bytes: 20480 created: 2008-06-01 14:47:23 fps: 26 bitrate: 376 UploadField: - id: 1 upload_id: 1 key: fps val: 26 - id: 2 upload_id: 1 key: bitrate val: 376
But it comes even better. Expandable also makes it dead-simple to create / update UploadField records. This is how it works:
Without you having to do anything, the following happens to your uploads resultset:
- Upload: id: 1 name: funny.mov type: video/quicktime bytes: 20480 created: 2008-06-01 14:47:23 fps: 30 bitrate: 376 rating: 0.7 UploadField: - id: 1 upload_id: 1 key: fps val: 30 - id: 2 upload_id: 1 key: bitrate val: 376 - id: 3 upload_id: 1 key: rating val: 0.7
As you can see the fps UploadField value has been updated and a new record with the key rating has been created. So this means you can use the CakePHP form helper to create different editors for your uploads like this:
$form->input('Upload.bitrate')
$form->input('Upload.rating', array('options' => array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)));
And even so none of those fields really exist on the Upload model, everything will work just as if they would ; ).
How to use it
- Download the behavior from: the debuggable Scraps repository at github
- Place the expandable.php file into /app/models/behaviors/expandable.php
- Optional: Place the expandable.test.php file into /app/tests/cases/behaviors/expandable.test.php
- Create a table and model for UploadField with at least the fields shown above. (replace Upload with the name of your base model)
- Setup a Upload hasMany UploadField and UploadField belongsTo Upload association
- Add this to your Upload model:
That is it. You are ready to go. Enjoy the magic ; ).
Pro | Contra |
---|---|
|
|
Please let me know what you think about this approach!
-- Felix Geisendörfer aka the_undefined
You can skip to the end and add a comment.
I like the concept, Felix, but the implementation appears to leave you a little exposed. The standard model validation would deal with many of the value issues, but only if the field is specified (you'd have to set allowEmpty for all expandable fields). So the conditional mandatory (i.e. if it is file type x, then fields a,b and c are required) validation would get a little interesting.
Of greater concern though is the ability to 'inject' a whole load of expandable fields direct from a form - i.e. if the fields are not specified in the base model, then they are automagically added to the expandable model - not sure this is a good thing. Whitelisting might help, but perhaps a mandatory 'expandableFields' array in the behavior setup would be better?
An alternative (for fields that will never be required in a where clause) is to serialize the expandable fields into a single text field - also eminently doable with a bit of cake magic.
Hm, why don't you create a model for each type (PDF, Image, etc), each with a hasOne association to Upload (would probably need a different name in this new context)?
GreyCells: Set up all your different rules for different upload types in an array and then apply the one you are interested in to Model::validates in the beforeValidate callback based on the data[Upload][type] key. I will add a key to limit fields at some point, or do you have a patch for me ; )?
Daniel: Simplicity and Flexibility. This behavior helped to simplify a 14 model app to a ~7 model one which had a very positive impact on the simplicity of things ; ).
Hey Felix,
think there was too much copy & paste with the test. ;) the class should be named ExpandableBehaviorTest instead of ContainableBehaviorTest, shouldn't it?
Thanks anyway!
leo: There is no such thing as too much copy & paste ; p. Anyway, thanks for catching this - fixed.
@Felix - yes, that would be the approach I would expect to use - despite all the advances in the model validation in 1.2, I still seem to have an awful lot of stuff in Model::before/validates()... (that's where I put a lot of my business validation) :)
Untested (shock, horror!), but this is what I had in mind:
--- expandable.php.felix 2008-06-01 19:20:58.000000000 +0100
+++ expandable.php 2008-06-01 19:18:57.000000000 +0100
@@ -75,21 +75,25 @@
$fields = array_diff_key($Model->data[$Model->alias], $schema);
$id = $Model->id;
foreach ($fields as $key => $val) {
- $field = $Model->{$with}->find('first', array(
- 'fields' => array($with.'.id'),
- 'conditions' => array($with.'.'.$foreignKey => $id, $with.'.key' => $key),
- 'recursive' => -1,
- ));
- $Model->{$with}->create(false);
- if ($field) {
- $Model->{$with}->set('id', $field[$with]['id']);
- } else {
- $Model->{$with}->set(array($foreignKey => $id, 'key' => $key));
- }
- $Model->{$with}->set('val', $val);
- $Model->{$with}->save();
+ if ( !isset($expandableFields)
+ || in_array($key, $expandableFields)
+ ) {
+ $field = $Model->{$with}->find('first', array(
+ 'fields' => array($with.'.id'),
+ 'conditions' => array($with.'.'.$foreignKey => $id, $with.'.key' => $key),
+ 'recursive' => -1,
+ ));
+ $Model->{$with}->create(false);
+ if ($field) {
+ $Model->{$with}->set('id', $field[$with]['id']);
+ } else {
+ $Model->{$with}->set(array($foreignKey => $id, 'key' => $key));
+ }
+ $Model->{$with}->set('val', $val);
+ $Model->{$with}->save();
+ }
}
}
}
i.e. if there is an array called 'expandableFields' (or an empty array?) in the behavior settings, then the field name must exist in that array, otherwise continue as before.
I've used this approach before (just not with CakePHP) and in retrospect I think that, unless it's absolutely necessary, it's usually VBI ("Very Bad Idea"). This approach works well with some cases, but it's definitely one of those approaches that one must use caution in implementation.
There's a great thread on this on the NYPHP mailing list (I asked a question about it) if anyone is interested in reading other opinions. The thread can be read here:
http://www.mail-archive.com/talk@lists.nyphp.org/msg02145.html
It's like an EAV data model but with two tables.
http://en.wikipedia.org/wiki/Entity-Attribute-Value_model
Felix: does it delete the meta records automatically when you delete the entity?
Thanks
Brian: As I said it works excellent for the scenario I'm describing in this post as well as for storing meta information for different types of Pages in a CMS I've worked on. But you are right, using this in the wrong context will not do you much good ; ).
Ramon: If you use foreign keys and have them setup that way, yes. If you mark the association as 'dependent' in CakePHP, yes. Otherwise, no. : )
Btw. does anybody like the pretty pictures and stuff?
One simple thing: consider using a join table ("UploadFieldKeys") between the upload table and the UploadField table. The join table would consist of an int primary key and a name. The UploadField table would be modified so that the key field is an integer and references the UploadFieldKeys-table to avoid key duplication.
@Felix: Sure, your solution requires less models and is quite flexible. On the other hand: do you really need this flexibility? And is it worth to move "data logic" (e.g. that a PDF has author and description properties) from the database layer to the application layer?
And regarding your question about the images, I like them (with the exception of the first one, which I don't understand)
Felix, I think I don't have to repeat that I love you for that expandable behavior ;)
@Others: I agree that in many cases it might be against the principles of MVC or other noble concepts. But in some cases it's just soooooo damn f***in useful!
Take an example: You have a CMS that you sell to your clients which you always custom-fit to the needs of each client. Some will want to add videos to their websites, others PDFs, again others Word, Excel, whatever... Some months later a client wants to be able to handle additional filetypes. You don't even have to touch the DB, just add a new controller method and you're done. Isn't that convenient for you as a programmer?
hi felix,
and thanks for this. I was using an 'extra data' approach to deal with special cases for categories when I found your Behavior. It helps a lot but ...
Seems like the behavior is not triggered when the model that has the 'extra data' is not populated with this extra data when it is part of an association.
Setting 'recursive' to 2 or more won't change anything.
Is it expectad behavior ?
Thanks in advance
thomas
This will definately come in handy in the next project! Had something alike in my head, but didn't yet have the time to think it out.
Thanks for saving my time! :D
Yeah the pictures are great , What is the program that you used to create them??
I saw expandable behavior two months ago, but I didn't pay attention to it ... or maybe I didn't understand it because there was no enough documentation !!! but with this post things changed it's pretty easy & effective .... thank you Felix ...
Khaled: OmniGraffle 5
Where did you see the behavior two month ago? I don't recall publishing it anywhere up until now ; p.
maybe in the bakery ... or it's something similar, but I remember that the title looks like yours "~Expandable Behavior" ... or maybe I'm wrong ... you know how many things you get when you're surfing ;-)
BTW, nice application "OmniGraffle 5" , but I don't have MAC :( ... I know some similar programs but not as power as this app ...
Thank Felix
I have a very similar approach to this. But My "UploadField" Table has 2 more fields: serialize and unserialize in which i store the name of a php-function executed as you would expect, to save even array-data and/or complex structures in the blob-field. This way, it is even more expandable.
I also created the modelclass in runtime (because they are pretty predictable in structure), so i do not need to create them manually.
Great Work, though. I really like your implementation.
Hey Felix,
thank you for that behaviour!! It's exactly what I need for my project, and what caused so much headaches on how maintaining such variable models...
Regards,
Alexander
Hi Felix!...
There is a bug inside your code.
More precisely, you should modify the line $conventions = array('foreignKey', $Model->hasMany[$settings['with']]['foreignKey']); from setup() function into $conventions = array('foreignKey' => $Model->hasMany[$settings['with']]['foreignKey']);
That way it will be no problem using this behavior when you're using the "with" variabile inside your model's Expandable behavior, like var $actsAs = array ('Expandable' => array ('with' => 'Chichi'));
Hi,
I would like to try the Expandable behavior, but the it doesn't seem to be there anymore. Is there another link?
Thanks
Brad: Sorry about that : ). The behavior can now be found here: http://github.com/felixge/debuggable-scraps/tree/master/cakephp/behaviors/expandable
I also updated the link in the post. Thanks for reporting this issue!
Hi Felix,
Just ran into a link to this post today. I must say your behavior looks nice. I was wondering if there were any plans to support find operations. For me, that is one of the major reasons not to simply serialize things into a "properties" field.
What I mean is something like:
$this->Upload->find('first',array(
'conditions'=>array('Upload.width'=>'400')
));
and this condition would actually be converted to something like:
array('UploadField.key'=>'width', 'UploadField.value'=>'400')
I might make use of the behavior but I would probably have to write a beforeFind to do something like that first. If I go that way I will send you the results but it would be nice to know if something like that is already in the works.
Martin Westin: nothing is in the works. Feel free to fork the github project and hack away, I'll gladly pull in any good changes you make!
The expandable behavior on the Github scraps is not correct. It doesn't match the older version I have.
1) note the explicit use of 'file_id' in the afterSave method
2) it doesn't create new Fields only tries to update pre-existing ones
Anthony: Do you have a patch to fix this? Or can you email me the version you have?
Anthony: Ok, I actually found the previously released version and pushed it to github. Sry for the inconvenience!
This post is too old. We do not allow comments here anymore in order to fight spam. If you have real feedback or questions for the post, please contact us.
I was confronted to similar problem and my first idea was to create more specific table which inherit main table, the all manage be a behavior.
But to complex to build for me!
Your solution is more flexible, so it accommodate me ^^
Useful contribution! Thanks!