How To Transform HTML To Textile Markup - The CakePHP TextileHelper Revisited

Posted by Tim Koschützki, on Aug 23, 2007 - in PHP & CakePHP » Views & Helpers

Hi folks. For a current project of mine I had to find a way to decode html into textile markup. Why? Because we are using tinyMCE to process our textareas as wyciwyg editors, which generate HTML. However, we want all output controlled via textile to allow only the textile tags. Yes, we could do it with strip_tags(), but textile is much more elegant. Plus, it was a requirement by the client. Come on and find out how to detextile html.

How Does It Work?

Well the code is not entirely trivial, but it looks like what you would expect: a bunch of regex processing. Here is the basic detextile() method, which takes some (html) text:

php
  1. function detextile($text) {
  2.    
  3.     $text = preg_replace("/<br \/>\s*/","\n",$text);
  4.  
  5.     $oktags = array('p','ol','ul','li','i','b','em','strong','span','a','h[1-6]',
  6.       'table','tr','td','u','del','sup','sub','blockquote');
  7.  
  8.     foreach($oktags as $tag){
  9.       $text = preg_replace_callback("/\t*< (".$tag.")\s*([^>]*)>(.*)< \/\\1>/Usi",
  10.       array($this,'processTag'),$text);
  11.     }
  12.  
  13.     $text = $this->detextile_process_glyphs($text);
  14.     $text = $this->detextile_process_lists($text);
  15.        
  16.         $text = preg_replace('/^\t* *p\. /m','',$text);
  17.        
  18.         return $this->decode_high($text);
  19.     }

Okay, so we processTag() all html tags that we want to cover, process glyphs (we will get to that in a minute) and lists, eliminate all tabs and paragraphs and return the text decoded, with UTF8 as the standard charset making use of the mb_decode_numericentity() function. So what does processTag do?

php
  1. function processTag($matches) {
  2.         list($all,$tag,$atts,$content) = $matches;
  3.         $a = $this->splat($atts);
  4.  
  5.         $phr = array(
  6.         'em'=>'_',
  7.         'i'=>'__',
  8.         'b'=>'**',
  9.         'strong'=>'*',
  10.         'cite'=>'??',
  11.         'del'=>'-',
  12.         'ins'=>'+',
  13.         'sup'=>'^',
  14.         'sub'=>'~',
  15.         'span'=>'%');
  16.        
  17.         $blk = array('p','h1','h2','h3','h4','h5','h6');
  18.  
  19.         if(isset($phr[$tag])) {
  20.             return $phr[$tag].$this->sci($a).$content.$phr[$tag];
  21.         } elseif($tag=='blockquote') {
  22.             return 'bq.'.$this->sci($a).' '.$content;
  23.         } elseif(in_array($tag,$blk)) {
  24.             return $tag.$this->sci($a).'. '.$content;
  25.         } elseif ($tag=='a') {
  26.             $t = $this->filterAtts($a,array('href','title'));
  27.             $out = '"'.$content;
  28.             $out.= (isset($t['title'])) ? ' ('.$t['title'].')' : '';
  29.             $out.= '":'.$t['href'];
  30.             return $out;
  31.         } else {
  32.             return $all;
  33.         }
  34.     }
  35.  
  36.     function sci($a)
  37.     {
  38.         $out = '';
  39.         foreach($a as $t){
  40.             $out.= ($t['name']=='class') ? '(='.$t['att'].')' : '';
  41.             $out.= ($t['name']=='id') ? '[='.$t['att'].']' : '';
  42.             $out.= ($t['name']=='style') ? '{='.$t['att'].'}' : '';
  43.             $out.= ($t['name']=='cite') ? ':'.$t['att'] : '';
  44.         }
  45.         return $out;
  46.     }

Here is where much of the converting takes place. We have a map of conversion strings from html tags to textile entities and convert them here. We preserve any classes, ids and attributes using the splat method and the sci method and return the text. The splat method is quite sophisticated and long to explain, but it should become clear when you look at it below.

Now on to the glyphs and the list methods:

php
  1. function detextile_process_glyphs($text) {
  2.     $glyphs = array(  
  3.       '&#8217;'=>'\'',        # single closing
  4.       '&#8216;'=>'\'',        # single opening
  5.       '&#8221;'=>'"',         # double closing
  6.       '&#8220;'=>'"',         # double opening
  7.       '&#8212;'=>'--',        # em dash
  8.       '&#8211;'=>' - ',       # en dash
  9.       '&#215;' =>'x',         # dimension sign
  10.       '&#8482;'=>'(TM)',      # trademark
  11.       '&#174;' =>'(R)',       # registered
  12.       '&#169;' =>'(C)',       # copyright
  13.       '&#8230;'=>'...'        # ellipsis
  14.     );
  15.  
  16.     foreach($glyphs as $f=>$r){
  17.       $text = str_replace($f,$r,$text);
  18.     }
  19.     return $text;
  20.   }

Easy. It simply converts some html entities for glyphs into their textile equivalents.

The list method:

php
  1. function detextile_process_lists($text) {
  2.     $list = false;
  3.  
  4.     $text = preg_split("/(< .*>)/U",$text,-1,PREG_SPLIT_DELIM_CAPTURE);
  5.     foreach($text as $line){
  6.  
  7.       if ($list == false && preg_match('/<ol /',$line)){
  8.         $line = "";
  9.         $list = "o";
  10.       } else if (preg_match('/<\/ol/',$line)){
  11.         $line = "";
  12.         $list = false;
  13.       } else if ($list == false && preg_match('/<ul/',$line)){
  14.         $line = "";
  15.         $list = "u";
  16.       } else if (preg_match('/<\/ul/',$line)){
  17.         $line = "";
  18.         $list = false;
  19.       } else if ($list == 'o'){
  20.         $line = preg_replace('/<li.*>/U','# ', $line);
  21.       } else if ($list == 'u'){
  22.         $line = preg_replace('/<li .*>/U','* ', $line);
  23.       }
  24.       $glyph_out[] = $line;
  25.     }
  26.  
  27.     return $text = implode('',$glyph_out);
  28.   }

This method is a bit more tricky. It wipes out any list starting tags (ul, ol) and converts all li-tags into their textile equivalents - either "# " (for ordered lists) or " *" (for unordered lists).

How Do You Use The Code?

Using the code is darn easy. You just invoke the detextile method upon your html code:

php
  1.      echo $textile->detextile($htmlText);

An Example

Here is some example html code we want to convert:

html
  1. <p>
  2.                 <strong>This is some bold text</strong>
  3.               </p>
  4.               <p>
  5.                  <em>This is italic text</em>
  6.               </p>
  7.               <p>
  8.                  
  9.  
  10.               </p>
  11.               <p>
  12.                 <u>Underline text man</u>
  13.               </p>
  14.               <p>
  15.                  
  16.               </p>
  17.               <ul>
  18.  
  19.                 <li>ul list item1
  20.                 </li>
  21.                 <li>ul list item2
  22.                 </li>
  23.                 <li>ul list item3
  24.                 </li>
  25.               </ul>
  26.               <ol>
  27.                 <li>ol list item1
  28.                 </li>
  29.                 <li>ol list item2
  30.                 </li>
  31.  
  32.                 <li>ol list item3
  33.                 </li>
  34.               </ol>

Detextile output:

html
  1. *This is some bold text*p.  _This is italic text_p.  p. <u>Underline text man</u>p.  * ul list item1 * ul list item2* ul list item3# ol list item1# ol list item2# ol list
  2.               item3

Cool! And it was easy as well!

Get The Code

Here are all methods for your cakephp textile helper. You can plug them in into any Textile Helper for other frameworks of course:

php
  1. // -------------------------------------------------------------
  2. // The following functions are used to detextile html, a process
  3. // still in development.
  4. // By Tim Koschützki
  5.  
  6. // Based on code from http://www.aquarionics.com
  7.  
  8. // -------------------------------------------------------------
  9.     function detextile($text) {
  10.    
  11.     $text = preg_replace("/<br \/>\s*/","\n",$text);
  12.  
  13.     $oktags = array('p','ol','ul','li','i','b','em','strong','span','a','h[1-6]',
  14.       'table','tr','td','u','del','sup','sub','blockquote');
  15.  
  16.     foreach($oktags as $tag){
  17.       $text = preg_replace_callback("/\t*< (".$tag.")\s*([^>]*)>(.*)< \/\\1>/Usi",
  18.       array($this,'processTag'),$text);
  19.     }
  20.  
  21.     $text = $this->detextile_process_glyphs($text);
  22.     $text = $this->detextile_process_lists($text);
  23.        
  24.         $text = preg_replace('/^\t* *p\. /m','',$text);
  25.        
  26.         return $this->decode_high($text);
  27.     }
  28.  
  29.   function detextile_process_glyphs($text) {
  30.     $glyphs = array(  
  31.       '&#8217;'=>'\'',        # single closing
  32.       '&#8216;'=>'\'',        # single opening
  33.       '&#8221;'=>'"',         # double closing
  34.       '&#8220;'=>'"',         # double opening
  35.       '&#8212;'=>'--',        # em dash
  36.       '&#8211;'=>' - ',       # en dash
  37.       '&#215;' =>'x',         # dimension sign
  38.       '&#8482;'=>'(TM)',      # trademark
  39.       '&#174;' =>'(R)',       # registered
  40.       '&#169;' =>'(C)',       # copyright
  41.       '&#8230;'=>'...'        # ellipsis
  42.     );
  43.  
  44.     foreach($glyphs as $f=>$r){
  45.       $text = str_replace($f,$r,$text);
  46.     }
  47.     return $text;
  48.   }
  49.  
  50.   function detextile_process_lists($text) {
  51.     $list = false;
  52.  
  53.     $text = preg_split("/(< .*>)/U",$text,-1,PREG_SPLIT_DELIM_CAPTURE);
  54.     foreach($text as $line){
  55.  
  56.       if ($list == false && preg_match('/<ol /',$line)){
  57.         $line = "";
  58.         $list = "o";
  59.       } else if (preg_match('/<\/ol/',$line)){
  60.         $line = "";
  61.         $list = false;
  62.       } else if ($list == false && preg_match('/<ul/',$line)){
  63.         $line = "";
  64.         $list = "u";
  65.       } else if (preg_match('/<\/ul/',$line)){
  66.         $line = "";
  67.         $list = false;
  68.       } else if ($list == 'o'){
  69.         $line = preg_replace('/<li.*>/U','# ', $line);
  70.       } else if ($list == 'u'){
  71.         $line = preg_replace('/<li .*>/U','* ', $line);
  72.       }
  73.       $glyph_out[] = $line;
  74.     }
  75.  
  76.     return $text = implode('',$glyph_out);
  77.   }
  78.  
  79.   function processTag($matches) {
  80.         list($all,$tag,$atts,$content) = $matches;
  81.         $a = $this->splat($atts);
  82.  
  83.         $phr = array(
  84.         'em'=>'_',
  85.         'i'=>'__',
  86.         'b'=>'**',
  87.         'strong'=>'*',
  88.         'cite'=>'??',
  89.         'del'=>'-',
  90.         'ins'=>'+',
  91.         'sup'=>'^',
  92.         'sub'=>'~',
  93.         'span'=>'%');
  94.        
  95.         $blk = array('p','h1','h2','h3','h4','h5','h6');
  96.  
  97.         if(isset($phr[$tag])) {
  98.             return $phr[$tag].$this->sci($a).$content.$phr[$tag];
  99.         } elseif($tag=='blockquote') {
  100.             return 'bq.'.$this->sci($a).' '.$content;
  101.         } elseif(in_array($tag,$blk)) {
  102.             return $tag.$this->sci($a).'. '.$content;
  103.         } elseif ($tag=='a') {
  104.             $t = $this->filterAtts($a,array('href','title'));
  105.             $out = '"'.$content;
  106.             $out.= (isset($t['title'])) ? ' ('.$t['title'].')' : '';
  107.             $out.= '":'.$t['href'];
  108.             return $out;
  109.         } else {
  110.             return $all;
  111.         }
  112.     }
  113.  
  114. // -------------------------------------------------------------
  115.     function filterAtts($atts,$ok)
  116.     {
  117.         foreach($atts as $a) {
  118.             if(in_array($a['name'],$ok)) {
  119.                 if($a['att']!='') {
  120.                 $out[$a['name']] = $a['att'];
  121.                 }
  122.             }
  123.         }
  124. #        dump($out);
  125.         return $out;
  126.     }
  127.  
  128. // -------------------------------------------------------------
  129.     function sci($a)
  130.     {
  131.         $out = '';
  132.         foreach($a as $t){
  133.             $out.= ($t['name']=='class') ? '(='.$t['att'].')' : '';
  134.             $out.= ($t['name']=='id') ? '[='.$t['att'].']' : '';
  135.             $out.= ($t['name']=='style') ? '{='.$t['att'].'}' : '';
  136.             $out.= ($t['name']=='cite') ? ':'.$t['att'] : '';
  137.         }
  138.         return $out;
  139.     }
  140.  
  141. // -------------------------------------------------------------
  142.     function splat($attr)  // returns attributes as an array
  143.     {
  144.         $arr = array();
  145.         $atnm = '';
  146.         $mode = 0;
  147.  
  148.         while (strlen($attr) != 0){
  149.             $ok = 0;
  150.             switch ($mode) {
  151.                 case 0: // name
  152.                     if (preg_match('/^([a-z]+)/i', $attr, $match)) {
  153.                         $atnm = $match[1]; $ok = $mode = 1;
  154.                         $attr = preg_replace('/^[a-z]+/i', '', $attr);
  155.                     }
  156.                 break;
  157.    
  158.                 case 1: // =
  159.                     if (preg_match('/^\s*=\s*/', $attr)) {
  160.                         $ok = 1; $mode = 2;
  161.                         $attr = preg_replace('/^\s*=\s*/', '', $attr);
  162.                     break;
  163.                     }
  164.                     if (preg_match('/^\s+/', $attr)) {
  165.                         $ok = 1; $mode = 0;
  166.                         $arr[] = array('name'=>$atnm,'whole'=>$atnm,'att'=>$atnm);
  167.                         $attr = preg_replace('/^\s+/', '', $attr);
  168.                     }
  169.                 break;
  170.    
  171.                 case 2: // value
  172.                     if (preg_match('/^("[^"]*")(\s+|$)/', $attr, $match)) {
  173.                         $arr[]=array('name' =>$atnm,'whole'=>$atnm.'='.$match[1],
  174.                                 'att'=>str_replace('"','',$match[1]));
  175.                         $ok = 1; $mode = 0;
  176.                         $attr = preg_replace('/^"[^"]*"(\s+|$)/', '', $attr);
  177.                     break;
  178.                     }
  179.                     if (preg_match("/^('[^']*')(\s+|$)/", $attr, $match)) {
  180.                         $arr[]=array('name' =>$atnm,'whole'=>$atnm.'='.$match[1],
  181.                                 'att'=>str_replace("'",'',$match[1]));
  182.                         $ok = 1; $mode = 0;
  183.                         $attr = preg_replace("/^'[^']*'(\s+|$)/", '', $attr);
  184.                     break;
  185.                     }
  186.                     if (preg_match("/^(\w+)(\s+|$)/", $attr, $match)) {
  187.                         $arr[]=
  188.                             array('name'=>$atnm,'whole'=>$atnm.'="'.$match[1].'"',
  189.                                 'att'=>$match[1]);
  190.                         $ok = 1; $mode = 0;
  191.                         $attr = preg_replace("/^\w+(\s+|$)/", '', $attr);
  192.                     }
  193.                 break;
  194.             }
  195.             if ($ok == 0){
  196.                 $attr = preg_replace('/^\S*\s*/', '', $attr);
  197.                 $mode = 0;
  198.             }
  199.         }
  200.         if ($mode == 1) $arr[] =
  201.                 array ('name'=>$atnm,'whole'=>$atnm.'="'.$atnm.'"','att'=>$atnm);
  202.      
  203.         return $arr;
  204.     }

The code is based on an unfinished start from http://www.aquarionics.com. Thanks to the guys over there!

Have fun!