Copyright Notice

This text is copyright by InfoStrada Communications, Inc., and is used with their permission. Further distribution or use is not permitted.

This text has appeared in an edited form in Linux Magazine magazine. However, the version you are reading here is as the author originally submitted the article for publication, not after their editors applied their creativity.

Please read all the information in the table of contents before using this article.
Download this listing!

Linux Magazine Column 79 (Mar 2006)

[Suggested title: ``Building a static website with Template Toolkit'']

As described in previous articles, I'm a big fan of Template Toolkit (or TT for short). I've frequently used TT to generate dynamic web pages based on input parameters, but TT can also help static websites as well. Let's take a look at a typical small static website and how TT can help things.

I'm a karaoke singer. I admit it. Karaoke helped get me through my initial stage fright when I started teaching: moving away from behind the keyboard and in front of a room wasn't what I had in mind when I studied software design in my youth. Karaoke also got me through the initial potential homesickness when I started travelling to teach: no matter what city I ended up in, I could find at least one night at a Karaoke Bar during the week, and things would seem ``familiar''. And yes, there are two reasons you clap at the end of every singer's performance: you're either happy they sang, or you're happy they're done!

Many of the local Karaoke Bars in my home area are serviced by a company (we'll call it, The Music Guy) that has the beginnings of a good website, but fails to keep the site up to date or even consistent. I can understand that: it's hard to make sure that the list of places a person working for them will appear in sync with the list of people that appear at each place. And then there's the ``look'' on each page: having the same navigation and fonts and colors can get to be troublesome. CSS fixes some of that, but you still have to have consistent HTML on which to hang the CSS.

This is a perfect place for a static web site built by a templating system. And TT2 even includes a nice tool (ttree) to take a file hierarchy and run some of the files as templates while simply copying or ignoring others.

So, I sketched up a sample website for The Music Guy using ttree and some dummy text. Let's look at the result.

To control ttree for a project of any size, we need a ttreerc file. The simplest way to use ttree for a project is to set the TTREERC environment variable to point at a file for the project. Yes, this is inconvenient: apparently Andy Wardley thinks you're only ever working on one thing at a time. My sample ttreerc file is given in [listing 1, below].

Line 1 enables recursive processing, which makes sense because we're doing a hierarchy. Line 2 enables a verbose trace of the processing. Lines 3 and 4 define source and destination directories, relative to our current directory. Files in src will be copied to the corresponding location in dest. I'll then publish the files that end up in dest to the website.

Line 5 defines a lib, which gets added to the INCLUDE_PATH for TT. This directory contains the files for INCLUDE and WRAPPER and PROCESS directives.

Line 6 defines filenames that will be copied as-is from the src to the dest hierarchy. We don't want TT to expand our images or CSS files! Similarly, line 7 defines files that will be ignored: in this case, files that end in tilde, such as the editing backup files made by my Emacs editing sessions. Any file that doesn't match either the copy or ignore patterns will be processed through TT.

Line 8 enables post-chomp mode, which I find convenient as it seems to eat about the right number of newlines by default.

Lines 10 through 12 control how each file processed through TT gets interpreted. Both data and templates will be pre-processed, meaning that their contents will be evaluated before each file. And process will be processed in place of the original file, passing the template variable into the template so that the template can completely control additional headers and footers to be added to the file. All three of these files are located in the lib directory. (This configuration uses the PROCESS configuration setting, so see the TT documentation for further explanation and examples of that.)

Let's look at lib/data, as shown in [listing 2, below]. This data file defines the TT variables that hold the venues, the DJs, and the shows. The venues TT variable is a hash of hashes. The keys of venues are the ``short id'' for each location. Rather than referring to The Hot Seat Bar and Grill everywhere else, we simply refer to hot. However, that generally won't make sense to the people reading the site, so the value at the key of name in the referenced hash gives the human-readable equivalent. Similarly, the djs variable contains the relevant information for the DJs.

The big data element is the shows mapping, which is an array of arrays: one array for each day of the week. Within that day-of-week array, the first element is a string with the day name, and the remaining elements are hashes giving the venue and the DJ (if known), using their short identifiers. We can also define notes if a particular show has unusual characteristics (starts late, has a contest, and so on). For routine maintenance, editing data should suffice.

Next, we have lib/templates [listing 3], defining blocks that are used for INCLUDE and WRAPPER directives in our pages. The djlink creates an anchor referencing a particular DJ, used like:

  [% INCLUDE djlink dj = "shannon" prefix = "../dj/" %]

which would refer to Shannon, showing her human-readable name (not just the internal tag), and adding ../dj/ in front of shannon.html. Similarly, venuelink lets us say:

  [% INCLUDE venuelink venue = "hot" prefix = "" %]

to refer to the Hot Seat in the same web directory.

Both djwrapper and venuewrapper are used in the individual DJ and venue pages to add common content. For example, as shown in [listing 7], src/dj/jd.html invokes djwrapper, passing the DJ id of jd, and including some text content specifically about ``J.D.''. This content is passed as content into djwrapper. Line 17 of listing 3 fetches the human readable name (here, J.D.), which is used in the h1 for the page in line 19. Line 20 inserts the wrapped content.

Since every DJ page will have the same bottom content (their schedule), we generate that next, in lines 21 to 40. For every day in the list of shows, we grab the dayname (line 26), and look at the hashes defining the shows on that day (line 27). The DJ for the show is compared to the current DJ, and if it's not the same, we skip the entry (line 29). Otherwise, we link to the venue (line 31), and add show notes if any (lines 33 to 35).

The output of this template for J.D. results in a file of dest/dj/jd.html that contains:

    <html><head><title>The Music Guy Karaoke</title>
    </head><body>
    <h1>J.D.</h1>
    <h2>Where is J.D.?</h2>
    J.D. can be seen at:
    <ul>
    <li> <a href="../venue/hot.html">The Hot Seat Bar and Grill</a>
         on Wednesday
    <li> <a href="../venue/wong.html">Wong's Chinese Restaurant</a>
         on Friday (Big cash prizes!)
    <li> <a href="../venue/hot.html">The Hot Seat Bar and Grill</a>
         on Saturday
    </ul>
    </body></html>

Some of the data comes from the master wrapper, which we'll see in a minute. Note that we have a consistent format for each of the menu items. In fact, if we didn't like the way they looked, we could edit the templates file and ttree would re-generate all files. Any file listed as a preprocess or process (as well as wrapper and postprocess) are automatically listed as dependencies, such that any changes will cause all files to be rebuilt. You can also explictly mark dependencies in the ttree file as well, even creating a Makefile-like syntax to describe the relationships.

To this wrapper we might also add other common items, such as an image. For example, ahead of content, we could add:

  <img src="[% dj | uri | html %]_headshot.jpg">

and then add headshots like jd_headshot.jpg, which would be automatically included.

The venuewrapper in lines 43 to 70 works in a similar way, providing a common look for every venue. The only tricky part is that a show might or might not have a DJ assigned, so lines 58 to 61 add that value in conditionally. For the Hot Seat, the resulting file looks like:

    <html><head><title>The Music Guy Karaoke</title>
    </head><body>
    <h1>The Hot Seat Bar and Grill</h1>
    <h2>What's happening at The Hot Seat Bar and Grill?</h2>
    The Hot Seat Bar and Grill is <i>the</i> place to be on:
    <ul>
    <li> Tuesday, hosted by <a href="../dj/adam.html">Adam</a>
    <li> Wednesday, hosted by <a href="../dj/jd.html">J.D.</a>
    <li> Thursday, hosted by <a href="../dj/shannon.html">Shannon</a>
    <li> Friday, hosted by <a href="../dj/adam.html">Adam</a>
    <li> Saturday, hosted by <a href="../dj/jd.html">J.D.</a>
    </ul>
    </body></html>

Now, on to [Listing 4], the lib/process file. This template gets control after all of the preprocess files have been executed, and instead of the actual template being processed. The value of template within the template refers to the now-deferred template. The META variables within the deferred template are available to this process template. For example, if you had:

  [% META type = "venue" %]

in the template, then template.type would be venue, allowing the process template to execute alternative code. For example, we've defined a default HTML title of The Music Guy Karaoke, but any specific page can alter that with a META directive.

To execute the actual template, we call

  [% PROCESS $template %]

within this process template. In this case, we're capturing the output into content, and reinserting that in the appropriate place in line 4. The reason for this may not be obvious in this trivial case, but typically the PROCESS will go inside a TRY block, making sure that errors trigger alternate behavior. We can also set global variables in global or template to alter how the wrapper for the page is rendered as well, such as controlling the sidebar navigation (not shown here for simplicity).

Speaking of navigation, let's look next at the top-level index.html, shown in [listing 5]. We're not using an individual page wrapper, but this still appears within the overall content from lib/process. The main data on the front page needs to be a dump of all shows, organized by day and venue, with links for more information. To do this, we loop over shows (yet again!) in lines 5 through 25. Each interation puts one array in day, whose dayname is extracted at the end of line 5 and shown in line 6.

Lines 8 through 23 put one show hash at a time into show. This time, we're creating a bullet list, so we place the bullet-list start on the first loop iteration in line 10. Line 12 adds the link to the venue. Lines 13 to 17 add a link to the DJ if known. And lines 18 to 20 add the show notes if any. Line 22 closes off the bullet list on the last item, and we're done. For the section on Thursday, the result looks like:

    <h3>Thursday</h3>
    <ul>
    <li> <a href="venue/hot.html">The Hot Seat Bar and Grill</a>,
         hosted by <a href="dj/shannon.html">Shannon</a>
    <li> <a href="venue/cheerful.html">The Cheerful Sports Page</a>
    <li> <a href="venue/kenzie.html">McKenzie Pub</a>
    <li> <a href="venue/jo.html">Jo's Saloon</a>,
         hosted by <a href="dj/rc.html">RC</a>
    </ul>

You can see that the venues are linked by their real name, and the DJs if known have been added and linked as well.

Just two more indexes to go. The list of DJs comes from [listing 6], for src/dj/index.html, and walks through the hash of DJs adding a link for each one. Because the DJ hash is unsorted, we first sort by the name subelement, causing them to be sorted by their first names. If we wanted a special sort order, we could have sorted by a field used only for sorting, adding a sort_by hash element, for example.

Similarly, [Listing 8] shows src/venue/index.html, the index for the venues. Similar strategy here, so I won't bore you with the details.

And there we have it. A handful of files, creating a fully templated static website. Any time a show gets cancelled or added, I edit data. When a new DJ gets hired, I add an entry in djs, and create a simple web page with any unique information (and a headshot, if we've added those graphics). The boilerplate comes for free. For a new venue, it's an entry in venues, and a new page under the src/venue/ directory.

If I decide that I want to show how drive to a venue, I could create a template in src/templates that I call from each venue, or I could put the addresses in the venues hash and generate it directly from the venuewrapper template. With templates, the commonality is captured, and the data defines the differences.

I hope this little exposure to ttree inspired you to template-ize some of your repetitive tasks. Until next time, enjoy!

Listings

        =0=     #################### 1: ttreerc ####################
        =1=     recurse
        =2=     verbose
        =3=     src = src
        =4=     dest = dest
        =5=     lib = lib
        =6=     copy = \.(gif|jpg|png|css)$
        =7=     ignore = ~$
        =8=     post_chomp 1
        =9=     
        =10=    preprocess = data
        =11=    preprocess = templates
        =12=    process = process
        =0=     #################### 2: lib/data ####################
        =1=     [%
        =2=     venues = {
        =3=       cheerful = { name = "The Cheerful Sports Page" }
        =4=       harmony = { name = "The Harmony Inn" }
        =5=       hot = { name = "The Hot Seat Bar and Grill" }
        =6=       jimmy = { name = "Jimmy's Sports Bar" }
        =7=       jo = { name = "Jo's Saloon" }
        =8=       johns = { name = "Dr. Johns Pub" }
        =9=       kenzie = { name = "McKenzie Pub" }
        =10=      lair = { name = "Pete's Lair" }
        =11=      sneakers = { name = "Sneakers Pub" }
        =12=      wong = { name = "Wong's Chinese Restaurant" }
        =13=    };
        =14=    djs = {
        =15=      adam = { name = "Adam" }
        =16=      brian = { name = "Brian" }
        =17=      chad = { name = "Chad" }
        =18=      jake = { name = "Jake" }
        =19=      jd = { name = "J.D." }
        =20=      melissa = { name = "Melissa" }
        =21=      rc = { name = "RC" }
        =22=      sean = { name = "Sean" }
        =23=      shannon = { name = "Shannon" }
        =24=      starr = { name = "Starr" }
        =25=    };
        =26=    shows = [
        =27=      ["Monday"
        =28=        { venue = "sneakers" notes = "Starts right after the game!" }
        =29=      ]
        =30=      ["Tuesday"
        =31=        { venue = "hot" dj = "adam" }
        =32=        { venue = "lair" dj = "brian" }
        =33=        { venue = "cheerful" dj = "starr" }
        =34=        { venue = "johns" dj = "rc" }
        =35=      ]
        =36=      ["Wednesday"
        =37=        { venue = "hot" dj = "jd" }
        =38=        { venue = "jimmy" dj = "brian" }
        =39=        { venue = "cheerful" dj = "shannon" }
        =40=        { venue = "jo" dj = "rc" }
        =41=        { venue = "johns" dj = "sean" }
        =42=      ]
        =43=      ["Thursday"
        =44=        { venue = "hot" dj = "shannon" }
        =45=        { venue = "cheerful" }
        =46=        { venue = "kenzie" }
        =47=        { venue = "jo" dj = "rc" }
        =48=      ]
        =49=      ["Friday"
        =50=        { venue = "hot" dj = "adam" }
        =51=        { venue = "lair" dj = "melissa" }
        =52=        { venue = "johns" dj = "rc" }
        =53=        { venue = "cheerful" dj = "jake" }
        =54=        { venue = "harmony" dj = "starr" }
        =55=        { venue = "jo" dj = "chad" }
        =56=        { venue = "wong" dj = "jd" notes = "Big cash prizes!" }
        =57=      ]
        =58=      ["Saturday"
        =59=        { venue = "hot" dj = "jd" }
        =60=        { venue = "harmony" dj = "shannon" }
        =61=        { venue = "cheerful" }
        =62=        { venue = "jimmy" }
        =63=        { venue = "wong" }
        =64=      ]
        =65=    ];
        =66=    %]
        =0=     #################### 3: lib/templates ####################
        =1=     [% BLOCK djlink;
        =2=       # dj = "dj short id"
        =3=       # prefix = "../dj/" or "dj/" or ""
        =4=     %]
        =5=     <a href="[% prefix; dj | uri | html %].html">[% djs.$dj.name %]</a>
        =6=     [%- END %]
        =7=     [%##################################### %]
        =8=     [% BLOCK venuelink;
        =9=       # venue = "venue short id"
        =10=      # prefix = "../venue/" or "venue/" or ""
        =11=    %]
        =12=    <a href="[% prefix; venue | uri | html %].html">[% venues.$venue.name %]</a>
        =13=    [%- END %]
        =14=    [%##################################### %]
        =15=    [% BLOCK djwrapper;
        =16=      # dj = "dj short id"
        =17=      djname = djs.$dj.name;
        =18=    %]
        =19=    <h1>[% djname | html %]</h1>
        =20=    [% content %]
        =21=    <h2>Where is [% djname %]?</h2>
        =22=    [% djname | html %] can be seen at:
        =23=    <ul>
        =24=    [%
        =25=      FOR day IN shows;
        =26=        dayname = day.0;
        =27=        FOR show IN day.slice(1);
        =28=          venue = show.venue;
        =29=          NEXT UNLESS show.dj == dj; # not here
        =30=          "<li> ";
        =31=          INCLUDE venuelink venue = venue prefix = "../venue/";
        =32=          " on "; dayname;
        =33=          IF show.notes;
        =34=            " ("; show.notes | html; ")";
        =35=          END;
        =36=          "\n";
        =37=        END;
        =38=     END;
        =39=    %]
        =40=    </ul>
        =41=    [% END %]
        =42=    [%##################################### %]
        =43=    [% BLOCK venuewrapper;
        =44=      # venue = "venue short id"
        =45=      venuename = venues.$venue.name;
        =46=    %]
        =47=    <h1>[% venuename | html %]</h1>
        =48=    [% content %]
        =49=    <h2>What's happening at [% venuename | html %]?</h2>
        =50=    [% venuename | html %] is <i>the</i> place to be on:
        =51=    <ul>
        =52=    [%
        =53=      FOR day IN shows;
        =54=        dayname = day.0;
        =55=        FOR show IN day.slice(1);
        =56=          NEXT UNLESS show.venue == venue; # not here
        =57=          "<li> "; dayname;
        =58=          IF show.dj;
        =59=            ", hosted by ";
        =60=            INCLUDE djlink dj = show.dj prefix = "../dj/";
        =61=          END;
        =62=          IF show.notes;
        =63=            " ("; show.notes | html; ")";
        =64=          END;
        =65=          "\n";
        =66=        END;
        =67=     END;
        =68=    %]
        =69=    </ul>
        =70=    [% END %]
        =0=     #################### 4: lib/process ####################
        =1=     [% content = PROCESS $template %]
        =2=     <html><head><title>[% template.title or "The Music Guy Karaoke" %]</title>
        =3=     </head><body>
        =4=     [% content %]
        =5=     </body></html>
        =0=     #################### 5: src/index.html ####################
        =1=     <h1>Welcome to Karaoke!</h1>
        =2=     
        =3=     <h2>Our show line-up</h2>
        =4=     
        =5=     [% FOR day IN shows; dayname = day.0; %]
        =6=     <h3>[% dayname %]</h3>
        =7=     [%
        =8=       FOR show IN day.slice(1);
        =9=         venue = show.venue;
        =10=        "<ul>\n" IF loop.first;
        =11=        "<li> ";
        =12=        INCLUDE venuelink venue = venue prefix = "venue/";
        =13=        IF show.dj;
        =14=          dj = show.dj;
        =15=          ", hosted by ";
        =16=          INCLUDE djlink dj = dj prefix = "dj/";
        =17=        END;
        =18=        IF show.notes;
        =19=          " ("; show.notes | html; ")";
        =20=        END;
        =21=        "\n";
        =22=        "</ul>\n" IF loop.last;
        =23=      END;
        =24=    %]
        =25=    [% END %]
        =0=     #################### 6: src/dj/index.html ####################
        =1=     <h2>Check out our DJs!</h2>
        =2=     
        =3=     [% FOR dj IN djs.keys.sort("name") %]
        =4=     [% "<ul>\n" IF loop.first %]
        =5=     <li> [% INCLUDE djlink dj = dj prefix = "" %]
        =6=     [% "</ul>\n" IF loop.last %]
        =7=     [% END %]
        =0=     #################### 7: src/dj/jd.html ####################
        =1=     [% WRAPPER djwrapper dj = "jd" %]
        =2=     Yes, the man that is "dj" spelled backwards!
        =3=     [% END %]
        =0=     #################### 8: src/venue/index.html ####################
        =1=     <h2>Check out our venues!</h2>
        =2=     
        =3=     [% FOR venue IN venues.keys.sort("name") %]
        =4=     [% "<ul>\n" IF loop.first %]
        =5=     <li> [% INCLUDE venuelink venue = venue prefix = "" %]
        =6=     [% "</ul>\n" IF loop.last %]
        =7=     [% END %]
        =0=     #################### 9: src/venue/hot.html ####################
        =1=     [% WRAPPER venuewrapper venue = "hot" %]
        =2=     Check out our daily drink special!
        =3=     [% END %]

Randal L. Schwartz is a renowned expert on the Perl programming language (the lifeblood of the Internet), having contributed to a dozen top-selling books on the subject, and over 200 magazine articles. Schwartz runs a Perl training and consulting company (Stonehenge Consulting Services, Inc of Portland, Oregon), and is a highly sought-after speaker for his masterful stage combination of technical skill, comedic timing, and crowd rapport. And he's a pretty good Karaoke singer, winning contests regularly.

Schwartz can be reached for comment at merlyn@stonehenge.com or +1 503 777-0095, and welcomes questions on Perl and other related topics.