Copyright Notice

This text is copyright by InfoStrada Communications, Inc., and is used with their permission. Further distribution or use is not permitted.

This text has appeared in an edited form in Linux Magazine magazine. However, the version you are reading here is as the author originally submitted the article for publication, not after their editors applied their creativity.

Please read all the information in the table of contents before using this article.

Linux Magazine Column 60 (Jun 2004)

[suggested title: ``Introduction to Template Toolkit (part 1)'']

In a few past columns, I've mentioned that my template toolkit of choice is the aptly named Template Toolkit, a marvelous work by one Andy Wardley, whom I've had the pleasure to meet on more than one occasion. And although I've demonstrated how I've used Template Toolkit (hereinafter called TT to save typing), I haven't really talked enough about what makes it so wonderfully useful. So, let's take a look at that.

Why Template Toolkit?

What is Template Toolkit? Some might start by saying that TT is ``Yet Another Templating System'', but that would be like saying that the Mona Lisa is ``just another painting''.

At a minimum, a templating system provides a way of replacing placeholders with values. The simplest templating system is a Perl ``one-liner'':

  my %v = (first => 'Randal', last => 'Schwartz');
  my $text = 'My name is <first> <last>.';
  ## here it comes:
  $text =~ s/<(\w+)>/$v{$1}/g;

Each word enclosed in angle brackets is replaced with a value found in the %v hash. Simple enough, and some would say ``deceptively simple''. Because it's this easy to code a trivial templating system, many people have started here and grown their own templating systems independently.

But the tricky parts are indeed tricky. Eventually, templating systems apparently grow to include those features that make them full-blown languages: the capabilities to make decisions and iterations. And different authors end up doing it differently. HTML::Template uses XSLT-like angle-bracket syntax. HTML::Mason uses actual Perl code for control structures.

But here's where I think Andy did the right thing: TT's control structures are handled by a mini-language. This mini-language carefully hides the differences between data structure access, method calls, and function calls, by sweeping them all under the same dot notation (just like Perl version 6). Thus, the TT mini-language is much more accessible to web designers who don't care to learn the full-blown intricacies of Perl element access and method calls.

Also, the TT mini-language has just enough features to handle templating, but not quite enough to do serious heavy lifting. This helps me to do the right thing in the right places, because I sense a continual increasing difficulty when I'm using TT code where full Perl code is really needed. For example, in a web application, the TT code is used for the ``view'' code of the MVC triad, whereas Perl works better for the ``model'' and ``controller'' parts.

Another thing that TT has going for it is that it was driven early in its development by two key projects: the (sadly now-defunct) etoys.com website, and slashcode.com, the code behind Slashdot and hundreds of other web-based communities. Key developers from both of these communities provided valuable feedback to Andy about real world issues and concerns, guiding Andy in further design and features.

The TT community is also quite active, with the mailing list getting one or two dozen emails a day. Novice questions are welcome, particularly because such questions occasionally point out shortcomings in the language or documentation that we ``experts'' overlook automatically.

The TT mini-language is compiled into Perl code, which is then compiled and loaded into memory for execution. The Perl source code can be cached to disk; the compiled subroutines can be cached in memory. Additional heavy-lifting code can be written directly in Perl, and loaded as modules with nice TT interfaces. For prototyping, you can also embed Perl directly in your templates if you prefer. This all works together to ensure fairly speedy execution, even for traffic-intensive web sites.

Almost every part of TT is configurable, sometimes excessively so. If you want a slightly different grammar for your TT mini-language, you can plug that in. If you want to load your templates from a database instead of a file, you can have that too. If you want your scalars to know how to ``rot13'' themselves, you can get that added. Perhaps this is the source of the ``Toolkit'' part of the name: what you're really getting for your application is a templating system built to your specifications from the large class of templating systems that TT supports.

The TT Language

The TT language consists of directives and variables. Directives provide the control instructions, like IF or WHILE. Variables map directly to Perl variables: scalars, arrays, hashes, and objects.

The TT code can be structured by having templates include other templates in various ways, similar to subroutines in a traditional language.

Let's look a sample template:

    Dear [% name %],
    It has come to our attention that your account is
    in arrears to the sum of [% debt %].
    Please settle your account before [% deadline %] or we
    will be forced to revoke your License to Thrill.
    The Management.

This template contains three directives, which will interpolate the TT variables as indicated. (We'll see later how the variables get their values.) The construct:

  [% GET variable %]

interpolates the value of variable. (The GET is optional, and frequently omitted.) And:

  [% SET variable = some + calculation %]

sets a value into a variable based on other calculations. (The SET is also optional.)

If the [% and %] tags conflict with desired data, they can be changed to nearly arbitrary start and end sequences, such as HTML comment markers. Whitespace between the tags is almost entirely ignored. Comments are like Perl (# to the end of the line).

By default, newlines are kept, so:

  Hello [% a = 3 %]
  World [% a %]

results in "Hello \nWorld 3\n". You can absorb the whitespace by placing - next to either %, as in:

  Hello [% a = 3 -%]
  World [% a -%]

which results in "Hello World 3". A configuration mode called post chomp treats all directives as if they have this trailing minus, which I find very useful and consistent.

Many control directives like FOREACH, WHILE, and IF nest as a block, terminating at a corresponding END. The output of a block directive may be captured into a variable:

  [% result = FOREACH user = userlist %]
  one user is [% user.name %]
  [% END %]

Here, the output from the FOREACH loop ends up in result for later processing.

Directives can be semicolon-separated within a tag pair, which saves an adjacent end-tag/start-tag combination:

  [% result = FOREACH user = userlist %]
  one user is [% user.name; END %]

Some directives can trail other directives, similar to Perl:

  [% b = b * a FOREACH a = [1, 2, 3] %]

Expressions are fairly Perl-like, supporting literal strings with 'single quotes' and "double quotes with $variable interpolation".

Like Perl 6, the concatenation operator is the is underscore, not the dot:

  [% this = that _ other %]

The INSERT directive brings in another chunk of text, often from another file:

    INSERT myfile

The contents are not processed for directives. The file is found along the INCLUDE_PATH, selected by a configuration parameter.

The INCLUDE directive:

    INCLUDE myfile

also brings in other data, but the contents are further examined for TT language constructs, thus making this more like a subroutine call. Additionally, the argument might also be a named block in the same file, providing for local common code for re-use:

  [% BLOCK myfile %]
  some text here, with [% first %] [% last %] names.
  [% END %]

Currently defined variables are visible to the included block or file, but variables set in the included section are not visible back to the rest of the file. Extra local variables can be defined during the invocation:

  [% INCLUDE myfile this = "that" %]

This provides a nice way to pass parameters down to the sub-template.

A PROCESS directive acts like INCLUDE, but the local variables remain in effect, thus sharing the same namespace as the invoker. Typically, these directives are used for speed (localization costs time) or for common variable initialization.

The WRAPPER directive:

  [% WRAPPER foo %]
  some stuff
  [% END %]

acts as if we had said:

  [% INCLUDE foo content = "some stuff" %]

This is great for writing text that wants to wrap some other text with enclosing materials:

  [% INCLUDE comment_label
     WRAPPER my_button color='blue' %]

Here, the contents of including file comment_label can be enclosed in text provided by my_button, possibly modified by the current value of color. In this example, my text and the contents of the myfile file are both enclosed in HTML bold tags:

  [% BLOCK b %]
    <b>[%content%]</b>
  [% END %]
  [% WRAPPER b %]
    my text
  [% END %]
  [% INCLUDE myfile WRAPPER b %]

The output of a block can be captured:

  [% disclaimer = BLOCK %]
    Portions of tonight's show not affecting the outcome were edited.
  [% IF sorry %] We're sorry. [% END %]
  [% END %]

This is a good way to define a large text string for boilerplate to use later.

The TT language provides a Perl-like IF structure:

  [% IF age < 10 %]
    Hello [% name %], does your mother
    know you're using her AOL account?
  [% ELSIF age < 18 %]
    Sorry, you're not old enough to enter
    (and too dumb to lie about your age)!
  [% ELSE %]
    Welcome [% name %].
  [% END %]

And not to be outpaced by Perl6, TT also provides a SWITCH structure:

  [% SWITCH myvar %]
  [% CASE value1 %]
    that value
  [% CASE [value2 value3] %]
    either of those
  [% CASE myhash.keys %]
    any of those keys
  [% CASE %] default
  [% END %]

The FOREACH loop acts like the Perl equivalent:

  [% FOREACH thing = [foo 'Bar' "$foo Baz" ] %]
    * [% thing %]
  [% END %]

When iterating over a hash, omitting the iteration variable causes the keys to be assigned directly as variables:

  [% userlist = [ { id => 'merlyn', name => 'Randal' }
                  { id => 'fred', name => 'Fred Flintstone'} ]
   %]
  [% FOREACH userlist %] 
    [% id %] is [% name %]
  [% END %]

This loop acts like:

  [% FOREACH u = userlist %] 
    [% u.id %] is [% u.name %]
  [% END %]

but with less typing. Nested loops are supported, as are Perl-like NEXT and LAST operations.

Unlike Perl's equivalent, TT's FOREACH directive understands where it is with regard to the loop, and provides meta-information via the loop variable. For example, loop.size gives the total iterations, and loop.last is true if this is the last item. Using these controls, we can make the loops act nicely. For example:

    [% FOREACH i = [ 'foo', 'bar',  'baz' ] %]
    [% IF loop.first %]<ul>[% END %]
      <li>[% loop.count %] of [% loop.size %]: [% i %]
    [% IF loop.last %]</ul>[% END %]
    [% END %]

generates:

    <ul>
      <li>1 of 3: foo
      <li>2 of 3: bar
      <li>3 of 3: baz
    </ul>

Without the loop controls, we'd typically move that start and end tag outside the loop, but then we'd get the tags even if the list was empty. Here, we'll get start and end tags only if we've entered the loop at least once. Very cool.

The WHILE directive provides the expected ``loop as long as an expression is true. NEXT and LAST work as they do in Perl:

  [% WHILE total < 100 %]
    [% total %]
  [% total += 1 %]
  [% END %]

Beware however: to prevent runaway programs, all loops are limited to 1000 iterations (an arbitrary value selected by Andy).

Well, I've run out of room already, and I still have a bit more to say. Next time, I'll finish up the descriptions of the directives, talk about exception handling, data structures, and using TT from Perl. I'll also cover configuration directives, the command-line tools that come with TT's distribution, and using TT with mod_perl. Until next time, enjoy!


Randal L. Schwartz is a renowned expert on the Perl programming language (the lifeblood of the Internet), having contributed to a dozen top-selling books on the subject, and over 200 magazine articles. Schwartz runs a Perl training and consulting company (Stonehenge Consulting Services, Inc of Portland, Oregon), and is a highly sought-after speaker for his masterful stage combination of technical skill, comedic timing, and crowd rapport. And he's a pretty good Karaoke singer, winning contests regularly.

Schwartz can be reached for comment at merlyn@stonehenge.com or +1 503 777-0095, and welcomes questions on Perl and other related topics.