Copyright Notice

This text is copyright by InfoStrada Communications, Inc., and is used with their permission. Further distribution or use is not permitted.

This text has appeared in an edited form in Linux Magazine magazine. However, the version you are reading here is as the author originally submitted the article for publication, not after their editors applied their creativity.

Please read all the information in the table of contents before using this article.

Linux Magazine Column 68 (Mar 2005)

[Suggested title: ``Fit to be tied (part 1)'']

What is a tied variable? You might have heard the term before, especially if you were accessing a DBM-based hash. But maybe you've wondered how it works, or why you would use it. Let's take a look.

A tied variable is a variable that presents a relatively normal outward appearance to the rest of the Perl program, but has additional wiring on the inside. Specifically, nearly all of the operations on the variable (fetching the value, updating a hash element, getting the length of an array) will trigger subroutine calls. These subroutine calls replace the normal Perl internal operations, and can perform anything you wish, although they must return an appropriate value if the corresponding outward action is looking for one.

For example, we could make $x tied, such that every time we stored a value into it, the value would be written to an external file, and every time we fetched the value from it, the external file would be consulted. By the end of this article, you should be able to write this code from scratch, but let's borrow the implementation from the CPAN as an illustration, using Tie::Persistent instead:

  use Tie::Persistent;
  tie my $x, 'Tie::Persistent', 'myfile';
  $x = 35; # writes to myfile
  my $y = $x; # reads from myfile, returning 35

A tied variable is implemented using Perl objects, so if you're not familiar with them, go study up. It's OK, I'll wait here. Alright then.

You can implement a class like Tie::Persistent by consulting the perltie manpage. There, we'll find that to tie a scalar variable, we need to implement the TIESCALAR, FETCH, and STORE methods at a minimum. How does this work? First, Perl turns the initial tie operation, such as:

  tie $x, MyTie, $a, $b, @c;

into the equivalent class method call of:

  MyTie->TIESCALAR($a, $b, @c);

That means that we need to write such a method definition for our package. But what does the method return? In order for a tied variable to work, the TIESCALAR method must return a Perl object (blessed reference). Any object will do: it doesn't need to be of the same ``type'' as the variable being tied. But for grins, let's make MyTie create a hash-based object, initializing the key of Value to the first parameter passed to TIESCALAR, if any:

  package MyTie;
  sub TIESCALAR {
    my $class = shift; # probably MyTie
    my %self;
    $self{Value} = shift; # first parameter to tie
    return bless \%self, $class;
  }

But where does this object go? Well, it is returned as the result of the tie operator, although that value is rarely captured. But it also gets magically associated with the scalar variable being tied. If we tie $x, we have a ``secret object'' associated with $x until we untie the variable (or it goes out of scope: more on that later).

We can get to this secret object with the tied operator:

   my $secret_object = tied $x;

And now we can define what happens when we try to fetch the current value of $x. Perl transforms:

  $y = $x;

into:

  $y = tied($x)->FETCH;

In other words, the FETCH instance method is called on the secret object behind $x. Let's write a simple one that simply returns the Value in our secret hash object:

  package MyTie;
  sub FETCH {
    my $self = shift; # the secret object
    return $self->{Value};
  }

And now we have the beginnings of a tied mechanism:

  use MyTie;
  tie $x, MyTie, 45; # calls MyTie->TIESCALAR(45)
  $y = $x; # calls tied($x)->FETCH returning 45

The other thing a scalar variable can do is get a new value. When we store something:

  $x = 28;

Perl turns that into:

  tied($x)->STORE(28);

In other words, we're again calling a method on the secret object, passing it the updated value. Let's write a simple method to update our Value element:

  package MyTie;
  sub STORE {
    my $self = shift; # the secret object
    my $newvalue = shift;
    $self->{Value} = $newvalue;
  }

The value returned by a STORE is ignored.

So far, we've created a scalar that does what all scalars do, albeit a bit slower, and with a lot more storage required. But let's take this a step further. We'll modify the variable so that it keeps track of every value it has ever been assigned, and then dump that history when the variable goes out of scope. To do that, we'll have to hook into the DESTROY method for the object, which gets called as a variable is reverting back to its former style as part of going out of scope.

First, we'll just add the debug message:

  package MyTie;
  sub DESTROY {
    my $self = shift;
    warn "deleting with value $self->{Value}\n";
  }

which gets us a notification as the variable is going away. We can verify this with a simple program. Next, let's modify the STORE method to keep a history of the values, and TIESCALAR to initialize the history as well as the initial value:

  package MyHistoricalTie;
  sub TIESCALAR {
    my $class = shift; # probably MyHistoricalTie
    my $self = { Value => shift, History => [] };
    return bless $self, $class;
  }
  sub STORE {
    my $self = shift; # the secret object
    my $newvalue = shift;
    push @{$self->{History}}, $self->{Value};
    $self->{Value} = $newvalue;
  }

Here, we create an additional History element in the hash, initialized to an empty arrayref. As each new value is stored into the tied variable, the previous Value element of this hash is shoved onto the end of that array, and the new value is put in its place. Finally, we'll need to update the DESTROY method to show us the results of our handiwork:

  package MyHistoricalTie;
  sub DESTROY {
    my $self = shift;
    my @history = @{$self->{History}};
    warn "deleting with value $self->{Value}",
      (@history ? " and history @history" : ""),
      "\n";
  }

And now, when we set up the following program:

  tie $x, MyHistoricalTie, 32;
  print $x, "\n";
  $x = 45;
  $x = 60;
  $x = 95;
  print $x, "\n";

We get output that looks like:

  32
  95
  deleting with value 95 and history 32 45 60

What if we wanted to access that history before the end of the program? We can add additional methods into our tie class. Let's add get_history to return the historical values, including the current value, as a list:

  package MyHistoricalTie;
  sub get_history {
    my $self = shift;
    @{$self->{History}}, $self->{Value};
  }

But we need to call this method on an instance, specifically our secret object. How do we get to that? Well, let's use tied again:

  tie $x, MyHistoricalTie, 32;
  $x = 45;
  $x = 60;
  $x = 95;
  print join(", ", tied($x)->get_history), "\n"

And in fact, this prints:

  32, 45, 60, 95

In this way, we can create additional methods on the secret object, and perform ``meta'' operations on the tied variable.

We can disassociate the variable from all of the magical properties using untie:

  untie $x;

At this point, the secret object is called with an UNTIE method, if it exists. There's no harm if it doesn't. Then, the secret object is disconnected from the tied variable. If that was the last reference for the object, the DESTROY method is also called. However, if there is still another reference to the secret object, we get a warning like:

  untie attempted while 1 inner references still exist

This happens if we save the result of tied (or the initial tie) somewhere else, as in:

  tie my $x ...;
  my $secret_object = tied $x;
  untie $x;

We get a warning here because the reference in $secret_object lives on after the variable has been untied.

When a variable goes out of scope, Perl performs an implicit untie without calling the UNTIE method. So, if you have any last requests for your object's data, you should put it into the DELETE method instead.

The scalar we've created as a tied scalar is now sensitive to fetches and stores and going out of scope. But Perl tries really hard to make this work transparently. For example:

  tie my $x, ...;
  some_sub(\$x); # pass a reference into the subroutine
  sub some_sub {
    my $scalarref = shift;
    my $y = $$scalarref; # triggers the FETCH on $x
    $$scalarref = 34; # triggers the STORE on $x
  }

Anything that would have been a fetch or store on the original variable is properly translated. Even something like:

  $_ = 19 for $x, $y, $z;

will cause the store method to be triggered on $x (and $y and $z if they also are tied).

Let's implement that persistent variable now, knowing what we've already seen. We'll have it fetch the old value on getting tied, and store the new value when the secret object is deleted. Other fetches and stores will work against an in-memory copy for speed. We'll use Storable to ensure that the data is written as accurately as possible.

  package MyPersist;
  sub TIESCALAR {
    my $class = shift;
    my $file = shift or die "missing filename";
    require Storable;
    my $self = {
      File => $file,
      Value => (-r $file and Storable::retrieve($file)),
    };
    return bless $self, $class;
  }

We'll be called as:

  tie my $p, MyPersist, 'somefile';

so we save the filename into $file here, and then construct the hash with File and Value elements. To write the value back out, we'll hook in to the DELETE method:

  package MyPersist;
  sub DELETE {
    my $self = shift;
    Storable::nstore($self->{Value}, $self->{File});
  }

And now we just need to make fetches and stores do the right thing:

  package MyPersist;
  sub FETCH {
    return shift->{Value};
  }
  sub STORE {
    shift->{Value} = shift;
  }

These are the same as the versions for MyTie, but I've shortened them up by eliminating the extra variables for grins.

And now, we have a persistent value, albeit only a scalar:

  use MyPersist;
  tie my $p, MyPersist, 'somefile';
  print ++$p;

If we run this program repeatedly, we'll see that it prints 1, 2, 3, 4, and so on.

Well, I've run out of room this month, and I didn't even get to talk about all the other things that can be tied: arrays, hashes, and even filehandles. So, we'll look forward to that for next month. Until then, enjoy!


Randal L. Schwartz is a renowned expert on the Perl programming language (the lifeblood of the Internet), having contributed to a dozen top-selling books on the subject, and over 200 magazine articles. Schwartz runs a Perl training and consulting company (Stonehenge Consulting Services, Inc of Portland, Oregon), and is a highly sought-after speaker for his masterful stage combination of technical skill, comedic timing, and crowd rapport. And he's a pretty good Karaoke singer, winning contests regularly.

Schwartz can be reached for comment at merlyn@stonehenge.com or +1 503 777-0095, and welcomes questions on Perl and other related topics.