Copyright Notice

This text is copyright by InfoStrada Communications, Inc., and is used with their permission. Further distribution or use is not permitted.

This text has appeared in an edited form in Linux Magazine magazine. However, the version you are reading here is as the author originally submitted the article for publication, not after their editors applied their creativity.

Please read all the information in the table of contents before using this article.
Download this listing!

Linux Magazine Column 59 (May 2004)

[suggested title: ``Using Perl/Tk for Simple Graphing'']

With the recent multiple outbreaks of varying kinds of Windows-based worms to generate ever-increasing Spam loads, I've been taxed in my role as the system administrator for the company server to maintain vigilence against the attacks. While the actual worms cannot infect my box, the onslaught of worm payloads (and the inevitable increase in spam from 0wn3d machines) has threatened a continuing denial of service attack. At one point recently, I was accepting and attempting to process over 2000 worm payloads per hour (including generating RFC-mandated bounce messages for those) as well as handling the 2000 extremely false ``you have a virus'' messages thoughtfully (not!) generated by the antivirus blockers.

On my machine, the most limited commodity during a heavy mail storm is the CPU. I've got plenty of RAM and disk space, but when I'm handling 20 to 50 simultaneous SMTP transactions, and sending them all over to Amavisd to be processed, my load average can start climbing up to 5 or 10 quite rapidly, which pretty much shuts down everything else that the system is trying to do for me. For a while, I was even running a script that watched the load average at 15-second intervals, and if the load exceeded 6, automatically shut down my mail system (gracefully) until the load returned to 2 or less.

Now that I've gotten things a bit more in hand, I knew that I wanted to monitor load average changes for at least the times that I was sitting at my laptop. I looked at xload, but didn't like the output format. And, xload also either has to run on the server or connect to the rwhod daemon for a remote probe. Well, I'm not running rwhod, so I'd have to run it on the server, causing lots of extra net traffic to update a graphic in real time.

But, I had remembered someone recently talking about how Perl-Tk can create nice graphical user interfaces. I'd never spent much time learning Perl-Tk, so I considered this the perfect opportunity to tackle the learning problem by using a practical problem and need. I cracked open my recently acquired Mastering Perl/Tk from O'Reilly, and started reading.

The Tk toolkit was originally created as an extension to the TCL command language, but was ported early-on to have a general interface, including bindings for Perl. (When I'm feeling bold, I often claim that Tk and Expect are the only two things that has kept TCL from just disappearing complete.) A Tk program draws widgets on an X11 Server display. These widgets range from simple text labels up to complicated graphs and more, allowing for a complex dynamic user-interface to be created in short order, once you get the basics down. In fact, there's even a web browser (with clickable links and embedded images and form support) built just with Perl-Tk.

Programming for Tk requires a bit of rethinking, if you're used to traditional command-line programming. A Tk program is event-loop driven, meaning that you set every thing up for your interface, and then just say ``go'' at the bottom of your program. Once the main loop has started running, any further actions of your program are triggered by responding to events. These events can come from mouse/keyboard interaction, from timers, or even from watching for a filehandle to become ready to read.

For example, you can set up a button on the screen with an event handler callback when the button is pressed. If the user presses the button, your callback gets called. These callbacks are single-threaded: while your callback is executing, the rest of the interface is locked out.

This simplifies the design of the application (over say, a threading or forked application), because you never have to worry that some variable is changing out from under you. However, this strategy also complicates things a bit: you should never have any long-running action for a callback. While I don't have the space here to get much more into the design strategies to help with this issue, let me say that it does take a bit of familiarization before it comes clear.

My application was actually quite simple, thanks to the Tk::Graph module (found in the CPAN). This module creates a Graph widget, ready to accept data values to display them. The only other thing in my application was a way to fetch the load average values. I realized that I could execute a command such as:

  while uptime; do sleep 5; done

on my server, and this would give me a line such as:

   9:36AM  up 3 days, 55 mins, 2 users, load averages: 0.58, 0.51, 0.59

and updated every five seconds. Through an ssh command, I could run this loop remotely, and I'd get a low-bandwidth data stream of my desired data. And, the appearance of new data is also the event that causes my application to update the display, so that's all there is to it. Let's take a look at the program in [listing one, below]:

Lines 1 and 2 are the standard Perl preamble, turning on warnings and compiler restrictions. (For those of you that are long-time readers of my work, I left out the traditional $|++ because this program doesn't produce output on STDOUT, and I was saving about six keystrokes that way.)

Lines 4 and 5 pull in the Tk and Tk::Graph module, respectively. Although Tk will dynamically pull in most widget module definitions, it has this annoying habit of spitting out trace messages to STDERR about additional things being pulled in on demand, so it seems to be better to add it directly into your program to keep it quiet.

Line 7 is the only configuration constant: the host to which we'll ssh to get the load average data.

Line 9 creates the MainWindow widget object. This object represents the ``application'' to the user. When this object disappears, it takes everything else down with it, so we treat it very specially. Note however that nothing has actually happened on the screen, and won't until we start the main loop at the end of the program.

Lines 13 to 39 create the Graph widget. We create this as a child of the main window to indicate that it logically belongs there. Widgets are always created in a hierarchical fashion, and almost always displayed in such a way that children are within their parent widgets.

Lines 15 to 37 define the configuration parameters for this widget, as defined by the widget specification. Most of these were tweaking various things to get the graph to ``look right''. For example, I'm asking for a line graph, with lines 3 pixels wide, automatically scrolling the last 60 values, labeled appropriately. The -config element subhash defines the three lines with their colors. Note that I'm not much of a graphics guy, but I thought red for the most important one (the one-minute load average) was the best.

Line 39 defines the visual relationship of this graph widget to its parent. Tk comes with a few different geometry managers, and probably the simplest to use and understand is the pack manager. In this case, we're telling pack to manage this widget, and to have it fill up the entire space of the parent widget. This means that my graph will always be as big as the enclosing window, so I can make it small when I want it in the corner of my laptop display, or take up a full screen when I want to see it across the room.

Had the main loop been running already, this would also have the effect of actually popping the graph onto the display from what was formerly an empty main window. But, since we're doing all this work before starting the main loop, we won't be seeing anything just yet.

The next thing to do (in lines 42 to 43) is to wire up the remote uptime loop. A simple pipe-process-open sufficies here. Note that I have to force /bin/sh syntax for this loop, because my default login shell is tcsh on the server, resulting in ugly triple-quoting.

Line 45 connects any ``ready-to-read'' condition on the filehandle to an event callback: here, calling my uptime_ready_to_read subroutine defined a few lines down. When the mainloop is waiting for something to do, one of the things it will now watch is if there is any data in the unbuffered handle of UPTIME. If so, my subroutine will be called, and I can process that data. The key here is the unbuffered handle: I must be very careful never to use the normal buffered I/O with this handle, or the event may not trigger later at the right time.

Lines 47 to 63 set up the response to data being available on the handle. I'm creating a static local variable called $buffer, initially an empty string, in line 48.

The problem I'm solving here is that the handle may be ready to read when only a part of a line has been seen across the socket connection, or even when two lines are available (if there's been a network lag). So, I read whatever I can, appending it to my own maintained buffer, and then process as many whole lines as I can from the front of the buffer. Line 52 reads whatever data is available on the unbuffered handle (up to 1K), and puts it at the of $buffer. Line 53 tries to repeatedly remove an entire line from the beginning of the buffer, and if successful, this line is placed in $input in line 54.

The 1024-byte limit in line 52 is just an arbitrary number with respect to functionality. If you set this to ``1 byte'', it just means that the subroutine would get called to read each of the 60-ish characters of a line and then immediately return, triggering yet another ``ready to read'' event again and again until a complete line had been seen. If the number is set too big, we'll still only get what is ready.

Lines 55 to 57 extract the three load-average numbers from the end of the string into the three named variables. Line 58 was a debugging trace I inserted to ensure that everything was being seen properly. I left it in in case I was changing the regex and wanted to ensure proper parsing.

Line 59 changes the X11 Window Title on the application's main window as we see each new value. This is a nice feedback item, because I can mouse-over the window when it is iconified and still get the load averages.

Line 60 is really the crux of the program. Take the three load averages, and plot them as the next item of data. The graph widget does all the hard work from there, figuring out the maximum scale, maintaining the last sixty data points of the 5 second intervals, giving me a nice 5-minute window of activity display. And once that's done, we're done.

Line 65 is the invocation of the MainLoop, which puts Tk's event manager in charge. This call returns only when the main window goes away. Since we didn't give any explicit buttons or actions to do that, the only way the main window goes away is when the user clicks on the standard close-window box. Once the MainLoop starts, window management events (like resize or iconify requests) are handled automatically, and the ``ready to read'' condition on the filehandle triggers more data to be added to the graph. It's all that simple.

Hopefully, you've seen how easy it is to create an arbitrary management tool in a few simple lines of code. Until next time, enjoy!

Listings

        =1=     #!/usr/bin/perl -w
        =2=     use strict;
        =3=     
        =4=     use Tk;                         # CPAN
        =5=     use Tk::Graph;                  # CPAN
        =6=     
        =7=     my $HOST = "blue.stonehenge.comm";
        =8=     
        =9=     my $mw = MainWindow->new;
        =10=    
        =11=    ## create the Graph widget:
        =12=    
        =13=    my $graph = $mw->Graph
        =14=      (
        =15=       -type => 'LINE',
        =16=       -linewidth => 3,
        =17=       -look => 60,
        =18=       -sortnames => 'alpha',
        =19=       -legend => 0,
        =20=       -headroom => 10,
        =21=       -ylabel => 'load',
        =22=       -xlabel => '5 sec units',
        =23=       -yformat => '%.2f',
        =24=       -config => {
        =25=                   av_01 => {
        =26=                             -title => 'One',
        =27=                             -color => 'red',
        =28=                            },
        =29=                   av_05 => {
        =30=                             -title => 'Five',
        =31=                             -color => 'orange',
        =32=                            },
        =33=                   av_15 => {
        =34=                             -title => 'Fifteen',
        =35=                             -color => 'yellow',
        =36=                            },
        =37=                  },
        =38=      );
        =39=    $graph->pack(-expand => 1, -fill => 'both');
        =40=    
        =41=    ## setup the uptime process
        =42=    open UPTIME, "ssh $HOST 'sh -c \"while uptime; do sleep 5; done\"'|"
        =43=      or die "child: $!";
        =44=    
        =45=    $mw->fileevent(\*UPTIME, readable => \&uptime_ready_to_read);
        =46=    
        =47=    BEGIN {
        =48=      my $buffer = "";
        =49=    
        =50=      sub uptime_ready_to_read {
        =51=    
        =52=        sysread(\*UPTIME, $buffer, 1024, length $buffer);
        =53=        while ($buffer =~ s/^(.*)\n//) {
        =54=          my $input = $1;
        =55=          my ($one, $five, $fifteen) =
        =56=            $input =~ /load av[^0-9.]+([0-9.]+)[^0-9.]+([0-9.]+)[^0-9.]+([0-9.]+)$/
        =57=              or die "cannot grok $input";
        =58=          ## warn "saw $one $five $fifteen";
        =59=          $mw->configure(-title => "$HOST: $one, $five, $fifteen");
        =60=          $graph->set({ av_01 => $one, av_05 => $five, av_15 => $fifteen });
        =61=        }
        =62=      }
        =63=    }
        =64=    
        =65=    MainLoop;

Randal L. Schwartz is a renowned expert on the Perl programming language (the lifeblood of the Internet), having contributed to a dozen top-selling books on the subject, and over 200 magazine articles. Schwartz runs a Perl training and consulting company (Stonehenge Consulting Services, Inc of Portland, Oregon), and is a highly sought-after speaker for his masterful stage combination of technical skill, comedic timing, and crowd rapport. And he's a pretty good Karaoke singer, winning contests regularly.

Schwartz can be reached for comment at merlyn@stonehenge.com or +1 503 777-0095, and welcomes questions on Perl and other related topics.