This page describes, in Unix manual page style, a Perl program available for downloading from this site which allows a system administrator to watch entries, as they are added, in any number of log files on one or more machines on a network. Logs which transfer to new files are automatically followed, and an option allows translation of numeric Internet addresses into the corresponding hostnames where possible. Log items can be relayed to one or more other hosts on the network, permitting a machine's local log files to be monitored there.
logtail — Monitor multiple log files on multiple machines
logtail [ −dnqu ] [ −ccols ] [ −ehostname ] [ −l[port] ] [ −pinterval ] [ −rtime ] [ −stime ] [ −tport ] [ −vlevel ] [ −wcols ] logfile…
Long-term survivors in the system administration game know that an excellent way to spot little emerging mouselike problems before they mature into moose-sized disasters is keeping an eye on the various system log files. In days of yore this was quite simple since almost everything of interest went into the master Unix logfile and, if a networked system supported the “loghost” facility, logs for a number of machines on a network could be consolidated onto a single server.
As more and more services have come into use, log files have proliferated. Today, an administrator might want to keep an eye on:
and various others. Many of the widely-used dæmons for these functions do not use the common Unix log mechanism, insisting instead on appending entries to their own private log files. In the case of commercial firewall, backup, and UPS dæmons, source code is not usually available, so modifying them to use the system log is not an option.
When there was a single log file, monitoring it couldn't be simpler—just use:
tail −f /var/adm/messages
(or whatever the log file is called on your system). The −f option causes tail to start at the end of the file and periodically monitor it for growth, printing any additions to the file. Unfortunately, tail only monitors a single file, so the only way to watch multiple files is to launch separate jobs each running tail on an individual file—yuck. Worse, to prevent log files from growing without bound, consuming all available disc space, many programs that write logs provide a mechanism, automatic or semi−automatic, to periodically cycle the log to a new file, renaming the previous log file. Unfortunately, tail −f does not detect that this has happened, and just waits forever at the end of the old log, not knowing a new file is now being written. If servers are run on a number of different machines (or you simply want to watch Unix log files on all the hosts you're responsible for to keep an eye out for developing disc problems, etc.), you would then have to rsh copies of tail −f on every machine you wanted to monitor, and then make sure the process got restarted when a machine rebooted.
Enter logtail, which allows you to monitor any number of log files on any number of machines (assuming they support Perl with networking and use ASCII log files). Logs from any machine can be echoed to as many other machines as you like. Log files are closed and re-opened at a user-defined interval, so cycling of log files into new files is followed automatically.
When the −r option is specified, logtail attempts to replace any sequences which resemble numeric Internet addresses (IP addresses) with the corresponding host name, looked up with the system's resolver. For example, an HTTP log item such as:
172.27.151.132 - - [05/Nov/1997:17:05:12 +0100] "GET / HTTP/1.0" 200 2668
might be translated into:
roswell.area51.af.mil - - [05/Nov/1997:17:05:12 +0100] "GET / HTTP/1.0" 200 2668
If the host name lookup is unsuccessful, the original numeric address remains intact.
Host names corresponding to numeric IP addresses are saved by logtail in an internal cache so multiple accesses from a given host do not require repeatedly querying the system's host name lookup mechanism. Entries in the name cache are purged after time minutes of inactivity, with 10 minutes the default if no time is given.
At first glance, this host name lookup may appear to be the niftiest single feature of logtail, but before you turn it on, ponder carefully the consequences for the system running the program. If, for example, you're monitoring the HTTP access log of a heavily loaded Web site, the −r option may result in thousands, perhaps millions, of calls to the resolver every hour, most of which will require multiple Internet accesses to determine the host name from the IP address. In order to make these requests without blocking the processing of other items, logtail performs each of these lookups in a separate Unix process, but if your system's resolver is single-threaded, everything may still grind to a halt due to the resolver bottleneck. Worse, your own resolver and those of upstream Internet sites may grow to enormous size due to all the requests cluttering up its own internal cache.
As long as the volume of log items is relatively small (say, a couple per minute), and/or the vast majority of references are to local hosts whose names can be looked up almost instantaneously, the −r option can substantially increase the comprehensibility of the log. But if you switch it on while monitoring a million-hit-per-day Web site, you're almost certain to end up regretting it.
logtail is a Perl program. In order to use it, you must have Perl installed on your system. logtail was developed using Perl 5.22.
The host name lookup (−r option) and relaying between hosts (−e and −l options) make extensive use of Unix networking and process control facilities. They are unlikely in the extreme to work on non-Unix systems, at least not without a great deal of modification. Messages are sent between hosts using the UDP (datagram) protocol, which is a “fire and forget” mechanism. If messages are lost due to network congestion or load on the receiving machine, they will simply disappear.
The −r option, which attempts to translate numeric Internet addresses into host names, can cripple a heavily loaded system by a flood of name server requests. Read the discussion of whether this option is appropriate to use on your system in the OPTIONS section above before enabling it.
When logtail begins monitoring a new cycle of a file, entries written between the time the file was cycled and the time logtail transferred to monitoring it are not printed; only subsequently added entries will be seen. This is deliberate: the HTTP log of a busy Web site, for example, can grow to a very substantial size in the interval between being cycled and logtail's discovering it. Processing all the backed up items may be very time-consuming, and relaying them to other hosts may run into maximum network packet size restrictions which would require substantial additional complexity to work around. Since logtail is intended to provide an overview of system activity, a few overlooked messages on those infrequent occasions when files are cycled doesn't seem unreasonable.
Each of the logfile arguments named on the command line is monitored for growth and any additional material is printed. Output from logtail is written to standard output, and may be redirected.
fork(2), getnameinfo(3), perl(1), rsh(1), tail(1)
Download logtail.tar.gz
(gzipped TAR archive)This software is in the public domain. Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, without any conditions or restrictions. This software is provided “as is” without express or implied warranty.