They are highly configurable and can be extended to arbitrary log files, not only log files in /var

Basic log rotation is not that difficult to implement in Perl.
There is a dozen or so ready-made scripts, but the problem is that each
of them has its own set of preconditions and approaches to rotation so you
might be better off writing your own.

There are two approaches, two Unix schools of log rotation: "version
based" and "date based.":

Version based school rotates the log in such a way that the last one has
suffix 1, the previous suffix 2, etc. Each time log rotate N files are
preserved, numbered from 1 to N.

Date based school uses daily, weekly of month rotation pattern and
each rotated log is marked by timestamp. Log for a certain period are
preserved (let's say 6 months; older logs are deleted. This is the approach
that I prefer.)

In Unixes such as AIX you need to provide your own tools. It make sense to
use logrotation program written in Perl as it is more understandable and
modifiable then alternatives for most sysadmins. There is the basic log rotation program in Perl for apache (please note
apache has the ability to rotate logs via piping them into special logrotation
program without restart):

Well-documented Perl log rotation script

rotatelog - rotates files when they reach a certain trigger size.

rotatelog [-cdht]

This is a program to rotate log files. It can rotate any file listed
in its configuration file. The file does not necessarily need to be
log file. This program imitates the logrotate and
newsyslog programs found on Linux and FreeBSD respectively. I
originally wrote this to make life easier in Solaris.

This program will exit if it cannot open its configuration
file(s) and if any of the system calls fail. This program will
also exit if the there are any syntax errors in the actions section.
Any other configuration file errors are ignored and a warning is issued.
All error messages are printed to STDERR. This program
is meant to be run as a cron job so error messages such as these will
be mailed to the owner of the cron job. Many times rotatelog
is run in the root crontab.

The rotatelog program depends upon its configuration
file for information concerning what files to rotate and when to rotate
them. The rotatelog.conf file is located in the /usr/local/etc
directory by default. This location is set in the configuration section
of the source code along with the paths to certain system utility functions.
It is possible to specify a new configuration file via the -c
command line option, and it is also possible to specify multiple configuration
files in this manner as well using the form -c file1,file2,file3.
Note that configuration files that appear later in a multiple configuration
specification can override settings in previous configuration files.
When installing this program it will be necessary to check this section
of the source code. Along with specifying the the files to rotate there
is also the ability to specify ownership and mode of the files after
rotation, the number of rotated files to keep, the compression used
or not used on the rotated files, and an auxiliary action in the form
of a shell command to take place when certain files are rotated. The
rotatelog program is normally run as root.
It is important that the config file is only writable by root
due to the ability to execute shell commands!

There are three sections in the configuration file. They can appear
in any order. Each section is identified by a keyword. Configuration
information follows the keyword and extends until the next keyword.
Keywords can come in any order and repeat, but it would be best to follow
a simple format such as:

Comment lines must contain a pound sign at the start of the line.
These comment lines are ignored along with blank lines and lines containing
only whitespace. In general there can be no leading whitespace on configuration
lines while there can be any amount of whitespace between elements.

The files section specifies which files are to be rotated, when they
are to be rotated, what their ownership and modes are to be post rotation,
how many rotated copies are to be retained, and the compression used
on the files once rotated. The beginning of the files section is noted
by the occurrence of the FILES: keyword.

The fields above can be separated by any amount of whitespace. The
first field is the full path to the file that requires rotation. The
second field is the trigger. The trigger specifies
when rotation is to occur. The trigger is the maximum size
of a file in bytes, kilobytes, or megabytes. Bytes can be specified
with B or b, kilobytes with K or k, or megabytes can be specified with
M or m. If a file listed is found to be larger than the trigger
it is rotated. The owner:group field specifies who is to
own the file once it is rotated. The mode is the file permission
mode in octal notation of the rotated file as specified in the
chmod(1) manual page. The mode does not include the optional
first digit of the four digit chmod(1) octal mode. The
mode is only the simple three digit format shown above. The compression
field specifies the type of compression to use on the rotated file.
A file can be compressed with gzip (gz), compress
(Z), or not at all (none). The archive limit field specifies
how many copies of a rotated file to keep. This number is the highest
count in the rotation scheme. The rotation count starts at 0 so this
is actually one higher then the total number of files kept. If the
archive limit were set to 4 for the file /var/log/wtmp
the maximum number of archived wtmp files would be 5. The
files would appear in the /var/log/ directory as:

wtmp <-- the current wtmp file.
wtmp.0 <-- the first archive of wtmp.
.
.
.
wtmp.4 <-- the last archive of wtmp.

If the last file of an archive exists, wtmp.4 in the above example,
it is removed during the rotation and the previous archive file wtmp.3
is renamed accordingly. All other files are rotated in the same manner
until the current file is reached and rotated.

The actions section specifies if any actions are to occur for any
of the rotated files. The actions section is denoted by the ACTIONS:
keyword. Sometimes a process must be notified that it must close its
file descriptors and open new ones due to something like a file rotation.
Many processes will handle this action when the receive a SIGHUP
signal. Most of the time a kill -HUP on the process will
accomplish this. One example is syslogd. We can see that
syslogd has certain files open all of the time by using
lsof to inquire about the status of one of its log files:

It is not enough to just move the current file into the archive and
touch a new one for writing. The syslogd process must be
told that a new file exists and that it is to close the current file
descriptor and open a new one on the same file name (not to keep writing
to the file pointed to by the current file descriptor). This is what
the actions section does. The actions section has the following syntax:

You are free to add whitespace around the : and
, characters. This extra whitespace will be ignored. When
the /var/log/messages file is to be rotated it is first
moved to a new name, but not compressed. The syslogd program
will still be writing to this file even though the name has changed
because its file descriptor points to this location on the filesystem
and the file descriptor is still open at this point. A new file is touched
and its permissions and ownership are set to the values specified in
the files section. Then the SIGHUP signal is the sent to
syslogd, and it closes the rotated file and opens the new
file that replaces it. After this is done it is safe to compress the
old file if compression was specified in the files section.

There is also a special action called rotate which allows
you to bind multiple files together for rotation. All of the files must
be present in the files section so that rotatelog knows
how to rotate them. When one file is rotated then the other files bound
with it are rotated immediately. The format of rotate is:

rotateN : file1, file2, ... fileN

The rotate command must be followed by an integer. This
allows for more than one binding but only for different groups of files.
You may not leave the integer off. The wtmp file is a good
candidate for this type of rotation. When the wtmp file
is rotated it is a good idea to also rotate the wtmpx file.
A binding of:

rotate1 : /var/adm/wtmp, /var/adm/wtmpx

will rotate both files whenever one of them needs rotation. When
binding files for rotation in this manner, there is only one way to
associate another action (a normal shell command) to the rotation of
the bound files. Using the rotate action indicates that
all files are to be rotated at the same time. If an action is also to
occur, we must be certain that all of the files have begun rotation
before executing that action. An action on the group of bound files
in the rotate group is specified by using that rotate
action as the bound file for the normal shell command action. The
syslogd example is perfect for this. In the syslogd
case one would probably want to send syslogd a SIGHUP
signal once all syslogd files have been rotated. This is
how that is accomplished:

1. Begin rotation on a file.
- Figure out the archive count of the current file.
- Rotate old logs up one count, removing the last if necessary.
- Rotate the current file up one count.
- Touch a new version of the file.

2. Perform any actions on the file.
- Check rotateN bound files.
[if file is in a rotateN group]
- Begin any file rotation on the other files in the
rotateN bound group. All other files are put
through step #1 at this point.
- Perform any normal shell command action associated
with this group of rotateN bound files.
- Finish rotation of all other files bound in the
rotateN group. This is step #3.
- Return. This causes the original file that triggered
this recursion to finish its rotation.
- Check normal shell command actions.
[in this case the file was not in a rotateN group]
- Perform the shell command action.
- Return. This moves the file into step #3.

3. Finish rotating a file.
- Compress the previous log file if the file has been specified
for compression in the files section.
- Add the file to the list of files which have been rotated.

This process ensures that any files bound together are in the beginning
of file rotation before any shell commands are executed as a result
of this rotation. Once the shell command action occurs, if there is
one, all of the files finish their rotation. Due to this flexibility
in file rotation actions, the following rules apply to the actions section:

1. A file may appear in only one rotateN group.

2. A file may appear in only one normal shell command group.

3. A file may not appear in both a rotateN group and a normal shell command action.
4. A normal shell command may be bound explicitly to a file that is
not in a rotateN group.

5. A normal shell command cannot be bound to a file in a rotateN
group explicitly. To bind a shell command to files in a rotateN
group one must implicitly bind the command to the rotateN action.

If there are no actions to perform when any of the files are rotated
this section may be omitted from the configuration file. If you rotate
log files written to by syslogd then you will most certainly
require one of the examples above with the appropriate path to your
syslogd.pid file. If you leave out this step and rotate
syslogd files, syslogd will most certainly
NOT be your friend. If you choose to bind syslogd files
together in a rotateN group, all of the files will be rotated when one
file is rotated. If you want to define the action for each file, only
when that file is rotated, do not use the rotateN form of the action.
It all depends on what you wish to do.

The notify section specifies who is to receive notification in the
event that any files are rotated. The notify section is started with
the NOTIFY: keyword. The notification is sent out via email
to the full email address specified in this section. This is the simplest
section. The email address is simply listed as in the following example:

There can be no leading whitespace on config file lines. Someday
I may fix that. The files must be specified with their full paths in
all sections. This is true in the actions section where you can list
an action for multiple files. If there are any syntax errors in the
configuration file those lines will be ignored and a warning will be
printed to STDERR. If this program is run as a cron job
this will result in an email message to the owner of the cron job. In
most cases this will be root.

This piece of code was written by Shaun Rowland (rowland@interhack.net)
mostly during the early hours of the morning. In my world that is considered
``day'' while afternoon is considered ``night''. Go figure.

Revision 1.5 2001/05/23 15:04:47 rowland Fixed a bug where extra
whitespace at the end of an ACTIONS line would cause the file eq check
file at the end of do action() to fail. Doh! Now we are
good to go.

* Added the -c and -h command line options.
* Added the B and b trigger size specification.
* Changed the release Makefile to create installation directories if
they do not already exist.

Revision 1.3 2001/04/22 23:01:36 rowland Updated perldoc and config
file to better reflect reality in rotating syslogd files. You wouldn
not normally put these files in a rotateN group and then define an action
for that group because this would cause all of the files to be rotated
when just one of them is rotated. You could do this if you wanted, but
it makes a perfect example. Better examples will follow later, but the
examples here should better reflect reality.

Revision 1.1.1.1 2001/04/22 21:58:26 rowland The rotatelog program
was designed to rotate files and perform actions on those files once
rotated if desired. This program began its life as logrotate. I changed
the name so it would not be confused with the GNU logrotate program.
This version includes improved code for handling rotateN bound files
and actions on those bound files (in other words it now works properly).

Perl-Logrotate is a safe log rotation
script. It is fully configurable and comes with compression. You can
define the number of files to keep rotated and when to rotate. If logfiles
cannot be rotated, it will put the log contents back into the file to
avoid data loss. It also works with all logging software, as it only
truncates the file and does not unlink it. There is no need to signal
any processes after this has been run.

[July 16, 1999]Lumberjack
Log rotation software, perl replacement for newsyslog
Jun 08th 1999, 08:48 stable: none - devel: none license: Public Domain.
Lumberjack is a perl replacement for the BSD newsyslog. Logs are rotated
and renamed to logfile.YYMMDD, and the daemon/signal are configurable. Contrary
to the URL for the file, this program should work on all systems, not just
NetBSD.
http://kludge.psc.edu/~ksulliva/netbsd/lumberjack.man.html

FAIR USE NOTICE This site contains
copyrighted material the use of which has not always been specifically
authorized by the copyright owner. We are making such material available
in our efforts to advance understanding of environmental, political,
human rights, economic, democracy, scientific, and social justice
issues, etc. We believe this constitutes a 'fair use' of any such
copyrighted material as provided for in section 107 of the US Copyright
Law. In accordance with Title 17 U.S.C. Section 107, the material on
this site is distributed without profit exclusivly for research and educational purposes. If you wish to use
copyrighted material from this site for purposes of your own that go
beyond 'fair use', you must obtain permission from the copyright owner.

ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no
less then 90 days. Multiple types of probes increase this period.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be
tracked by Google please disable Javascript for this site. This site is perfectly usable without
Javascript.

Original materials copyright belong
to respective owners. Quotes are made for educational purposes only
in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains
copyrighted material the use of which has not always been specifically
authorized by the copyright owner. We are making such material available
to advance understanding of computer science, IT technology, economic, scientific, and social
issues. We believe this constitutes a 'fair use' of any such
copyrighted material as provided by section 107 of the US Copyright Law according to which
such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free)
site written by people for whom English is not a native language. Grammar and spelling errors should
be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development
of this site and speed up access. In case softpanorama.org is down currently there are
two functional mirrors: softpanorama.info (the fastest) and softpanorama.net.

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or
referenced source) and are
not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with.We do not warrant the correctness
of the information provided or its fitness for any purpose.