Manual Reference Pages - ARCLOG (1)
arclog - Archive the log files monthly
arclog [options] logfile... [output]
arclog archives the log files monthly. It strips off log entries
that belongs to previous months, and then compresses and saves them
to archived files named logfile.yyyymm.gz.
Currently, arclog supports Apache access log, Syslog, NTP, Apache
1 SSL engine log and my own bracketed, modified ISO date/time log
file formats, and gzip and bzip2 compression methods. Several
software projects log (or can log) in a format compatible with the
Apache access log, like CUPS, ProFTPD, Pure-FTPd... etc., and
arclog can archive their Apache-like log files, too.
Notice: Archiving takes time. To reduce the time occupying the
source log file, arclog copies the content of the source log
file to a temporary working file and restart the source log file
first. Then arclog can take its time working on the temporary
working file. However, please note:
1. If you have a huge log file (several hundreds of MBs), merely
copying still takes a lot of time. In that case, you had better stop
logging first, archive the log file and restart logging, to avoid
racing condition in writing. If you archive the log file periodly,
it shall not grow too big.
2. If arclog stops in the middle of the execution, it will leave
a temporary working file. The next time arclog runs, it will stop
when it sees that temporary working file. You have to process that
temporary working file first. That temporary working file is merely
a copy of the original log file. You can rename and archive it like
an ordinary log file to solve this.
Do not sort unless you have a particular reason. Sorting has the
following potential problem:
1. Sorting may eat huge memory on large log files. The amount of
the memory required depends on the number of records in each archived
month. Modern Linux and MSWin32 have memory consuming protection by
killing processes that eats too much memory, but it still takes
minutes, and your system will hang during that time. I do not know
the memory consuming protection on other operating systems. If you
try, you are at your own risk.
2. The time units of all recognized log formats are second.
Log records happen in a same second will be sorted by the log file
order (if you are archiving several log files at a time) and then
the log record order. I try to ensure that the sorted archived
records are in a correct order of the happening events, but I cannot
guarantee. You have to watch out if the order in a second is
Be careful on the Syslog(2) and NTP log files:
Syslog(2) and NTP does not record the year. arclog
uses Date::Parse(3) to parse the date, which assumes
the year between this month and last next month if the year is
missing. For ex., if today is 2001-06-08, it will then assume the
year between 2001-06-30 back to 2000-07-01 if the year is missing. I
think this is smart enough. However, if you do have a
Syslog(2) or NTP log file that has records older than one
year, do not use arclog. It will destroy your log file.
If read from STDIN, please note:
1. You MUST specify the output prefix if you want to read from
STDIN, since what it needs is an output pathname prefix, not an
2. STDIN cannot be deleted, restarted or partially kept. If you
read from STDIN, the keep mode will fall back to keep all. if
you archive several source log files including STDIN, the keep
mode will fall back to keep all for all source log files, to prevent
3. The answers of the ask mode is obtained from STDIN, too.
Since you have only one STDIN, you cannot specify the ask mode
while reading from STDIN. It will fall back to the fail mode
in that case.
I suggest you to install File::MMagic(3) instead of
counting on the file executable. The internal magic file of
File::MMagic(3) seems to work better than the
file(1) executable. arclog treats everything not
gzip(1) nor bzip2(1) compressed as plain text.
When a compressed log file is wrongly recognized as an image,
arclog will treat it as plain text, read log records directly from
it and fail. This failure does not hurt the source log files, but is
The log file to be archived. Specify - to read from STDIN.
Multiple log files are supported. gzip(1) or
bzip2(1) compressed files are supported, too.
The prefix of the output files. The output files will be named as
output.yyyymm, ie: output.200101, output.200101. If not
specified, the default is the same as the log file. You must specify
this if you want to read from STDIN. You cannot specify -
(STDIN), since this is only a name prefix, not the output file.
Specify the compression method for the archived files. Log files
usually have large number of simular lines. Compress them saves
you lots of disk spaces. (And this is why we want to archive
them.) Currently the following compression methods are supported:
Compress with gzip(1). This is the default. arclog
can use Compress::Zlib(3) to compress instead of
calling gzip(1). This can be safer and faster for not
calling foreign binaries. But if
Compress::Zlib(3) is not installed, it will try
to use gzip(1) instead. If gzip(1) is not
available, either, the program will fail.
Compress with bzip2(1). arclog can use
Compress::Bzip2(3) to compress instead of
calling bzip2(1). This can be safer and faster for not
calling foreign binaries. But if
Compress::Bzip2(3) is not installed, it will try
to use bzip2(1) instead. If bzip2(1) is not
available, either, the program will fail.
No compression at all. (Why? :p)
Do not compress the archived files. This is equal to
Sort the records by time (and then the record order). Sorting eats
huge memory and CPU, so it is disabled by default. See the
description above for a detailed illustration on sorting.
Do not sort the records. This is the default.
Whether we should overwrite the existing archived files. Currently
the following modes are supported:
Overwrite existing target files. You will lost these existing
records. Use with care. This is helpful if you are sure the master
log file has the most complete records.
Append the records to the existing target files. You may destroy the
log file completely by putting irrelevant entries altogether
accidently. Use with care. This is helpful if you append want to
merge 2 or more log files, for ex., 2 log files of different periods.
Ignore any existing target file, and discard all the records of those
months. You will lost these log records. Use with care. This is
helpful if you are supplying log records for the missing months, or
if you are merging the log records in a complex manner.
Stop processing whenever a target file exists, to prevent destroying
existing files by accident. This should be mostly wanted when run
from some automatic mechanism, like crontab(1). So,
this is the default if no terminal is found at STDIN.
Ask you what to do when a target file exists. This should be most
wanted if you are running arclog interactively. So, this is the
default if a terminal is found at STDIN. The answers are read
from STDIN. Since you have only one STDIN, you cannot specify
this mode if you want read the log file from STDIN. In that case,
it will fall back to the <samp>fail</samp> mode. Also, if
arclog cannot get its answer from STDIN, for ex., on a
closed STDIN like crontab(1), it will fall back to
What to keep in the source file. Currently the following modes are
Keep the source file after records are archived.
Restart the source file after records are archived.
Delete the source file after records are archived.
Archive and strip records of previous months off from the log file.
Keep the records of this month in the source log file, to be archived
next month. This is designed to be run from crontab(1)
monthly, so this is the default.
Show the detailed debugging messages.
Shihhhhhh. Only yell when errors.
Display the help message and exit.
Output version information and exit.
Copyright (c) 2001-2007 imacat. All rights reserved.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
imacat <email@example.com>. Please visit arclogs
websites at http://arclog.sourceforge.net/ and
arclog has a mailing list at SourceForge:
firstname.lastname@example.org. It is for
arclogs users to discuss and report problems. Its web page is
at http://lists.sourceforge.net/lists/listinfo/arclog-users .
If you have any problem or question on arclog, please go to
this page, join the list, and send your questions on this list.
Support multi-lingual, either with Text::Iconv(3) or
perl 5.8.0s Encode(3).
|perl v5.20.3 ||ARCLOG (1) ||2007-12-03 |
Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.