Logsurfer is a program that works on single textlines from a
logfile (or from standard input). It compares each line against regular
expressions of a set of "rules" and performs certain actions that are
defined for this match.
The logsurfer uses "rules" to define actions for messages (single
lines). Messages may be collected in "contexts" which are stored in memory.
Those contexts have their own default actions (e.g. for timeouts). At
starttime you have to define a set of rules in the configuration file.
While processing logmessages the logsurfer is able to dynamically add or
delete rules, create or destroy contexts, invoke external programs for
alerting people or to perform further automated actions etc.
A rule starts with a set of regular expressions. The messageline
that is being checked has to match against the first regular expression
but must not
match the second expression. The string "-" can be used as the second
regex if you dont need this "match not" expression. If the messageline
meets these requirements the associated action is taken. Currently the
following actions are allowed: ignore this message, execute an external
program, pass this messageline as standard input to an external program,
create or delete a context, generate a "report" or dynamically add a new
rule. In addition to these actions rules may have a configuration to
delete themself if another message (again specified by two regular
expressions) is being processed by the logsurfer or if an optional
timeout has occured. Matching of messagelines against rules is usually done
until the first match occurs. This can be circumvented by specifying a
"continue" keyword in the rule. In this case the program will continue
to search for another match in the rule definitions.
A context is a set of messagelines that have been read by the
logsurfer. Again the messagelines are specified by two regular expressions.
Contexts can be used in "reports" to present a collection of messages
associated with a specific action (e.g. all messagelines from one ftp
session). Every context has a default action which is launched if either
a specified timeout has occured or a configured limit (of linenumbers) has
been reached. Default actions may be: ignore (drop) this context, execute
an external program, feed all collected logmessages as standard input to
an external program or generate a "report".
A report summarizes a number of contexts and pipes the result
into an external program.
The normal procedure for each logline is: Store this line in all matching
contexts (as defined by their regular expressions), if the self-destroy
regex matches, too, then the default action of the context is executed and the
context is deleted, find the first matching rule and execute the given action
(continue rule matching if the rule has the keyword "continue" --- see
logsurfer.conf(4) manpage), check for the stop-pattern of the rule and in case
of a match delete the rule, check for timeouts of rules and delete the
affected rules, check for context timeouts and execute their default action.
Errors (like broken configuration statements or memory problems) are
reported on standard error. The syslog mechanism is currently not
being used because this might result in a loop if the logsurfer is processing
those syslog-messages. It is suggested to start the logsurfer and redirect
the output (incl. error output) to another file. If necessary you might
want to start a second logsurfer that is watching the output of the first
If the program exits the following steps are executed:
The status of all active rules and contexts is dumped to the dumpfile.
If contexts have timeout values (other than 0) then a timeout is send to
those functions. This will result in launching the default actions of the
Rules and contexts are deleted from memory (cleanup). This is only important
for debugging purposes and should have no effect.
If there are still external programs running the logsurfer will wait up to
20 seconds to catch them.
The logsurfer exits with a return value of zero (0).
This program may consume a lot of memory. Resource control should be
done by the operating system (for example by setting limits). The program
was designed to fall back to a defined state if some resources are not
Please do not run the logsurfer with a privileged userid (like root). You
usually do not need the extra privileges. If your logfiles are not
world-readable change them to be group readable and start the logsurfer
with a non-privileged account that is a member of that group. Keep in
mind, that the logsurfer might execute external programs (depending on
your configuration file) which are not designed to run with higher
privileges. The logsurfer program will print a warning message if its
started with root privilege (this feature can be disabled at compile time).
Be careful with external programs. The logsurfer is able to process any
input (for example arbitrary long lines) but some other programs may
crash or produce other inpredictable results if feeded with long lines.
The following signals have the specified effect when sent to the logsurfer
program using the
Reopens the logfile (to be analyzed). Processing starts with the first line
in this file. This can be used after "shifting" logfiles and creating new
Dump the internal state of all active rules and the collected contexts to
the dumpfile "/dev/null".
Dumps the state to the dumpfile (see above) and exits.