smokeping_master_slave - How to run multiple distributed instances of SmokePing
Normally smokeping probes run their tests from the host where smokeping runs to
some target host and monitor the latency of the connection between the two.
The Master/Slave concept enables all smokeping probes to run remotely. The use
case for this is to measure the overall connectivity in a network. If you are
interested in checking that your central DNS server or your file server works
for everyone, you could setup several smokeping instances checking up on on
the two servers from multiple locations within your network. With the
Master/Slave smokeping configuration this process becomes much simpler, as one
smokeping master server can control multiple slaves.
All monitoring data is stored and presented on the server, but collected by the
slaves. The slaves will also get their configuration information from the
master, so that you just have to maintain the master server configuration file
and the rest is taken care of automatically.
The slaves communicate with the master smokeping server via the normal smokeping
web interface. On initial startup each slave connects to the master server and
asks for its assignments. When the slave has done a round of probing it
connects to the master again to deliver the results.
If the assignment for a slave changes, the master will tell the slave after the
slave has delivered its results.
The master and the slaves sign their messages by supplying an HMAC-MD5 code (RFC
2104) of the message and a shared secret. Optionally the whole communication
can run over ssl.
[slave 1] [slave 2] [slave 3]
| | |
+-------+ | +--------+
| | |
v v v
| master |
The slave is a normal smokeping instance setup where the configuration comes
from the master instead of a local configuration file. The slave tries to
contact the master server after every round of probing, supplying its results.
If the master server can not be reached, the results will be sent to the
server together with the next round of results. Results will be stored in a
file in Perl storable form so that they survive a restart of the smokeping
To configure a master/slave setup, add a slaves section to your smokeping
configuration file. Each slave has a section in the slaves part of the master
configuration file. The section name must match the hostname of the slave. If
some configuration parameter must be set to a special value for the slave, use
an override section to configure this.
The slave names must be the names the hosts think they have, not their outside
hostnames or ip addresses or anything like that. When the slave calls the
master to get its config or report its measurements it will tell the master
its 'hostname'. This together with the shared secret is used to authenticate
and identify who is who.
*** Slaves ***
Probes.FPing.binary = /usr/bin/fping
Then in the targets section you can define slaves at every level. Again the
settings get inherited by lower order targets and can be overwritten anywhere
in the tree.
A slave will then get the appropriate configuration assigned by the server.
*** Targets ***
slaves = slave1 slave2
slaves = slave1
The data from the slaves will be stored in TargetName~SlaveName.rrd
the example above would create the following files:
file contains a colon separated list of hostnames
A smokeping slave setup has no configuration file. It just needs to know that it
runs in slave-mode and its secret. The secret is stored in a file for optimal
protection. By default the persistent data cache will be located in
./smokeping --master-url=http://smokeping/smokeping.cgi \
file contains a single word, the secret of this slave. It
is NOT the same as the slavesecrets.conf
file the master uses.
The master effectively has full access to slave hosts as the user running the
slave smokeping instance. The configuration is transferred as Perl code that
is evaluated on the slave. While this is done inside a restricted
"Safe" compartment, there are various ways that a malicious master
could embed arbitrary commands in the configuration and get them to run when
the slave probes its targets.
The strength of the shared secret is thus of paramount importance. Brute forcing
the secret would enable a man-in-the-middle to inject a malicious new
configuration and compromise the slave.
Copyright (c) 2007 by Tobias Oetiker, OETIKER+PARTNER AG. All right reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 2 of the License, or (at your option) any later
This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc., 675 Mass
Ave, Cambridge, MA 02139, USA.
Tobias Oetiker <email@example.com>