Manual Reference Pages - DNSCHECK::OVERVIEW (3)
The DNSCheck System
The central part of the DNSCheck system are the Perl modules that perform the
actual tests. All of them should be used by creating a DNSCheck.pm object
and letting it create other objects as needed and/or requested. These modules
will make lots of queries to DNS servers all around the world, and sooner or
later return a result in the form of a list of messages.
At this level, the libs can be used for any number of tasks. In the examples
directory in the source distribution there is, for example, a script that lets
one monitor the status of a domain via the Nagios monitoring system.
The system as distributed, however, contains more than the lib. There are also
a couple of programs and a web interface. We will not talk about the web
interface here, except to mention that it like the rest of the programs
communicate with the other parts via the database tables to be mentioned
The program that actually runs tests is dnscheck-dispatch. It polls the
table queue table in the database every 0.25 seconds, and forks children to
run tests on what it finds there. It monitors the state of the children, and
cleans up after they exit. If a child exits uncleanly, the test it was running
is re-scheduled. If the same domain causes unclean exits five times or more,
that domain will be blacklisted and will not be tested again until the
dispatcher is restarted.
If the dispatcher receives a TERM signal, it will stop starting new tests,
wait for the ones currently running to exit and then exit in good order. This
may at worst take several minutes.
If the dispatcher receives a HUP signal, it will do an orderly exit as above,
and then re-execute itself. If it was not originally started with a full
pathname, the re-execution may fail.
The dispatcher is written to do a small number of tasks (get queue entry,
start child, clean up after it) in as robust fashion as possible. As of this
writing, the program appears to be very reliable as long as the database
server is not too heavily loaded. Exactly what too heavily loaded means in
practice will vary, of course, but as a rule of thumb, if a simple UPDATE
takes more than a second to finish the system is almost certainly there.
It is <B>vitalB> that the queue table use the InnoDB storage engine. If it is
MyISAM, there will be data corruption under heavy load. Other storage engines
with sane transaction handling should also work, but have not been tested.
At the moment, only MySQL is supported. Patches to add support for more
databases are welcome.
The dispatchers child processes run the default tests on the given domain,
write the results to the tests and results tables in the database and
The zone change detector
One of the programs that insert entries into the queue table is
dnscheck-zonediff. This program does a zone transfer of a specified domain
and looks at all the NS and DS records. For ever subdomain, itll
concatenate the textual representations of the records and calculate an MD5
sum of the resulting string. If that sum is not identical with what it was the
previous time the program was run, the subdomain in question will be scheduled
for testing (that is, itll be written to the queue table, where the
dispatcher will pick it up).
The change detector stores data between runs in a simple text file, with a
zone name and its corresponding MD5 sum on each line.
At the moment, the time it takes to do the AXFR is limited by the speed at
which the Perl module Net::DNS can create new records. It also uses a fair bit
of RAM (100-150MB for around 750 thousand domains), so if youre planning to
run it against .com youll want a pretty hefty machine.
The results mailer
The dnscheck-12hourmailer script reads results of tests from the database
tables tests and results, aggregates them per registrar and sends an
email to each registrar with a summary of serious problems found among their
This is pretty specific to .SE, particularly the bit that maps domain names
to registrars and fetches contact information for the registrars. If you want
to do something similar to this, youll probably need to modify the code of
the script. At the moment, the emails are also generated entirely
programmatically, so changing their formatting also involves changing code.
The preflight script
dnscheck-preflight is used by the web GUI. It takes a name as a command
line argument and prints the string TRUE if its a domain and FALSE otherwise.
The hostsyntax script
dnscheck-hostsyntax is also used by the web GUI. It takes a name as a
command line argument, and prints the string TRUE if it is a syntactically
valid hostname and FALSE otherwise.
The CLI interface
dnscheck is a command line tool to check domains. Run it with the switch
--help to get a short description of how to use it.
Its entirely possible to use the lib part of DNSCheck without a database. All
the surrounding functionality, however, needs one. All communication between
various parts of the entire system is done entirely through database tables.
While this may not be the most convenient interface ever, it does mean that
anything that can talk to MySQL can be a part of the system, which is useful.
There are a number of tables. Lets start looking at them from the top of, so
to speak, the food chain.
The source table
Anythingt that can write to the queue table can be a source of domains to
be tested. In order to make it possible to keep track of which source
requested which test, there is the source table. It has only three fields,
and is supposed to be mostly read-only. The fields are id, name and
contact. The id is an autoincrementing integer as usual. The name is a
human-readable name for the source. The contact is some sort of human-useable
contact information for the source in question (email address, phone number,
ham radio call sign,...).
One example of the use of source information is the mailer described above,
which only mails results about tests that were scheduled for testing by the
zone change detector, not once that came from (for example) the web GUI.
The queue table
This table has six fields apart from the autoincrementing id. There is
domain, which is the name of a domain to be tested. There is priority,
which is an integer. dnscheck-dispatch will always pick the queue entry
with the highest priority first (and if there are several with the same
priority, the one with the lowest id). There is inprogress, which is NULL
to begin with and gets a timestamp written to it by the dispatcher when its
been scheduled for testing. There is a tester_pid field, where the
particular child process thats running the test for the domain writes its
system PID (if we ever need to have more than one machince running tests, thre
maybe a host field added here). source_id pints into the source table.
source_data is any data specific to this test that is relevant for the
specific source. The two source fields are simply copied over to the tests
table when a test is started.
The tests table
This table has one row for every test run, an contains information on the
domain tested, the start and finish times of the test, a summary of the
results and the source information that was copied from the queue table. The
fields are: id as usual, domain for the domain name, begin for the
start time (with whole-second resolution), end for the finish time,
count_critical, count_error, count_warning, count_notice and
count_info for the number of problems at the respecive severity levels,
source_id to point into the source table and finally source_data to hold
whatever data that source wants.
The results table
This table has a number of rows for every test. They reflect the list of
results returned after a test is run, and several of the fields are rather
specific to the web GUI application. The fields that can be useful to others
are: test_id, which points into the tests table. timestamp with is
the time at which the log entry was entered (with microsecond resolution).
level which is one of the all-caps strings CRITICAL, ERROR, WARNING, NOTICE
or INFO. message which is the message tag that describes the reason the log
entry was made. arg0 to arg9, which are arguments to the message.
The message tags are specified in YAML files.
The messages table
The messages table contains a copy of a localy YAML file. It is used by the
The delegation_history table
This table contains information about which nameservers has historically been
authoritative for domains. Some tests use it, if its available.
The nameservers and nsp tables
The function of these tables have been obsoleted by the registrar system.
The domains table
A list of domains. More of a convenience thing than an actual part of the
The various programs need different sets of permissions in the database in
order to work correctly. It breaks down as follows:
In practice, the easiest reasonably secure setup is to give the web GUI one
user with very limited permissions (INSERT, UPDATE, SELECT on sources and
INSERT on queue), and one user with full rights on all tables for the rest
of the system. Indeed, since all of the programs use the database connection
info given in the site_config.yaml file, you will need to point out a
different such file to any programs you want to use different connection
Everything that schedules tests needs INSERT, UPDATE and SELECT on sources
and INSERT on queue.
Everything that reads test results needs SELECT on sources, tests and
Anything that runs tests needs SELECT on delegation_history (or no database
access at all).
dnscheck-dispatcher needs a lot: INSERT, SELECT, UPDATE and DELETE on
queue, INSERT, SELECT and UPDATE on tests, INSERT on results plus
everything listed above.
The system uses a number of files.
config.yaml and site_config.yaml
These contain configuration information in YAML format. The config.yaml
file is installed with DNSCheck, and should not be modified (if you do modify
it, your changes will be overwritten when you upgrade DNSCheck).
site_config.yaml overrides what is in config.yaml, and is where local
changes (such as database access information and TSIG keys) should be entered.
It will not be touched by the installation process.
One key that is usually not in either of these files but that can be added if
needed is the disable key. It has two levels under it, being the name a test
module (converted to lowercase) and the name of a method in that module. If
the value for the method is true, that method will not be run. Note that no
kind of sanity checking is done, and that some tests rely on each other, so
disabling one may make others give false results. Be cautious and know what
youre doing. As an example, the following would disable all DNSSEC tests:
All test methods mentioned in the documentation for the respective modules are
possible to disable this way.
policy.yaml and site_policy.yaml
policy.yaml specifies the severity of various problems. As distributed, it
represents .SEs view on best practices for DNS. If you want to change
anything, do so in site_policy.yaml, or your changes will be overwritten on
the next install.
locale/en.yaml and locale/sv.yaml
These files contain information on how to transform the raw test log entries
into human-readable messages. If you want to do your own translation, you need
to know that the format key is a sprintf() format string, and args the
number of arguments after the format. If the latter number does not match the
Perl code, a warning will be printed at runtime.
|perl v5.20.3 ||DNSCHECK::OVERVIEW (3) ||2010-06-29 |
Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.