GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  DNSCHECK::OVERVIEW (3)

.ds Aq ’

NAME

The DNSCheck System

CONTENTS

DESCRIPTION

    Introduction

The central part of the DNSCheck system are the Perl modules that perform the actual tests. All of them should be used by creating a DNSCheck.pm object and letting it create other objects as needed and/or requested. These modules will make lots of queries to DNS servers all around the world, and sooner or later return a result in the form of a list of messages.

At this level, the libs can be used for any number of tasks. In the examples directory in the source distribution there is, for example, a script that lets one monitor the status of a domain via the Nagios monitoring system.

The system as distributed, however, contains more than the lib. There are also a couple of programs and a web interface. We will not talk about the web interface here, except to mention that it like the rest of the programs communicate with the other parts via the database tables to be mentioned later.

    The programs

The dispatcher

The program that actually runs tests is dnscheck-dispatch. It polls the table queue table in the database every 0.25 seconds, and forks children to run tests on what it finds there. It monitors the state of the children, and cleans up after they exit. If a child exits uncleanly, the test it was running is re-scheduled. If the same domain causes unclean exits five times or more, that domain will be blacklisted and will not be tested again until the dispatcher is restarted.

If the dispatcher receives a TERM signal, it will stop starting new tests, wait for the ones currently running to exit and then exit in good order. This may at worst take several minutes.

If the dispatcher receives a HUP signal, it will do an orderly exit as above, and then re-execute itself. If it was not originally started with a full pathname, the re-execution may fail.

The dispatcher is written to do a small number of tasks (get queue entry, start child, clean up after it) in as robust fashion as possible. As of this writing, the program appears to be very reliable as long as the database server is not too heavily loaded. Exactly what too heavily loaded means in practice will vary, of course, but as a rule of thumb, if a simple UPDATE takes more than a second to finish the system is almost certainly there.

It is <B>vitalB> that the queue table use the InnoDB storage engine. If it is MyISAM, there will be data corruption under heavy load. Other storage engines with sane transaction handling should also work, but have not been tested.

At the moment, only MySQL is supported. Patches to add support for more databases are welcome.

The dispatcher’s child processes run the default tests on the given domain, write the results to the tests and results tables in the database and then exits.

The zone change detector

One of the programs that insert entries into the queue table is dnscheck-zonediff. This program does a zone transfer of a specified domain and looks at all the NS and DS records. For ever subdomain, it’ll concatenate the textual representations of the records and calculate an MD5 sum of the resulting string. If that sum is not identical with what it was the previous time the program was run, the subdomain in question will be scheduled for testing (that is, it’ll be written to the queue table, where the dispatcher will pick it up).

The change detector stores data between runs in a simple text file, with a zone name and its corresponding MD5 sum on each line.

At the moment, the time it takes to do the AXFR is limited by the speed at which the Perl module Net::DNS can create new records. It also uses a fair bit of RAM (100-150MB for around 750 thousand domains), so if you’re planning to run it against .com you’ll want a pretty hefty machine.

The results mailer

The dnscheck-12hourmailer script reads results of tests from the database tables tests and results, aggregates them per registrar and sends an email to each registrar with a summary of serious problems found among their domains.

This is pretty specific to .SE, particularly the bit that maps domain names to registrars and fetches contact information for the registrars. If you want to do something similar to this, you’ll probably need to modify the code of the script. At the moment, the emails are also generated entirely programmatically, so changing their formatting also involves changing code.

The preflight script

dnscheck-preflight is used by the web GUI. It takes a name as a command line argument and prints the string TRUE if it’s a domain and FALSE otherwise.

The hostsyntax script

dnscheck-hostsyntax is also used by the web GUI. It takes a name as a command line argument, and prints the string TRUE if it is a syntactically valid hostname and FALSE otherwise.

The CLI interface

dnscheck is a command line tool to check domains. Run it with the switch --help to get a short description of how to use it.

    The database

It’s entirely possible to use the lib part of DNSCheck without a database. All the surrounding functionality, however, needs one. All communication between various parts of the entire system is done entirely through database tables. While this may not be the most convenient interface ever, it does mean that anything that can talk to MySQL can be a part of the system, which is useful.

There are a number of tables. Let’s start looking at them from the top of, so to speak, the food chain.

The source table

Anythingt that can write to the queue table can be a source of domains to be tested. In order to make it possible to keep track of which source requested which test, there is the source table. It has only three fields, and is supposed to be mostly read-only. The fields are id, name and contact. The id is an autoincrementing integer as usual. The name is a human-readable name for the source. The contact is some sort of human-useable contact information for the source in question (email address, phone number, ham radio call sign,...).

One example of the use of source information is the mailer described above, which only mails results about tests that were scheduled for testing by the zone change detector, not once that came from (for example) the web GUI.

The queue table

This table has six fields apart from the autoincrementing id. There is domain, which is the name of a domain to be tested. There is priority, which is an integer. dnscheck-dispatch will always pick the queue entry with the highest priority first (and if there are several with the same priority, the one with the lowest id). There is inprogress, which is NULL to begin with and gets a timestamp written to it by the dispatcher when it’s been scheduled for testing. There is a tester_pid field, where the particular child process that’s running the test for the domain writes it’s system PID (if we ever need to have more than one machince running tests, thre maybe a host field added here). source_id pints into the source table. source_data is any data specific to this test that is relevant for the specific source. The two source fields are simply copied over to the tests table when a test is started.

The tests table

This table has one row for every test run, an contains information on the domain tested, the start and finish times of the test, a summary of the results and the source information that was copied from the queue table. The fields are: id as usual, domain for the domain name, begin for the start time (with whole-second resolution), end for the finish time, count_critical, count_error, count_warning, count_notice and count_info for the number of problems at the respecive severity levels, source_id to point into the source table and finally source_data to hold whatever data that source wants.

The results table

This table has a number of rows for every test. They reflect the list of results returned after a test is run, and several of the fields are rather specific to the web GUI application. The fields that can be useful to others are: test_id, which points into the tests table. timestamp with is the time at which the log entry was entered (with microsecond resolution). level which is one of the all-caps strings CRITICAL, ERROR, WARNING, NOTICE or INFO. message which is the message tag that describes the reason the log entry was made. arg0 to arg9, which are arguments to the message.

The message tags are specified in YAML files.

The messages table

The messages table contains a copy of a localy YAML file. It is used by the web GUI.

The delegation_history table

This table contains information about which nameservers has historically been authoritative for domains. Some tests use it, if it’s available.

The nameservers and nsp tables

The function of these tables have been obsoleted by the registrar system.

The domains table

A list of domains. More of a convenience thing than an actual part of the DNSCheck system.

Needed permissions

The various programs need different sets of permissions in the database in order to work correctly. It breaks down as follows:
o Everything that schedules tests needs INSERT, UPDATE and SELECT on sources and INSERT on queue.
o Everything that reads test results needs SELECT on sources, tests and results.
o Anything that runs tests needs SELECT on delegation_history (or no database access at all).
o dnscheck-dispatcher needs a lot: INSERT, SELECT, UPDATE and DELETE on queue, INSERT, SELECT and UPDATE on tests, INSERT on results plus everything listed above.
In practice, the easiest reasonably secure setup is to give the web GUI one user with very limited permissions (INSERT, UPDATE, SELECT on sources and INSERT on queue), and one user with full rights on all tables for the rest of the system. Indeed, since all of the programs use the database connection info given in the site_config.yaml file, you will need to point out a different such file to any programs you want to use different connection information.

    Files

The system uses a number of files.

config.yaml and site_config.yaml

These contain configuration information in YAML format. The config.yaml file is installed with DNSCheck, and should not be modified (if you do modify it, your changes will be overwritten when you upgrade DNSCheck). site_config.yaml overrides what is in config.yaml, and is where local changes (such as database access information and TSIG keys) should be entered. It will not be touched by the installation process.

One key that is usually not in either of these files but that can be added if needed is the disable key. It has two levels under it, being the name a test module (converted to lowercase) and the name of a method in that module. If the value for the method is true, that method will not be run. Note that no kind of sanity checking is done, and that some tests rely on each other, so disabling one may make others give false results. Be cautious and know what you’re doing. As an example, the following would disable all DNSSEC tests:



  disable:
      dnssec:
         test: 1



All test methods mentioned in the documentation for the respective modules are possible to disable this way.

policy.yaml and site_policy.yaml

policy.yaml specifies the severity of various problems. As distributed, it represents .SE’s view on best practices for DNS. If you want to change anything, do so in site_policy.yaml, or your changes will be overwritten on the next install.

locale/en.yaml and locale/sv.yaml

These files contain information on how to transform the raw test log entries into human-readable messages. If you want to do your own translation, you need to know that the format key is a sprintf() format string, and args the number of arguments after the format. If the latter number does not match the Perl code, a warning will be printed at runtime.

Search for    or go to Top of page |  Section 3 |  Main Index


perl v5.20.3 DNSCHECK::OVERVIEW (3) 2010-06-29

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.