NAME

dnstable_convert - convert passive DNS NMSG data to dnstable MTBL format

SYNOPSIS

dnstable_convert [-DprS] [-b DNS_BSIZE[,DNSSEC_BSIZE]] [-c TYPE] [-l LEVEL] [-m #] [-s NAME] [-t COUNT] NMSG-FILE DB-DNS-FILE DB-DNSSEC-FILE

DESCRIPTION

Converts passive DNS data from NMSG format to MTBL format. The input NMSG data in NMSG-FILE must be encoded using the SIE/dnsdedupe message schema, and the output data will be written to two separate MTBL files: the DB-DNS-FILE containing "plain" DNS records, and the DB-DNSSEC-FILE, containing DNSSEC-related records.

If NMSG-FILE has the value - then stdin will be read from.

Specifically, all resource records of type DS, RRSIG, NSEC, DNSKEY, NSEC3, NSEC3PARAM, DLV, CDS, CDNSKEY, and TA are stored in the DB-DNSSEC-FILE, and resource records of all other types are stored in the DB-DNS-FILE. If the -D flag is provided then CDS, CDNSKEY, and TA will also be stored in the DB-DNS-FILE file for transition compability.

dnstable_convert will create both the DB-DNS-FILE and DB-DNSSEC-FILE, and will abort if either exists. If the input file contains no DNS-related records, DB-DNS-FILE will be removed. Conversely, if the input file contains no DNSSEC-related records, DB-DNSSEC-FILE will be removed. If the -p flag is provided then MTBL files without records are not removed.

The output files produced by dnstable_convert are suitable for use with other dnstable tools like dnstable_dump(1) and dnstable_merge(1), or with the dnstable_reader(3) API. Since they are in MTBL format, they can also be read with the mtbl_reader(3) API.

By default, dnstable_convert will use /var/tmp/ to store temporary mtbl files. The optional VARTMPDIR environment variable can be set to override the location. Specifying a filesystem for RAM-based storage or specifying the same filesystem as the source data file might be useful.

OPTIONS

-D

This flag enables a transitional compatibility mode described above.

-b DNS_BSIZE[,DNSSEC_BSIZE]

Produce the DNS MTBL output file using a data block size of DNS_BSIZE bytes, optionally producing the DNSSEC MTBL output file using a data block size of DNSSEC_BSIZE.

-c TYPE

Use TYPE compression (Default: zlib).

-l LEVEL

Use numeric LEVEL of compression. Default varies based on TYPE.

-m #

This flag takes a positive integer value. Specifies the maximum amount of memory to use for in-memory sorting, in megabytes. Defaults to 2048 megabytes (e.g. 2 gigabytes). See the mtbl_sorter(3) API.

-p

Preserve empty DNS/DNSSEC MTBL files.

-r

Emit RDATA and RDATA_RNAME_REV dnstable entries for SOA rname field.

-s NAME

Source metadata to include in output. Required if input is stdin, otherwise defaults to input file name.

-S

Include nmsg source information in output.

-t COUNT

Use an MTBL thread pool of COUNT worker threads for sorting and writing (default is zero).