GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  RWPACKCHECKER (8)

.ds Aq ’

NAME

rwpackchecker - Find unusual patterns that may indicate a corrupt file

CONTENTS

SYNOPSIS



  rwpackchecker [--value=TEST=VALUE] [--allowable-count=TEST=ALLOWED]
        [--print-all]
        {[--xargs] | [--xargs=FILENAME] | [FILE [FILE ...]]}

  rwpackchecker --help

  rwpackchecker --version



DESCRIPTION

rwpackchecker reads SiLK Flow records and checks for unusual patterns that may indicate the file has been corrupted.

rwpackchecker has a default series of tests it runs on every flow record in an input file. Each default test has two numbers associated with it: a value threshold and an allowed count threshold. A test compares a value on the flow record to the value threshold, and if the value violates the threshold, a counter for that test is incremented. In addition, if the flow record violates the value threshold for any test, a global counter is incremented to denote a suspect record.

Once rwpackchecker finishes processing a file, it determines whether the file appears to be valid. A file is considered valid if either
o the global counter of suspect records is 0, or
o no test has a counter that exceeds the test’s allowed count threshold
If rwpackchecker determines that all files are valid, it does not print any output by default. If rwpackchecker does find an invalid file, it will print the name of the input file, the global number of suspect records it found, and information for those tests where the counter exceeds the allowed count threshold.

As an example, if there are 10 tests and the count threshold for each is 10, it is possible for the global suspect counter to be 90 and for rwpackchecker to consider the file valid.

To force rwpackchecker to print the results for all tests and for all input files, specify the --print-all switch.

Some of the tests that run by default include checking the number of packets, the bytes per second ratio, the bytes per packet ratio, and the bytes per packet ratio for a particular protocol (TCP, UDP, and ICMP).

The --value and --allowable-count switches modify the value threshold and allowed count threshold for a test, respectively. The argument to the switch is the test name and the threshold, separated by a equals sign (=). Repeat the switches to set multiple thresholds. For example, to change the value thresholds for the max-bytes test to 20000 and for the max-packets test to 15000, specify the following:



 rwpackchecker --value=max-bytes=20000 --value max-packets=15000 ...



To get the most value from rwpackchecker, one should customize it for the particular site where it is being used, since the default value for a threshold may or may not be unusual for a particular installation. For example, a router that has Ethernet connections should have no more than 1500 bytes per packet, since that is the Ethernet MTU; however, the default value for that ratio is 16384. In some cases the default value is the largest value that a SiLK IPv4 hourly repository file can store, making it impossible for a record to violate the threshold.

rwpackchecker supports additional tests which are not run by default. Representative tests include checking whether an IP is (not) in an IPset or whether a port is (not) in an integer list. To run an additional test, specify the name of the test using the --value switch and provide an argument for the test.

To see all of the tests that rwpackchecker supports as well as the value threshold and allowed count threshold for each test, see the OPTIONS section below, or run rwpackchecker with the --help switch.

rwpackchecker reads SiLK Flow records from the files named on the command line or from the standard input when no file names are specified and --xargs is not present. To read the standard input in addition to the named files, use - or stdin as a file name. If an input file name ends in .gz, the file will be uncompressed as it is read. When the --xargs switch is provided, rwpackchecker will read the names of the files to process from the named text file, or from the standard input if no file name argument is provided to the switch. The input to --xargs must contain one file name per line.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.
--value=TEST=VALUE Set the value of TEST to the specified VALUE; separate the test name from value by =. The available TESTs are given below; the test name can be shortened to the shortest unique prefix. The form of VALUE depends on the type of TEST:
o If TEST expects a minimum or maximum, VALUE should be a number.
o If TEST expects a list of IPs, VALUE should the name of a file containing an IPset (see rwsetbuild(1)).
o If TEST expects a list of numbers (for example, ports or protocols), VALUE should contain a comma separated list of integers and integer-ranges where a range is two integers separated by a hyphen (-).

Repeat this switch for each value that you wish to set.

--allowable-count=TEST=ALLOWED Allow the named TEST to be violated ALLOWED of times before treating it as unusual. ALLOWED is an integer value. Separate the test name from the allowed count by =. Repeat this switch for each allowable count you wish to set.
--print-all Print the result of all tests for all input files. Normally only tests that are deemed unusual are printed.
--xargs
--xargs=FILENAME Causes rwpackchecker to read file names from FILENAME or from the standard input if FILENAME is not provided. The input should have one file name per line. rwpackchecker will open each file in turn and read records from it, as if the files had been listed on the command line.
--help Print the available options and exit.
--version Print the version number and information about how SiLK was configured, then exit the application.
The following tests are always run:
min-bpp-ratio=NUMBER Byte-per-packet ratio is less than NUMBER. Default value: 1. Allowed count: 0.
max-bpp-ratio=NUMBER Byte-per-packet ratio is greater than NUMBER. Default value: 16384. Allowed count: 0.
min-bps-ratio=NUMBER Byte-per-second ratio is less than NUMBER. Default value: 0. Allowed count: 0.
max-bps-ratio=NUMBER Byte-per-second ratio is greater than NUMBER. Default value: 4294967295. Allowed count: 0.
min-packets=NUMBER Packet count is less than NUMBER. Default value: 1. Allowed count: 0.
max-packets=NUMBER Packet count is greater than NUMBER. Default value: 67108864. Allowed count: 0.
min-bytes=NUMBER Byte count is less than NUMBER. Default value: 1. Allowed count: 0.
max-bytes=NUMBER Byte count is greater than NUMBER. Default value: 4294967295. Allowed count: 0.
min-tcp-bpp-ratio=NUMBER TCP byte-per-packet ratio is less than NUMBER. Default value: 1. Allowed count: 0.
max-tcp-bpp-ratio=NUMBER TCP byte-per-packet ratio is greater than NUMBER. Default value: 16384. Allowed count: 0.
min-udp-bpp-ratio=NUMBER UDP byte-per-packet ratio is less than NUMBER. Default value: 1. Allowed count: 0.
max-udp-bpp-ratio=NUMBER UDP byte-per-packet ratio is greater than NUMBER. Default value: 16384. Allowed count: 0.
min-icmp-bpp-ratio=NUMBER ICMP byte-per-packet ratio is less than NUMBER. Default value: 1. Allowed count: 0.
max-icmp-bpp-ratio=NUMBER ICMP byte-per-packet ratio is greater than NUMBER. Default value: 16384. Allowed count: 0.
The following tests are only run when the --value switch is used to specify a value for the test.
match-protocol=LIST Protocol is present in LIST. No default. Allowed count: 0.
nomatch-protocol=LIST Protocol is not present in LIST. No default. Allowed count: 0.
match-flags=LIST TCP Flag Combination is present in LIST. No default. Allowed count: 0.
nomatch-flags=LIST TCP Flag Combination is not present in LIST. No default. Allowed count: 0.
match-sip=IPSET_FILE Source IP is present in IPSET_FILE. No default. Allowed count: 0.
nomatch-sip=IPSET_FILE Source IP is not present in IPSET_FILE. No default. Allowed count: 0.
match-dip=IPSET_FILE Destination IP is present in IPSET_FILE. No default. Allowed count: 0.
nomatch-dip=IPSET_FILE Destination IP is not present in IPSET_FILE. No default. Allowed count: 0.
match-sport=LIST Source Port is present in LIST. No default. Allowed count: 0.
nomatch-sport=LIST Source Port is not present in LIST. No default. Allowed count: 0.
match-dport=LIST Destination Port is present in LIST. No default. Allowed count: 0.
nomatch-dport=LIST Destination Port is not present in LIST. No default. Allowed count: 0.
match-nhip=IPSET_FILE Next Hop IP is present in IPSET_FILE. No default. Allowed count: 0.
nomatch-nhip=IPSET_FILE Next Hop IP is not present in IPSET_FILE. No default. Allowed count: 0.
match-input=LIST SNMP Input is present in LIST. No default. Allowed count: 0.
nomatch-input=LIST SNMP Input is not present in LIST. No default. Allowed count: 0.
match-output=LIST SNMP Output is present in LIST. No default. Allowed count: 0.
nomatch-output=LIST SNMP Output is not present in LIST. No default. Allowed count: 0.

EXAMPLES

In these examples, the dollar sign ($) represents the shell prompt and a backslash (\) is used to continue a line for better readability. The examples do not use the optional = between the --value switch and the switch’s argument for better readability.

Given the SiLK Flow file data.rw where the number of flows with various byte and packet counts are described by this table:



 Number of flows    bytes <= 2000000    bytes > 2000000     TOTAL
 packets <= 500         379303                308          379611
 packets >  500         119586               2679          122265
 TOTAL                  498889               2987          501876



Running rwpackchecker:



 $ rwpackchecker --value max-bytes=2000000            \
        --value max-packets=500  data.rw
 data.rw:
              122573/501876 flows are bad or unusual
              122265 flows where Packet Count > 500
                2987 flows where Byte Count > 2000000



The counts for the individual tests are greater than the overall total since 2679 records fall into both categories.

To see the effect of the --allowable-count switch:



 $ rwpackchecker --value max-packets=500                              \
        --value max-bytes=2000000 --allowable max-bytes=3000 data.rw
 data.rw:
              122573/501876 flows are bad or unusual
              122265 flows where Packet Count > 500

 $ rwpackchecker --value max-bytes=2000000                            \
        --value max-packets=500 --allowable max-packets=150000 data.rw
 data.rw:
              122573/501876 flows are bad or unusual
                2987 flows where Byte Count > 2000000



In each case the total number of unusual flows did not change; the violation of the other limit is still noted, even when the test is not printed since the test’s allowed count threshold was not reached.

When the allowed count thresholds for none of the tests are reached, rwpackchecker produces no output by default:



 $ rwpackchecker --value max-bytes=2000000 --allowable max-bytes=3000 \
        --value max-packets=500 --allowable max-packets=150000 data.rw
 $



Specify the --print-all switch to print the results:



 $ rwpackchecker --value max-bytes=2000000 --allowable max-bytes=3000 \
        --value max-packets=500 --allowable max-packets=15000         \
        --print-all data.rw
 data.rw:
              122573/501876 flows are bad or unusual
                   0 flows where BPP Calculation is incorrect
                   0 flows where Elapsed Time > 4096
                   0 flows where Byte/Packet Ratio < 1
                   0 flows where Byte/Packet Ratio > 16384
                   0 flows where Byte/Second Ratio < 0
                   0 flows where Byte/Second Ratio > 4294967295
                   0 flows where Packet Count < 1
              122265 flows where Packet Count > 500
                   0 flows where Byte Count < 1
                2987 flows where Byte Count > 2000000
                   0 flows where TCP Byte/Packet Ratio < 1
                   0 flows where TCP Byte/Packet Ratio > 16384
                   0 flows where UDP Byte/Packet Ratio < 1
                   0 flows where UDP Byte/Packet Ratio > 16384
                   0 flows where ICMP Byte/Packet Ratio < 1
                   0 flows where ICMP Byte/Packet Ratio > 16384



SEE ALSO

rwflowpack(8), rwsetbuild(1), silk(7)
Search for    or go to Top of page |  Section 8 |  Main Index


SiLK 3.11.0.1 RWPACKCHECKER (8) 2016-04-05

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.