GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  RWBAGCAT (1)

.ds Aq ’

NAME

rwbagcat - Output a binary Bag as text.

CONTENTS

SYNOPSIS



  rwbagcat [ --network-structure[=STRUCTURE] | --bin-ips[=SCALE] ]
        [--print-statistics[=OUTFILE]]
        [--minkey=VALUE] [--maxkey=VALUE] [--mask-set=PATH]
        [--mincounter=VALUE] [--maxcounter=VALUE] [--zero-counts]
        [--output-path=OUTPUTFILE]
        [--key-format=FORMAT] [--integer-keys] [--zero-pad-ips]
        [--no-columns] [--column-separator=C]
        [--no-final-delimiter] [{--delimited | --delimited=C}]
        [--pager=PAGER_PROG] [BAGFILE...]

  rwbagcat --help

  rwbagcat --version



DESCRIPTION

rwbagcat reads a binary Bag as created by rwbag(1) or rwbagbuild(1), converts it to text, and outputs it to the standard output or the specified file. It can also print various statistics and summary information about the Bag.

rwbagcat reads the BAGFILEs specified on the command line; if no BAGFILE arguments are given, rwbagcat attempts to read the Bag from the standard input. BAGFILE may also explicitly be the keyword stdin or a hyphen (-) to allow rwbagcat to combine files and piped input. If any input does not contain a Bag, rwbagcat prints an error to the standard error and exits abnormally.

When multiple BAGFILEs are specified, each is handled individually; to process the combination of the BAGFILEs, invoke rwbagcat on the output from rwbagtool(1).

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.
--network-structure
--network-structure=STRUCTURE For each numeric value in STRUCTURE, group the IPs in the Bag into a netblock of that size and print the number of hosts, the sum of the counters, and, optionally, print the number of smaller, occupied netblocks that each larger netblock contains. When STRUCTURE begins with v6:, the IPs in the Bag are treated as IPv6 addresses, and any IPv4 addresses are mapped into the ::ffff:0:0/96 netblock. Otherwise, the IPs are treated as IPv4 addresses, and any IPv6 address outside the ::ffff:0:0/96 netblock is ignored. Aside from the initial v6: (or v4:, for consistency), STRUCTURE has one of following forms:
1. NETBLOCK_LIST/SUMMARY_LIST. Group IPs into the sizes specified in either NETBLOCK_LIST or SUMMARY_LIST. rwbagcat prints a row for each occupied netblock specified in NETBLOCK_LIST, where the row lists the base IP of the netblock, the sum of the counters for that netblock, the number of hosts, and the number of smaller, occupied netblocks having a size that appears in either NETBLOCK_LIST or SUMMARY_LIST. (The values in SUMMARY_LIST are only summarized; they are not printed.)
2. NETBLOCK_LIST/. Similar to the first form, except all occupied netblocks are printed, and there are no netblocks that are only summarized.
3. NETBLOCK_LISTS. When the character S appears anywhere in the NETBLOCK_LIST, rwbagcat provides a default value for the SUMMARY_LIST. That default is 8,16,24,27 for IPv4, and 48,64 for IPv6.
4. NETBLOCK_LIST. When neither S nor / appear in STRUCTURE, the output does not include the number of smaller, occupied netblocks.
5. Empty. When STRUCTURE is empty or only contains v6: or v4:, the NETBLOCK_LIST prints a single row for the total network (the /0 netblock) giving the number of hosts, the sum of the counters, and the number of smaller, occupied netblocks using the same default list specified in form 3.

NETBLOCK_LIST and SUMMARY_LIST contain a comma separated list of numbers between 0 (the total network) and the size for an individual host (32 for IPv4 or 128 for IPv6). The characters T and H may be used as aliases for 0 and the host netblock, respectively. In addition, when parsing the lists as IPv4 netblocks, the characters A, B, C, and X are supported as aliases for 8, 16, 24, and 27, respectively. A comma is not required between adjacent letters. The --network-structure switch disables printing of the IPs in the Bag file; specify the H argument to the switch to print each individual IP address and its counter.

--bin-ips
--bin-ips=SCALE Invert the bag and count the total number of unique IP addresses for a given value of the volume bin. For example, turn a Bag {sip:flow} into {flow:count(sip)}. SCALE is a string containing the value linear, binary, or decimal.
o The default behavior is linear: Each distinct counter gets its own bin. Any counter in the input Bag file that is larger than the maximum possible key will be attributed to the maximum key; to prevent this, specify --maxcounter=4294967295.
o binary creates a bag of {log2(flow):count(sip)}. Bin n contains counts in the range [ 2^n, 2^(n+1) ).
o decimal creates one hundred bins for each counter in the range [1,100), and one hundred bins for each counter in the range [100,1000), each counter in the range [1000,10000), etc. Counters are logarithmically distributed among the bins.
--print-statistics
--print-statistics=OUTFILE Print out breakdown of the network hosts seen, and print out general statistics about the keys and counters.
o count of unique keys
o sum of all the counters
o minimum key
o maximum key
o minimum counter
o maximum counter
o mean of counters
o variance of counters
o standard deviation of counters
o skew of counters
o kurtosis of counters
o count of nodes allocated
o total bytes allocated for nodes
o count of leaves allocated
o total bytes allocated for leaves
o density of the data

OUTFILE is a filename, named pipe, the values stdout or - to print to the standard output, or the value stderr to print to the standard error. Defaults to printing to the standard output.

--minkey=VALUE Output records whose key value is at least VALUE. VALUE may be an IP address or an integer in the range 0 to 4294967295 inclusive. The default is to print all records with a non-zero counter.
--maxkey=VALUE Output records whose key value is not more than VALUE. VALUE may be an IP address or an integer in the range 0 to 4294967295 inclusive. The default is to print all records with a non-zero counter.
--mask-set=PATH Output records whose key appears in the binary IPset read from the file PATH. (To build an IPset, use rwset(1) or rwsetbuild(1).) When used with --minkey and/or --maxkey, output records whose key is in the IPset and is also within when the specified range.
--mincounter=VALUE Output records whose counter value is at least VALUE. VALUE is an integer in the range 1 to 18446744073709551615. The default is to print all records with a non-zero counter; use --zero-counts to show records whose counter is 0.
--maxcounter=VALUE Output records whose counter value is not more than VALUE. VALUE is an integer in the range 1 to 18446744073709551615, with the default being the maximum counter value.
--zero-counts Print keys whose counter is zero. Normally, keys with a counter of zero are suppressed since all keys have a default counter of zero. In order to use this flag, either --mask-set or both --minkey and --maxkey must be specified. When this switch is specified, any counter limit explicitly set by the --maxcounter switch will still be applied.
--output-path=OUTPUTFILE Redirect output of the --network-structure or --bin-ips options to OUTPUTFILE. OUTPUTFILE is a filename, named pipe, or the values stdout or - to print to the standard output.
--key-format=FORMAT Specify the format to use when printing the keys. When this switch is not specified, a Bag whose keys are known not to be IP addresses are printed as decimal numbers, and the keys for all other Bags are printed as IP addresses in the canonical format. The FORMAT is one of:
canonical Print keys as IP addresses in the canonical format: dotted quad for IPv4 (127.0.0.1) and hexadectet for IPv6 (2001:db8::1). Note that IPv6 addresses in ::ffff:0:0/96 and some IPv6 addresses in ::/96 will be printed as a mixture of IPv6 and IPv4.
zero-padded Print keys as IP addresses in their canonical form, but add zeros to the output so it fully fills the width of column. The addresses 127.0.0.1 and 2001:db8::1 are printed as 127.000.000.001 and 2001:0db8:0000:0000:0000:0000:0000:0001, respectively.
decimal Print keys as integers in decimal format. The addresses 127.0.0.1 and 2001:db8::1 are printed as 2130706433 and 42540766411282592856903984951653826561, respectively.
hexadecimal Print keys as integers in hexadecimal format. The addresses 127.0.0.1 and 2001:db8::1 are printed as 7f000001 and 20010db8000000000000000000000001, respectively.
force-ipv6 Print all keys as IP addresses in the canonical form for IPv6 without using any IPv4 notation. Any integer key or IPv4 address is mapped into the ::ffff:0:0/96 netblock. The addresses 127.0.0.1 and 2001:db8::1 are printed as ::ffff:7f00:1 and 2001:db8::1, respectively.
--integer-keys This switch is equivalent to --key-format=decimal, it is deprecated as of SiLK 3.7.0, and it will be removed in the SiLK 4.0 release.
--zero-pad-ips This switch is equivalent to --key-format=zero-padded, it is deprecated as of SiLK 3.7.0, and it will be removed in the SiLK 4.0 release.
--no-columns Disable fixed-width columnar output.
--column-separator=C Use specified character between columns and after the final column. When this switch is not specified, the default of ’|’ is used.
--no-final-delimiter Do not print the column separator after the final column. Normally a delimiter is printed. When the network summary is requested (--network-structure=S), the separator is always printed before the summary column and never that column.
--delimited
--delimited=C Run as if --no-columns --no-final-delimiter --column-sep=C had been specified. That is, disable fixed-width columnar output; if character C is provided, it is used as the delimiter between columns instead of the default ’|’.
--pager=PAGER_PROG When output is to a terminal, invoke the program PAGER_PROG to view the output one screen full at a time. This switch overrides the SILK_PAGER environment variable, which in turn overrides the PAGER variable. If the value of the pager is determined to be the empty string, no paging will be performed and all output will be printed to the terminal.
--help Print the available options and exit.
--version Print the version number and information about how SiLK was configured, then exit the application.

EXAMPLES

In the following examples, the dollar sign ($) represents the shell prompt. The text after the dollar sign represents the command line.

To print the bag:



 $ rwbagcat mybag.bag
      172.23.1.1|              5|
      172.23.1.2|            231|
      172.23.1.3|              9|
      172.23.1.4|             19|
   192.168.0.100|              1|
   192.168.0.101|              1|
   192.168.0.160|             15|
  192.168.20.161|              1|
  192.168.20.162|              5|
  192.168.20.163|              5|



To print it with full network:



 $ rwbagcat --network-structure=TABCHX mybag.bag
           172.23.1.1      |              5|
           172.23.1.2      |            231|
           172.23.1.3      |              9|
           172.23.1.4      |             19|
         172.23.1.0/27     |            264|
       172.23.1.0/24       |            264|
     172.23.0.0/16         |            264|
   172.0.0.0/8             |            264|
           192.168.0.100   |              1|
           192.168.0.101   |              1|
         192.168.0.96/27   |              2|
           192.168.0.160   |             15|
         192.168.0.160/27  |             15|
       192.168.0.0/24      |             17|
           192.168.20.161  |              1|
           192.168.20.162  |              5|
           192.168.20.163  |              5|
         192.168.20.160/27 |             11|
       192.168.20.0/24     |             11|
     192.168.0.0/16        |             28|
   192.0.0.0/8             |             28|
 TOTAL                     |            292|



Or an abbreviated network structure by class A and C only, including summary information:



 $ rwbagcat --network-structure=ACS mybag.bag
     172.23.1.0/24     |            264| 4 hosts in 1 /27
 172.0.0.0/8           |            264| 4 hosts in 1 /16, 1 /24, and 1 /27
     192.168.0.0/24    |             17| 3 hosts in 2 /27s
     192.168.20.0/24   |             11| 3 hosts in 1 /27
 192.0.0.0/8           |             28| 6 hosts in 1 /16, 2 /24s, and 3 /27s



To bin by number of unique IP addresses by volume:



 $ rwbagcat --bin-ips mybag.bag
               1|              3|
               5|              3|
               9|              1|
              15|              1|
              19|              1|
             231|              1|



This means there were 3 source hosts in the bag that had a single flow; 3 hosts that had 5 flows; and one host each that had 9, 15, 19, and 231 flows.

For a log2 breakdown of the counts:



 $ rwbagcat --bin-ips=binary mybag.bag
    2^0 to 2^1-1|              3|
    2^2 to 2^3-1|              3|
    2^3 to 2^4-1|              2|
    2^4 to 2^5-1|              1|
    2^7 to 2^8-1|              1|



Statistics:



 $ rwbagcat --stats mybag.bag

 Statistics
               keys:  10
    sum of counters:  292
        minimum key:  172.23.1.1
        maximum key:  192.168.20.163
      minimum count:  1
      maximum count:  231
               mean:  29.2
           variance:  5064
 standard deviation:  71.16
               skew:  2.246
           kurtosis:  8.1

 $ rwbagcat --tree-stats mybag.bag
    nodes allocated:  5 (10240 bytes)
   leaves allocated:  4 (1024 bytes)
      keys inserted:  10 (10 unique)
    counter density:  7.81%



ENVIRONMENT

SILK_CLOBBER The SiLK tools normally refuse to overwrite existing files. Setting SILK_CLOBBER to a non-empty value removes this restriction.
SILK_PAGER When set to a non-empty string, rwbagcat automatically invokes this program to display its output a screen at a time. If set to an empty string, rwbagcat does not automatically page its output.
PAGER When set and SILK_PAGER is not set, rwbagcat automatically invokes this program to display its output a screen at a time.

SEE ALSO

rwbag(1), rwbagbuild(1), rwbagtool(1), rwset(1), rwsetbuild(1), silk(7)
Search for    or go to Top of page |  Section 1 |  Main Index


SiLK 3.11.0.1 RWBAGCAT (1) 2016-04-05

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.