NAME

nfcapd — flow collector for netflow version v1, v5/v7 v9 and ipfix

SYNOPSIS

DESCRIPTION

nfcapd reads netflow data from the network and stores the records into binary formatted files. It accepts netflow v1, v5/v7, v9 and ipfix transparently. It is mostly compatible with a lot of other flow implementations such as cflow, jflow, pflow and accepts a wide range of exporters including CISCO Flexible Netflow (FNF), ASA firewalls and NAT devices for event logging. It has also support for a wide range of different vendors and their implementation of netflow, such as Juniper, VMware, PaloAlto devices and yaf. Sflow is a different technology. nfcapd supports a large number of netflow v9 and ipfix elements according to the IANA assignments.

If you want to collect sflow data, please have a look at sfcapd which is also part of the nfdump tools.

nfcapd also accepts pre-processed records from its companion collector nfpcapd. nfcapd safes the flows in an output file, which is automatically rotated at a given interval - typically every 5min. These rotated output files are stored in the flowdir directory and are organized by timestamps. The output files are named according to the time interval in the following format: nfcapd.YYYYMMddhhmm e.g. nfcapd.202207110845 which contains flow data from July 11th 2022 08:45 onwards. If the rotation interval is set to a time, smaller then 60s, the naming extends to seconds e.g. nfcapd.20220711084510.

nfcapd can run in auto-expire mode -e , which automatically expires old flow files, at the end of every rotation interval. nfexpire(1) explains in more details how to setup flow expiration.

nfcapd can run any given command -x or shell script at the end of each rotation interval.

nfcapd can send universal flow metric information about the collected flow data (flow summary) to a UNIX socket. Programs, such as nfinflux or nfexporter may be used to send the metric information to an InfluxDB or to a Prometheus monitoring system.

The options are as follows:

-w flowdir

Set the flow directory to store the output files. If a sub hierarchy is specified with -S the final directory is concatenated to flowdir/subdir.

-C config

Reads additional configuration parameters from config file. nfcapd tries to read the config file from the install default path $prefix/etc/ which may be overwritten by the environment variable NFCONF , which again is overwritten by this option -C. If -C none is specified, then no config file is read, even if found in the search path.

-p portnum

Set the port number to listen. Default port is 9995

-d interface

Reads flow data from an erspan encoded datalink. All traffic sent to this interface is interpreted as flow data stream.

-b bindhost

Specifies the hostname/IPv4/IPv6 address to bind for listening. This can be an IP address or a hostname, resolving to a local IP address.

-4

Forces nfcapd to listen on IPv4 addresses only. Can be used together with -b if a hostname has IPv4 and IPv6 addresses.

-6

Forces nfcapd to listen on IPv6 addresses only. Can be used together with -b if a hostname has IPv4 and IPv6 addresses.

-J mcastgroup

Join the specified IPv4 or IPv6 multicast group for listening.

-R host[/port]

Enables the packet repeater. All incoming packets are sent additionally to another host and port. host is either a valid IPv4/IPv6 address, or a symbolic hostname, which resolves to a valid IP address. port may be omitted and defaults to 9995. Note: As IPv4/IPv6 are accepted the host/port separator is '/'. Up to 8 additional repeaters my be defined. Use this method to daisy chain collectors.

-A

Sets source address spoofing mode for the repeater. The source address of the repeated packages is set to the original IP address. This needs nfcapd to be started with root privileges. Please note, that source spoofing may be blocked by firewalls or routers in your network.

-I ident

Sets ident as identification string for the current source. This string is written into the output file to identify the source. Default is 'none'. If you have multiple sources, see option -n below.

-n ident,IP,flowdir

Configures a netflow source identified by the string ident, IP flowdir If you have multiple sources per collector, add multiple -n options. All exporters send the flows to the same port -p. Do not mix single source configuration -I with multiple -n options.

-M flowdir

Set the flow directory for dynamic allocated exporters. New exporters are dynamically added when sending data. All exporters send netflow data to the same port and IP. For each dynamically added source, a new sub directory is created under flowdir with the name of the IP address of the exporter. All '.' and ':" in IP addresses are replaced be '-'.

-D

Set daemon mode: fork to background and detach from terminal. nfcapd terminates on signal TERM, INT or HUP.

-P pidfile

Writes the running process ID into pidfilw. Use this option to integrate nfcapd in start/stop files.

-u userid

Drop privileges of running process to user userid. nfcapd needs to be started as user root.

-g groupid

Drop privileges of running process to group groupid. nfcapd needs to be started as user root.

-B bufflen

Sets the network socket input buffer to bufflen bytes. For high volume traffic it is recommended to raise this value to typically > 100k, otherwise you risk to lose packets. The default is OS (and kernel) dependent.

-S num

Adds an additional directory sub hierarchy to store the data files. The default is 0, no sub hierarchy, which means all files go directly into flowdir. The flowdir is concatenated with the specified sub hierarchy format to create the final data directory. The following hierarchies are defined:

• 0 default no hierarchy levels
• 1 %Y/%m/%d year/month/day
• 2 %Y/%m/%d/%H year/month/day/hour
• 3 %Y/%W/%u year/week_of_year/day_of_week
• 4 %Y/%W/%u/%H year/week_of_year/day_of_week/hour
• 5 %Y/%j year/day-of-year
• 6 %Y/%j/%H year/day-of-year/hour
• 7 %Y-%m-%d year-month-day
• 8 %Y-%m-%d/%H year-month-day/hour

-t interval

Sets the time interval in seconds to rotate files. The default value is 300s ( 5min ). The smallest available interval is 2s.

-s rate

Apply sampling rate rate to all netflow records, unless the sampling rate is announced by the exporting device. In that case the announced sampling rate is applied. If rate is negative, this will hard overwrite any device specific announced sampling rates. The sampling rate is used to multiply the number of packets and bytes in a record. Please note, this may vary from other volume counters such as SNMP etc.

-z=lzo

Compress flow files with LZO1X-1 compression. Fastest compression.

-z=bz2

Compress flow files with bz2 compression. Slow but most efficient. It is not recommended to use bz2 in a real time capturing.

-z=lz4[:level]

Compress flow files with LZ4 compression. Fast and efficient. Optional level should be between 1..10 Changing the level results in smaller files but uses up more time to compress. Levels > 5 may need more workers. See -W.

-z=zstd[:level]

Compress flow files with ZSTD compression. Fast and efficient. Optional level should be between 1..10 Changing the level results in smaller files but uses up more time to compress. Levels > 5 may need more workers. See -W.

-W num

Sets the number of workers to compress flows. Defaults to 4. Must not be greater than the number of cores online. Useful for higher levels of compression for lz4 or zstd and large amount of flows per second.

-e

Sets auto-expire mode. At the end of every rotate interval -t nfcapd runs an expire cycle to delete files according to max lifetime and max filesize as defined by nfexpire(1)

-x command

At the end of every -t interval and after the file rotate has completed, nfcapd runs the command command. The string for command may contain the following place holders, which are expanded before running:

• %f File name of new data file including any sub hierarchy.
• %d Top flowdir. The full path of the new file is: %d/%f
• %t Time slot string in ISO format e.g. 201107110845.
• %u Time slot string in UNIX time format.
• %i Identification string ident string supplied by -I

-X extensionList

extensionList is a ',' separated list of extensions to be stored by nfcapd. The numbers correspond to the extension list in nfxV3.h. By default extensions are added dynamically to store all data sent by the exporter. If extensionList is given, only those elements matching the extension are processed and stored. Usually this option is not needed, unless for specific requirements.

-m metricpath

Enables the flow metric exporter. Flow metric information is sent to the UNIX socket metricpath at the rate specified by -i This option may by used to export flow metric information to other systems such as InfluxDB or Prometheus. Please note: The flow metric does not include the full record. Only the flow statistics is sent.

-i metricrate

Sets the interval for the flow metric exporter. This interval may be different from the file rotation interval t and is therefore independent from file rotation.

-v

Increase verbose level by 1. The verbose level may be increased for debugging purpose up to 3.

-E

Equal to -v -v -v. Print netflow records in block format to stdout. Please note, that not all elements are printed, which are available in the flow record. To inspect all elements, use nfdump -o raw This option is for debugging purpose only, to verify if incoming netflow data is processed correctly.

-V

Print nfcapd version and exit.

-h

Print help text on stdout with all options and exit.

RETURN VALUES

nfcapd returns 0 on success and 255 if initialization failed.

SEE ALSO

https://www.iana.org/assignments/ipfix/ipfix.xhtml

https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html

nfdump(1) nfpcapd(1) sfcapd(1)

BUGS

No software without bugs! Please report any bugs back to me.