|o||The sensor typically denotes the location where the flow data was collected; e.g., an organization that is instrumenting its border routers would create a sensor to represent each router. Each sensor has a unique name and numeric ID.|
|o||The flowtype represents information about how the flow was routed (e.g., as incoming or outgoing) or other information about the flow (e.g., web or non-web). The packing process categorizes each flow into a flowtype. Each flowtype has a unique name and numeric ID.|
For historic reasons, one rarely speaks of the flowtype of a SiLK Flow record, but instead refers to its class and type. Every flowtype maps to a unique class/type pair. The classes and types have names only; they do not have numeric IDs. Note that flowtype and type are different concepts despite the similarity of their names.
A class is generally used to represent topological features of the network with different collections of sensors, since every active sensor must belong by one or more classes. Every class must have a unique name.
A type is used to distinguish traffic within a single topological area based on some other dimension. For example, incoming and outgoing traffic is generally distinguished into different types. Web traffic is also frequently split into a separate type from normal traffic in order to partition the data better. The type names within a class must be unique, but multiple classes may have a type with the same name.
As stated above, each class/type pair maps to a unique flowtype.
The silk.conf file defines
|o||the mapping between sensor names and sensor IDs|
|o||the names of the available classes|
|o||the sensors that belong to each class|
|o||the names of the types in each class|
|o||the mapping from a class/type pair to a flowtype ID|
|o||the mapping between a flowtype name and a flowtype ID|
|o||the default class to use for rwfilter(1) and rwfglob(1) queries|
|o||for each class, the default types to use for rwfilter and rwfglob|
|o||the layout of the directory tree for the SiLK archive relative to the root directory|
|o||a default value for the --packing-logic switch to rwflowpack(8)|
If you cannot place the site configuration file in the data root directory, or the file in that location is incorrect, you can use the SILK_CONFIG_FILE environment variable to specify the location of your configuration file (including the file name). Many SiLK commands provide the --site-config-file switch which allows you to specify the name of the site configuration file on the command line.
By having the site configuration information outside of the SiLK tools, a single SiLK installation can be used to query different data stores (though each invocation of a command can only query one storage location).
Any additions or modifications to the silk.conf file will be seen by all SiLK applications upon their next invocation. There are some important things to keep in mind when modifying the silk.conf file:
|o||Once data has been collected for a sensor or a flowtype, the sensor or flowtype should never be removed or renumbered. SiLK Flow files store the sensor ID and flowtype ID as integers; removing or renumbering a sensor or flowtype breaks this mapping. In order to keep the mapping consistent, old sensor and flowtype definitions should remain indefinitely. Completely unused sensors or flowtypes may be removed, but the IDs of the remaining sensors and flowtypes must not be modified.|
|o||The path to the files in the SiLK data store often involve the sensor name, flowtype name, class name, and/or type name. If any of those names are changed, it will be necessary to rename all the previously packed data files that have the former name as part of their path.|
|o||If the SiLK installation at your site is distributed across multiple hosts (for example, if packing occurs on a machine separate from analysis), it is important to synchronize changes to the silk.conf files.|
|o||The packing logic plug-in file, packlogic-*.so (e.g., packlogic-twoway(3), packlogic-generic(3)), used by rwflowpack(8) checks for specific class names, type names, and flowtype names at start up, and it will exit with an error if the names it expects do not exist. In addition, it checks that the flowtype IDs it has match with those in the silk.conf file. When new flowtypes are added, the packlogic-*.so file will need to be updated if rwflowpack is to generate SiLK Flow records with the new flowtype.|
|o||When rwflowpack reads incoming flow records, those records are associated with a sensor name as determined by the sensor.conf(5) file. rwflowpack uses the silk.conf file to map the sensor name to the sensor ID, and it stores the sensor ID in the SiLK records it creates. Changes to the silk.conf and sensor.conf files may need to be coordinated.|
In the site configuration file, each line may be blank, or contain any amount of leading whitespace, which is ignored. At any location in a line, the character # indicates the beginning of a comment, which reaches until the end of the line. (If a literal # symbol is required in the argument of any command, it may be quoted as described below.) These comments are ignored.
Each non-empty line begins with a command name, followed by one or more arguments. Command names are a sequence of non-whitespace characters, not including the characters # or " (see below for valid commands). Arguments may either be textual atoms (any sequence of non-whitespace characters, non-#-or-" characters, including numerals and punctuation), or quoted strings. Quoted strings begin with the character " and end with the character ", and allow for C-style backslash escapes in between. The character # inside a quoted string does not begin a comment, and whitespace is allowed inside a quoted string.
For the commands supported by silk.conf and described below, unless a command explicitly states that it is used by particular applications, it should be considered used by all of the SiLK analysis tools and the packing tools flowcap(8), rwflowpack(8), and rwflowappend(8).
There are three contexts for commands: top-level, class block, and group block contexts. The class block and group block contexts are used to describe individual features of classes and groups, while top-level commands are used to describe the entire configuration, and to define sensors.
The valid commands for each context are described below.
class class-name The class command begins a new class block. It takes as an argument the name of the class being defined. Each class must have a unique name. A class block is closed with the end class command. See below for a list of commands valid inside class blocks.
The class name must begin with a letter, must not be longer than 32 characters, and may not contain whitespace characters or the character slash (/).
A site that does not use multiple classes should define a single class with a name like all or default.
To be valid, a configuration file must contain at least one class definition.
Example: class all
default-class class-name rwfilter(1) and rwfglob(1) will use a default class when the user does not specify an explicit --class. This command specifies that default class; the class must have been created prior to this command. If more than one default class is set, the last definition encountered is used.
Example: default-class all
group group-name Sensor groups are a convenient way of defining named groupings of sensors for inclusion in classes. They cannot currently be used in the SiLK command-line tools, but only in the configuration file. The group command takes as an argument the group to be defined, and begins a group block. A group block is closed using the end group command. See below for details on valid commands within group blocks.
Example: group test-sensors
include file-name The include command is used to include the contents of another file. This may be used to separate large configurations into logical units. (Note, however, that all sensors, classes, groups, and types must be declared before they may be referenced.)
Example: include silk-2.conf
packing-logic file-name The packing-logic command provides a default value for the --packing-logic switch on rwflowpack(8). The value is the path to a plug-in that rwflowpack loads; the plug-in provides functions that determine into which class and type a flow record will be categorized. The path specified here will be ignored when the --packing-logic switch is explicitly specified to rwflowpack or when SiLK has been configured with hard-coded packing logic.
Example: packing-logic packlogic-twoway.so
path-format format-string File and directory locations relative to the SILK_DATA_ROOTDIR may be defined using the path-format command. The path-format is used by rwflowpack and rwflowappend(8) when writing data to the data repository, and it is used by rwfilter and rwfglob when reading or listing files in the data repository. This command takes a format string specification that supports the following %-conversions:
%C The textual class name %F The textual flowtype name for this class/type pair (see also %f) %H The hour (24-hour clock) as a two-digit, zero-padded number %N The textual sensor name (see also %n) %T The textual type name %Y The year as a four-digit, zero-padded number %d The day of the month as a two-digit, zero-padded number %f The flowtype ID, as an unpadded number (see also %F) %m The month of the year as a two-digit, zero-padded number %n The sensor ID, as an unpadded number (see also %N) %x The default file name, which is equivalent to %F-%N_%Y%m%d.%H %% A literal % character
A % followed by any other character is an error.
For example, to place all spooled files directly in the data root directory, the path format %x could be used. To use two levels of hierarchy, the first containing the year and month, and the second containing the day and sensor name, like 2006-01/23-alpha/..., the format would be %Y-%m/%d-%N/%x.
If no path format is set by the configuration file, the default path format of %T/%Y/%m/%d/%x is used.
All path formats are currently required to end in /%x so that information may be extracted from the file name. This requirement may be lifted in the future.
sensor sensor-id sensor-name sensor sensor-id sensor-name sensor-description Individual sensor definitions are created with the sensor command. This command creates a new sensor with the given name and numeric ID. Sensor names must begin with a letter, must not be longer than 64 characters, and may not contain whitespace characters or the characters slash (/) or underscore (_).
The sensor line may may also provide an optional description of the sensor, enclosed in double quotes. The description can be used however your installation chooses to use it. The description may be viewed by specifying the describe-sensor field to rwsiteinfo(1). (When using sensor descriptions, the files version must be 2.)
It is an error to define two different sensors with the same sensor ID or the same sensor name.
NOTE: It is extremely important not to change the sensor-id or sensor-name for a given sensor once that sensor is in use. The sensor-id field is stored numerically in SiLK data files, and the sensor-name field is used to construct file names within the data root directory.
Example: sensor 0 S001
Example: sensor 0 S001 Primary connection to ISP
version version-number The version command declares that this configuration file conforms to a given version of the configuration file format. If the tools do not support this version of the configuration file, they will report an error. Currently, versions 1 and 2 of the format is defined, where version 2 indicates that sensor descriptions are present.
It is a recommended practice to include the version number at the beginning of all configuration files for compatibility with future versions.
Example: version 1
The commands inside a class block define the classs types, its default types, the sensors that belong to it, and the mapping from the class/type pair to the flowtype name and flowtype ID.
end class The end class command ends the definition of a class. Following an end class command, top-level commands are again accepted.
Example: end class
default-types type-name ... When no types are specified for the rwfilter or rwfglob commands, the default set of types for the selected class is used. Each of the types listed in this command is included as a default type of the class.
Example: default-types in inweb
sensors sensor-name-or-group-ref ... The sensors command is used to associate sensors with a class. In short, to declare that these sensors have data for this class. Sensors may be listed by name or numeric ID, and sensor groups may also be added by prefixing their names with an at (@) character. When you add a sensor group, it is the same as individually adding each sensor in that group to the class.
Example: sensors my-sensor-1 my-sensor-2 @my-group-1
type flowtype-id type-name [ flowtype-name ] The type command defines a type name within the current class, and it specifies the flowtype ID to use for that class/type pair. In addition, the type command may specify a flowtype name. The flowtype ID and flowtype name must be unique across the entire silk.conf file (and any included files). If a flowtype name is not specified, a default flowtype name is constructed by concatenating the name of the class and the name of the type. (e.g. the type in in the class all would have a flowtype name of allin.) Within a class, each type must have a unique name, but multiple classes may use the same type name. The type name and flowtype name must begin with a letter, must not be longer than 32 characters, and may not contain whitespace characters or the character slash (/).
As with sensors, it is important to be careful when renumbering flowtype IDs or renaming types or flowtypes because the numeric IDs are stored in data files, and the textual names are used as portions of file and path names.
Example: type 0 in
Example: type 1 out out
A group block is a convenience used to define a list of sensors.
end group Close a group block by using the end group command. Following this command, top-level commands are again accepted.
Example: end group
sensors sensor-name-or-group-ref ... Sensors are associated with a sensor group by means of the sensors command within a group block. Sensors may be listed by name or numeric ID, and sensor groups may also be added by prefixing their names with an at (@) character. When you add a sensor group, it is the same as individually adding each sensor to the group being defined.
Example: sensors my-sensor-1 my-sensor-2 @my-group-1
rwfilter(1), rwfglob(1), rwsiteinfo(1), sensor.conf(5), flowcap(8), rwflowpack(8),packlogic-twoway(3), packlogic-generic(3), rwflowappend(8), silk(7), SiLK Installation Handbook
|SiLK 22.214.171.124||SILK.CONF (5)||2016-04-05|