hpct-importer
—
HiPerConTracer Trace Data Importer
hpct-importer |
[-C importer_configuration
| --importer-config
importer_configuration] [-D
database_configuration |
--database-config
database_configuration] [-L
level | --loglevel
level] [-Z
on|off | --logcolor
on|off] [-O
file | --logfile
file] [-q |
--quiet ] [-v |
--verbose ] [-X
mode | --import-mode
mode] [-Y
depth | --import-max-depth
depth] [-I
path | --import-file-path
path] [-B
path | --bad-file-path
path] [-G
path | --good-file-path
path] [-F
filter_regexp |
--import-file-path-filter
filter_regexp] [-Q |
--quit-when-idle ]
[--ping-workers number]
[--ping-files number]
[--traceroute-workers
number] [--traceroute-files
number] |
hpct-importer |
[-h | --help ] |
hpct-importer |
[--version ] |
hpct-importer
imports HiPerConTracer
results into an SQL or NoSQL database. Currently, the following database
backends are supported: MySQL/MariaDB (SQL), PostgreSQL (SQL), MongoDB
(NoSQL).
Results files imported successfully can be moved into a "good"
directory, deleted or kept as-in (for debugging only!). Results files which
failed to import a re moved into a "bad" directory.
The input of multiple input files can be combined into a single database
transaction, to improve performance on short files. In case of an import
error, each file is imported sequentially to identify faulty ones.
The following argument may be provided:
-L
level |
--loglevel
level
- Sets the minimum logging level to the provided value: 0=trace, 1=debug,
2=info, 3=warning, 4=error, 5=fatal.
-Z
on|off |
--logcolor
on|off
- Enables (on) or disables (off) the usage of ANSI color escape sequences to
colorise the log output according to its level.
-O
file |
--logfile
file
- Sets a file name for appending the log output to. If a file is set, no log
output is made to the console. Otherwise, all log output will be written
to the console.
-q
|
--quiet
- Sets the minimum logging level to 3 (warning).
-v
|
--verbose
- Sets the minimum logging level to 0 (trace).
-C
importer_configuration |
--importer-config
importer_configuration
- Sets the name of the importer configuration file. This option must be
provided!
See /usr/share/hipercontracer/hipercontracer-importer.conf for an
example.
-D
database_configuration |
--database-config
database_configuration
- Sets the name of the database configuration file. This option must be
provided!
See /usr/share/hipercontracer/hipercontracer-database.conf for an
example.
-X
mode |
--import-mode
mode
- Specifies the import mode, overriding the mode read from the database
configuration file: DeleteImportedFiles (delete successfully imported
files) MoveImportedFiles (move successfully imported files into
"good" directory), KeepImportedFiles (keep successfully imported
files where they are). Do not use KeepImportedFiles execpt for testing.
Otherwise, the next re-run of hpct-importer tries to import them again!
They may create duplicates when the underlying database system (e.g. NoSQL
without unique indices) does not ensure uniqueness!
-Y
depth |
--import-max-depth
depth
- Specifies the import directory traversal depth, overriding the mode read
from the database configuration file.
-I
path |
--import-file-path
path
- Specifies the import file path, overriding the mode read from the database
configuration file. This path is used to look for files to import.
-B
path |
--bad-file-path
path
- Specifies the "good" file path, overriding the mode read from
the database configuration file. This path is used to move successfully
imported files to, when the import mode is set to KeepImportedFiles.
-G
path |
--good-file-path
path
- Specifies the "bad" file path, overriding the mode read from the
database configuration file. This path is used to move files to which
failed to import.
-F
filter_regexp |
--import-file-path-filter
filter_regexp
- Specifies the import path filter, overriding the mode read from the
database configuration file. This filter can be used to only import a
subset of the import files for testing. The filter expression is an
std::regex regular expression.
-Q
|
--quit-when-idle
- Quit importer when all workers are idle. This will perform only one import
round.
--ping-workers
number
- Sets the number of Ping import workers. Default: 1. An import file is
distributed to a worker by hashing the source address given in the file
name.
--ping-files
number
- Sets the number of Ping import files per database transaction. Default:
1.
--traceroute-workers
number
- Sets the number of Traceroute import workers. Default: 1. An import file
is distributed to a worker by hashing the source address given in the file
name.
--traceroute-files
number
- Sets the number of Traceroute import files per database transaction.
Default: 1.
-h
|
--help
- Prints command help.
--version
- Prints program version.
- hpct-importer -C hipercontracer-importer.conf -D
hipercontracer-database.conf
-
- hpct-importer --importer-config hipercontracer-importer.conf
--database-config hipercontracer-database.conf --verbose
-
- hpct-importer -C hipercontracer-importer.conf -D
hipercontracer-database.conf -F 1234/.*
-
- hpct-importer --version
-
- hpct-importer --h
-
Thomas Dreibholz
https://www.nntb.no/~dreibh/hipercontracer
mailto://dreibh@simula.no