NAME

eclat.conf - configuration file for eclat(1).

DESCRIPTION

Eclat configuration file consists of statements and comments.

There are three classes of lexical tokens: keywords, values, and separators. Blanks, tabs, newlines and comments, collectively called white space are ignored except as they serve to separate tokens. Some white space is required to separate otherwise adjacent keywords and values.

COMMENTS

Comments may appear anywhere where white space may appear in the configuration file. There are two kinds of comments: single-line and multi-line comments. Single-line comments start with # or // and continue to the end of the line:

# This is a comment
// This too is a comment

Multi-line or C-style comments start with the two characters /* (slash, star) and continue until the first occurrence of */ (star, slash).

Multi-line comments cannot be nested. However, single-line comments may well appear within multi-line ones.

Pragmatic Comments

Pragmatic comments are similar to usual single-line comments, except that they cause some changes in the way the configuration is parsed. Pragmatic comments begin with a # sign and end with the next physical newline character.

#include <file>

#include file

Include the contents of the file file. If it is an absolute file name, both forms are equivalent. Otherwise, the form with angle brackets searches for the file in the include search path, while the second one looks for it in the current working directory first, and, if not found there, in the include search path.

The default include search path is:

prefix/share/eclat/VERSION/include
prefix/share/eclat/include

where prefix stands for the installation prefix (normally /usr or /usr/local), and VERSION stands for eclat version number.

The include search path can be modified using the -I (--include-directory) command line option.

#include_once <file>

#include_once file

Same as #include, except that, if the file has already been included, it will not be included again.

#line num

#line num "file"

This line causes the parser to believe, for purposes of error diagnostics, that the line number of the next source line is given by num and the current input file is named by file. If the latter is absent, the remembered file name does not change.

# num "file"

This is a special form of the #line statement, understood for compatibility with the C preprocessor.

These statements provide a rudimentary preprocessing features. For more sophisticated ways to modify configuration before parsing, see PREPROCESSOR.

STATEMENTS

Simple statement

A simple statement consists of a keyword and value separated by any amount of whitespace. Simple statement is terminated with a semicolon (;).

The following is a simple statement:

standalone yes;
pidfile /var/run/slb.pid;

A keyword begins with a letter and may contain letters, decimal digits, underscores (_) and dashes (-). Examples of keywords are:

region
format-file

A value can be one of the following:

number

A number is a sequence of decimal digits.

boolean

A boolean value is one of the following: yes, true, t or 1, meaning true, and no, false, nil, 0 meaning false.

unquoted string

An unquoted string may contain letters, digits, and any of the following characters: _, -, ., /, @, *, :.

quoted string

A quoted string is any sequence of characters enclosed in double-quotes ("). A backslash appearing within a quoted string introduces an escape sequence, which is replaced with a single character according to the following rules:

	Sequence	Replaced with
	\a	Audible bell character (ASCII 7)
	\b	Backspace character (ASCII 8)
	\f	Form-feed character (ASCII 12)
	\n	Newline character (ASCII 10)
	\r	Carriage return character (ASCII 13)
	\t	Horizontal tabulation character (ASCII 9)
	\v	Vertical tabulation character (ASCII 11)
	\\	A single backslash
	\"	A double-quote.

In addition, the sequence \newline is removed from the string. This allows to split long strings over several physical lines, e.g.:

    "a long string may be\
     split over several lines"

If the character following a backslash is not one of those specified above, the backslash is ignored and a warning is issued.

Two or more adjacent quoted strings are concatenated, which gives another way to split long strings over several lines to improve readability. The following fragment produces the same result as the example above:

    "a long string may be"
    " split over several lines"

Here-document

A here-document is a special construct that allows to introduce strings of text containing embedded newlines.

The <<word construct instructs the parser to read all the following lines up to the line containing only word, with possible trailing blanks. Any lines thus read are concatenated together into a single string. For example:

    <<EOT
    A multiline
    string
    EOT

The body of a here-document is interpreted the same way as a double-quoted string, unless word is preceded by a backslash (e.g. <<\EOT) or enclosed in double-quotes, in which case the text is read as is, without interpretation of escape sequences.

If word is prefixed with - (a dash), then all leading tab characters are stripped from input lines and the line containing word. Furthermore, - is followed by a single space, all leading whitespace is stripped from them. This allows to indent here-documents in a natural fashion. For example:

    <<- TEXT
        The leading whitespace will be
        ignored when reading these lines.
    TEXT

It is important that the terminating delimiter be the only token on its line. The only exception to this rule is allowed if a here-document appears as the last element of a statement. In this case a semicolon can be placed on the same line with its terminating delimiter, as in:

    help-text <<-EOT
        A sample help text.
    EOT;

list

A comma-separated list of values, enclosed in parentheses. The following example shows a statement whose value is a list of strings:

    alias (test, null);

In any context where a list is appropriate, a single value is allowed without being a member of a list: it is equivalent to a list with a single member. This means that, e.g.

    alias test;

is equivalent to

    alias (test);

provided that the alias statement is defined as taking a list as its argument.

block statement

A block statement introduces a logical group of statements. It consists of a keyword, followed by an optional value, called a tag, and a sequence of statements enclosed in curly braces, as shown in the example below:

    map InstanceId {
        type gdbm;
        file "instances.txt";
    }

The closing curly brace may be followed by a semicolon, although this is not required.

PREPROCESSOR

Before being parsed, the configuration file is preprocessed. The built-in preprocessor handles only file inclusion and #line statements, while the rest of traditional preprocessing facilities, such as macro expansion, is supported via m4(1), which is used as external preprocessor.

The external preprocessor is invoked with the -s flag, which instructs it to include line synchronization information in its output. This information is then used by the parser to display meaningful diagnostic.

MACRO EXPANSION

Arguments of some statements undergo macro expansion before use. During the macro expansion any occurrence of ${NAME} is replaced by the value of macro NAME. Macro names follow the usual convention: they begin with a letter and contain letters digits and underscores. The curly braces around the NAME are optional. They are required only if the macro reference is followed by a character that is not to be interpreted as part of its name, as in ${command}string.

If a statement is eligible for macro expansion, the following macros are always defined:

login: Current user login name.
user: Same as above, except if explicitly overridden by the user name supplied from the command line. In particular, this happens when the prompt statement in the LDAP map declaration is set to true (see the LDAP MAP subsection below).
home: Current user home directory.

Each statement can have a set of additional macros. These are listed in the corresponding sections below.

EC2 CONFIGURATION

The statements in this group configure EC2 endpoints and regions.

A region determines physical location of servers supporting given AWS resources.

An endpoint is a URI of the Amazon server. The endpoint to use is selected based on the region name.

default-endpoint hostname;: Defines the endpoint to use if no region-specific endpoint is configured.
default-region name;: Defines the name of the default region.
region name endpoint;: Declares a region and the corresponding endpoint.

The region is determined by the following algorithm:

1.: If the --region option is given, take its argument as the region name.
2.: If the default-region statement is defined in the configuration file, use its argument as the region name.
3.: Otherwise, attempt to obtain region from the instance store. Obviously, this step can succeed only if eclat is run on an EC2 instance.

If none of these steps succeed, the program aborts.

Endpoint is selected by looking up a region statement with the name argument matching the currently selected region. If such a statement is found, its endpoint argument defines the endpoint to use. Otherwise, endpoint is taken from the default-endpoint statement.

An example of the EC2 endpoint configuration follows:

# Use this endpoint by default.
default-endpoint "ec2.amazonaws.com";
# Declare what endpoints to use for each availability region:
#  US East (Northern Virginia) Region
region  us-east-1       ec2.us-east-1.amazonaws.com;
#  US West (Oregon) Region
region  us-west-2       ec2.us-west-2.amazonaws.com;

HTTP METHOD AND SIGNATURE

http-method ARG;: Configures HTTP method. Allowed values for ARG are GET (the default), and POST. ARG is case-insensitive. The POST method implies using signature version 4.
signature-version 2|4;: Use the given AWS signature version algorithm. Version 4 is more secure and is used by default.

SSL CONFIGURATION

The ssl statement has two forms, and can be used as scalar or as a block statement. In scalar form it is used to enable SSL:

ssl yes: Enables SSL.

The block form allows for more detailed control over the SSL configuration:

    ssl {
        enable bool;
        verify boolean;
        ca-file file;
        ca-path directory;
    }

Its statements are:

enable bool: Enable or disable SSL. This statement is equivalent to ssl in scalar form.
verify boolean: Verify peer certificate. The default is yes.
ca-file file: Supplies the name of the file with CA certificates.
ca-path directory: Supplies the name of the directory where CA certificates are stored.

By default the CA certificates shipped with libcurl(3) will be used. You would rarely need to use ca-file or ca-path statements.

AUTHENTICATION

authentication-provider TYPE [ARG];

Defines authentication provider to use. Authentication provider is a service that supplies AWS access key ID and secret key. See eclat(1), section AUTHENTICATION, for a detailed description.

The TYPE argument defines the provider. Allowed values are file, and instance-store.

If TYPE is file, the ARG parameter is treated as a shell globbing pattern: all files matching this pattern are attempted in turn, until a keypair is found in one of them.

If TYPE is instance-store, credentials will be obtained from the instance store. ARG is optional. If supplied, it should be the name of the IAM role this instance is launched with. At the time of this writing, an instance can be associated with a single role, which will be used by default.

access-file name;

This is a shortcut for authentication-provider file name.

signature-version N;

Declares the signature version. Valid values for N are 2, which is the default, and 4, which provides a better security.

RETRY ON ERRORS

If AWS responds with a RequestLimitExceeded code, eclat retries the request using exponential backoff with jitter algorithm. The algorithm is controlled by two configuration values: max-retry-interval and total-retry-timeout. When the RequestLimitExceeded error is returned, eclat will sleep for up to 2 seconds and then retry the request. For each subsequent RequestLimitExceeded error, it will calculate the timeout using the following formula:

   t = rand(0, min(M, 2 ** N)) + 1

where N is the attempt number, M is the value of max-retry-interval parameter, 'rand(a,b)' selects the integer random number X such that 0 <= X <= b, and '**' is the power operator. The attempts to resend the request will continue until either a response other than RequestLimitExceeded is received (be it a response to the query or another error response), or the total time spent in the retry loop becomes equal to or greater than total-retry-timeout, whichever occurs first.

Default values are:

max-retry-interval 600;
total-retry-timeout 1800;

INSTANCE STORE CONFIGURATION

The instance-store compound statement configures HTTP access to the instance store. By default, eclat uses standard AWS values. This statement is intended mainly as an aid in debugging:

    instance-store {
        base-url URL;
        port NUMBER;
        document-path STRING;
        credentials-path STRING;
    }

base-url URL;: Base URL to use, instead of http://169.254.169.254/latest.
port NUMBER;: Port to use (defaults to 80).
document-path STRING;: Pathname (relative to base-url) of the instance identity document file. Default: dynamic/instance-identity/document.
credentials-path STRING;: Pathname (relative to base-url) of the instance store credentials directory. Default is meta-data/iam/security-credentials.

FORMAT DEFINITIONS

This group of statements declares the formats to use.

format command text;

Sets default format for use with command. The command argument is either the command name, as defined in section COMMANDS of eclat(1), or the corresponding Amazon action name (see http://docs.amazonwebservices.com/AWSEC2/latest/APIReference/query-apis.html, for the action names). Generally speaking, command names are derived from the corresponding action names by inserting a dash between each capitalized part and then downcasing all capital letters.

The text argument is the forlan text to use as the format. See forlan(5), for a detailed description of the forlan syntax.

It is common practice to use the "here-document" syntax with this statement, as shown in the example below:

format "DescribeTags" <<EOT
  if (.DescribeTagsResponse) {
    for (var in .DescribeTagsResponse.tagSet.item) {
      print(var.resourceId,"\t",
            var.resourceType,"\t",
            var.key,"\t",
            var.value,"\n");
    }
  }
EOT;

define-format name text;

Defines a new format and assigns name to it. This format can then be used via the --format=name option. The meaning of the text parameter is the same as with the format statement.

format-file filename;

Read format from filename. Before using, the filename undergoes macro expansion. The following macros are defined in addition to the standard ones:

command: Replaced with the name of the eclat command being executed.
action: Replaced with the corresponding Amazon action name.

These macros allow you to have separate format files for each EC2 request. For example:

format-file "/usr/share/eclat/$command.forlan";

Given this statement, the command eclat lstag, for instance, will read configuration from file /usr/share/eclat/lstag.forlan.

When selecting the format to use, eclat uses the following algorithm:

1.: If the --format-expression (-e) option is given, its argument is parsed as the format text.
2.: Otherwise, if the --format-file (-F) option is given, the format is read from the file named by its argument.
3.: Otherwise, if the --format (-H) option is given, the format is looked up among named formats (declared via the define-format statement), using the argument as the look-up key.
4.: Otherwise, if the default format is defined for this option (see the format configuration statement), it is used.
5.: Finally, if the format-file configuration statement is used, the format is read from the file named in its argument, after the macro expansion.
6.: If the format cannot determined by the above steps, an error is reported and the program terminates.

CONFIRMATION

Many eclat commands result in modification of your EC2 resources. Some of them are destructive, in the sense that such modifications cannot be undone (e.g. deleting of a volume or termination of an instance). To reduce the possibility of careless usage, eclat can be configured to interactively ask for a confirmation when such a command is requested. This is configured by the confirm statement:

confirm mode command;
confirm mode (command[, command...]);
confirm mode class;

The mode argument specifies the requested confirmation mode. Its valid values are:

tty: Ask for confirmation if the controlling terminal is a tty, i.e. if eclat is started from the command line.
always: Always ask for confirmation. If the controlling terminal is not a tty, abort the command.
positive: Never ask. Assume positive confirmation. This is the default.
negative: Never ask, assuming negative confirmation.

The second argument specifies the commands to which this mode is applied. It can be a single command name or tag, a comma-separated list of command names or tags, or a class of commands. Valid values for class are:

all: All commands that modify EC2 resources.
destructive: Commands that destructively modify resources.

Consider the following example:

confirm tty destructive;
confirm tty (StopInstance, StartInstance);

It instructs eclat to ask for confirmation if one of the destructive commands is requested, or if the command is start-instance or stop-instance.

Here is an example of how this modifies the behavior of rmaddr command:

$ eclat rmaddr 192.168.0.1
Proceed with release-address [Y/n] _

If the response begins with Y (case-insensitive), it is taken for a positive answer, and the command will be executed. Otherwise, eclat exits returning code 16 to the shell.

The current confirmation setting can be overridden using the -Y (--yes) or -N (--no) command line option. The former forces positive, and the latter negative confirmation mode for the requested command, e.g.:

$ eclat -Y rmvol vol-d1234aef

MAPS

Maps provide a way to translate arbitrary symbolic names to the Amazon resource identifiers. See the section MAPS in eclat(1), for the discussion of the subject. The translation is disabled by default. This can be changed using the following statement:

translate BOOL: Depending on its argument, enables or disables ID translation by default.

A map is declared using the following statement:

map name {
    type backend-type;
    key trans;
    /* Type-specific statements. */
}

The name argument is used to identify the map. A map can be used from the command line using the --map=name option. The following names are special:

ImageId: Translates AMI names to IDs.
InstanceId: Translates instance names to IDs.
GroupId: Translates security group names to Amazon group IDs.
GroupName: Translates security group names to Amazon names.
SnapshotId: Translates snapshot names to IDs.
VolumeId: Translates volume names to IDs.
VpcId: Translates VPC names to IDs.
InternetGatewayId: Translates internet gateway names to IDs.
SubnetId: Translates subnet names to IDs.
RouteTableId: Translates route table names to IDs.
AZ: Translates availability region names to Amazon names.
reg: Translates region names to Amazon region names.

The backend-type declares the type of this map. Depending on this type, several sets of type-specific statements can be present. These are described in detail below.

The optional key statement defines the key format. Its argument is subject to macro expansion. The following macros are defined:

key: The original key.
map: The name of this map.

NULL MAP

The null map always returns the key value (eventually modified using the key statement). It is configured by setting

type null;

in the corresponding map section.

FILE MAP

File map is the simplest type of maps. The data are stored in a plain-text file. Empty lines and lines beginning with a # are ignored. Non-empty lines contain a key and the corresponding value, separated by a colon. Both key and value are taken verbatim, including any whitespace characters, if such are present. Following is an example of the file map, which defines two instance names:

# Database server
dbserver:i-3456aefd
# Access server
access:i-deadbeef

The file map is declared using the following syntax:

    map name {
        type file;
        key trans;
        file filename;
    }

The only type-specific statement is:

file filename: Defines the name of the file to read the map from.

GDBM MAP

GDBM map, as its name implies, looks up the data in a gdbm(3) database. It is defined using the following statement:

    map name {
        type gdbm;
        key trans;
        file filename;
        null bool;
    }

file filename: Defines the name of the database file to use.
null bool: If set to true this statement instructs the backend to count terminating null character as a part of key length. By default, the null character is not counted.

SEQUENCE MAP

Maps of this type consult several other eclat maps in sequence, using return value from the previous map as a key to the subsequent one. The map returns the value produced by the last map in sequence.

    map name {
        type sequence;
        key trans;
        sequence maps;
    }

sequence maps: This statement defines the sequence of maps to query. Its argument is a list of map names. The maps it references must be declared elsewhere in the configuration file (not necessarily before the sequence map that references them).

For example:

    map InstanceId {
        type sequence;
        sequence (keytab, instancetab);
    }

If the sequence consists of a single element, such a map is effectively an alias to the map named by that element:

    map i {
        type sequence;
        sequence InstanceId;
    }

LDAP MAP

This map uses LDAP database to translate names. It is defined using the following syntax:

    map <name: string> {
        type ldap;
        key trans;
        ldap-version number;
        uri string;
        base string;
        binddn string;
        bindpw string;
        passfile string;
        prompt boolean;
        tls [no | yes | only];
        filter string;
        attr string;
        debug number;
    }

ldap-version n: Defines the LDAP version to use. Valid values for n are 2 and 3 (the default).
uri string: Defines the URI of the LDAP server. This statement is mandatory.
base string: Defines the search base.
binddn dn: Specifies the default bind DN to use when performing ldap operations. The dn argument must be specified as a Distinguished Name in LDAP format.
The argument is subject to macro expansion using the default set of macros.
bindpw string: Specifies the password to use when performing the bind. Use this option only if the configuration file is protected by appropriate permissions. Otherwise, use passfile or prompt statements.
passfile file: Read the first line from file and use it as the password. The newline character is stripped from the input.
The file argument is subject to macro expansion, using the default set of macros.
prompt boolean: Prompt the user for the user name and password, unless the latter is determined using the passfile statement.
tls value: Controls whether TLS is desired or required. If value is no (the default), TLS will not be used. If it is yes, the module will issue the ``StartTLS'' command, but will continue anyway if it fails. Finally, if value is only, TLS is mandatory, and the backend will not establish LDAP connection unless ``StartTLS'' succeeds.
filter expr: Sets the LDAP filter expression. The expr should conform to the string representation for search filters as defined in RFC 4515.
This statement is mandatory.
attr string: Defines the attribute to use for the return value.
This statement is mandatory.
debug n: Sets the LDAP debug number. Refer to the OpenLDAP documentation for the valid values of n and their meaning.

EC2 MAP

This map makes it possible to use EC2 services to translate identifiers. The idea is to query EC2 using the symbolic ID and obtain the real resource ID from the request. The map is defined as follows:

    map <name: string> {
        type ec2;
	key trans;
        action action-name;
        arguments arg-list;
        return return-path;
    }

action action-name: Defines the EC2 action name to use. Refer to the Amazon API documentation for a list of action names.
arguments arg-list: Defines the list of arguments to send with the request. Any occurrence of the $key variable within that list is replaced with the actual key value.
return return-path: Defines the return value, using a path notation.

The example below illustrates the use of the DescribeImages action to implement the ImageId map:

    map "ImageId" {
        type ec2;
        action DescribeImages;
        arguments ("Owner.1=self",
                   "Filter.1.Name=tag:Name",
                   "Filter.1.Value.1=${key}");
        return ".DescribeImagesResponse.imagesSet.item.imageId";
    }

AUTHORS

Sergey Poznyakoff

BUG REPORTS

Report bugs to <bug-eclat@gnu.org.ua>.

COPYRIGHT

Copyright © 2012-2018 Sergey Poznyakoff
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.