NAME

appjail-healthcheck — Keep your jails healthy and running

SYNOPSIS

DESCRIPTION

The appjail healthcheck utility creates a healthchecker to supervise the jail's health. Technically it supervise whether a healthchecker runs correctly or not, so it checks the health of each healthchecker, not the jail, but we can assume that if a healthchecker fails, it's probably that the jail or the application running inside the jail will get the same result.

The options are as follows:

get [-eHIipt] -n nro jail

Get information about current healthcheckers, that is, the keyword that represent the information to be obtained. Multiple keywords can be specified, which are displayed as a table-like interface in the order in which they are specified. If no keyword is specified, the defaults are nro, enabled, name, status, health_cmd and recover_cmd.

See KEYWORDS for a list of available keywords.

-e

Not required when using -p. The \t character is used to delimit columns, so as not to show strange values, this option shows <TAB> instead of \t in the case that a value contains the latter.

-H

Shows the name of the columns.

-I

Include empty values. By default, a minus sign is displayed when a value is empty.

-i

Don't complain when nro doesn't exist.

-p

Columnate the list.

-t

Tabulate columns and values.

-n nro

Identifier.

list [-eHIipt] [-n nro] jail

Similar to get but shows each keyword for each healthchecker in a nice table.

-e, -H, -I, -p, -t

All of these options perform the opposite task of the options described in get.

-i

Perform the same task as described in get.

-n nro

Only show information for nro.

remove [all|nro nro] jail

Remove a given healthchecker.

all

Remove all healthcheckers.

nro nro

Remove the healthchecker specified by nro.

run jail

Run enabled healthcheckers in foreground.

Note that you should run this subcommand only for debugging, although you can view the log that appjail-startup(1) generates for this purpose. See appjail-logs(1) for more details.

One misconception is that you might think that healthcheckers work forever. They were not designed with this in mind because a service that constantly crashes is useless and should be fixed as soon as possible. What Healthcheckers do is inform you when the command they run fails and attempts to repair the service they supervise time out, but all of this works within limits defined by the set subcommand.

set [-E|-e] [-h command] [-i seconds] [-K seconds] [-k seconds] [-l signal] [-N name] [-n [auto|nro]] [-R number] [-r command] [-S signal] [-s seconds] [-T number] [-t seconds] [-u seconds] jail

[-E|-e]

Enable (-E) or disable (-e) this healthchecker.

-h command

Command to evaluate the jail's health. If this option is not set, the command specified by the DEFAULT_HEALTH_CMD parameter is used.

Prefix command with host: to run the command from the host or with jail: to run the command from the jail. If no prefix is specified, the default value is the one specified by the DEFAULT_HEALTH_TYPE parameter.

If the keyword %j is in command, it will be replaced by the jail name.

-i seconds

Interval to check the jail's health. If this option is not set, the value specified by the DEFAULT_HEALTH_INTERVAL parameter is used.

-K seconds

Send a SIGKILL signal to the process created by the command specified by -r after seconds has been reached only if it is still running. If this option is not set, the value specified by the DEFAULT_RECOVER_TIMEOUT_KILL_AFTER parameter is used.

-k seconds

Send a SIGKILL signal to the process created by the command specified by -h after seconds has been reached only if it is still running. If this option is not set, the seconds specified by the DEFAULT_TIMEOUT_KILL_AFTER parameter is used.

-l signal

Signal to send on timeout. See -u. If this option is not set, the signal specified by the DEFAULT_RECOVER_TIMEOUT_SIGNAL parameter is used.

-N name

Healthchecker name.

-n nro

Identifier. An identifier is composed of a positive number. Use auto (default) to get the lowest identifier value.

-R number

Number of attempts to heal the jail when it is failing and to start the command specified by -r. If this option is not set, the number specified by the DEFAULT_HEALTH_RETRIES parameter is used.

-r command

Command to heal the jail if it is considered to be failing. If this option is not set, the command specified by the DEFAULT_RECOVER_CMD parameter is used.

Prefix command with host: to run the command from the host or with jail: to run the command from the jail. If no prefix is specified, the default value is the one specified by the DEFAULT_RECOVER_TYPE parameter.

If the keyword %j is in command, it will be replaced by the jail name.

-S signal

Signal to send on timeout. See -t. If this option is not set, the signal specified by the DEFAULT_TIMEOUT_SIGNAL parameter is used.

-s seconds

Delay before running the healthchecker. If this option is not set, the value specified by the DEFAULT_HEALTH_START_PERIOD parameter is used.

-T number

The maximum number of attempts reached before the jail is considered unhealthy. If this option is not set, the value specified by the DEFAULT_RECOVER_TOTAL parameter is used.

-t seconds

Send the signal specified by -t to the process created by the command specified by -h after seconds has been reached only if it is still running. If this option is not set, the value specified by the DEFAULT_HEALTH_TIMEOUT parameter is used.

-u seconds

Send the signal specified by -l to the process created by the command specified by -r after seconds has been reached only if it is still running. If this option is not set, the value specified by the DEFAULT_RECOVER_TIMEOUT parameter is used.

HOW HEALTHCHECKERS WORK

All healthcheckers run in parallel and are described step by step as follows:

1. Set the status to starting.
2. If the start period is greater than 0, the process sleeps for the indicated seconds.
3. Sleep the process for the given interval.
4. Execute the health command. If the health type is host, it executes the given command on the host, otherwise if it is jail it executes the command on the jail.
5. If the timeout (in seconds) is reached, the signal configured for the health command is sent.
6. The SIGKILL signal is sent to the health command when its kill after (in seconds) is reached. You should probably set it to be greater than its timeout.
7. If the health command is successful, set the status to healthy and repeat step 3, otherwise set the status to failing and if the current retry count is reached, continue with step 8, otherwise continue with step 3.
8. If the current total of recoveries is reached, set the status to unhealthy and close the healthchecker, otherwise add one to the recovery count and continue with step 9.
9. Execute the recover command. If the recover type is host, it executes the given command on the host, otherwise if it is jail it executes the command on the jail.
10. If the timeout (in seconds) is reached, the signal configured for the recover command is sent.
11. The SIGKILL signal is sent to the recover command when its kill after (in seconds) is reached. You should probably set it to be greater than its timeout.
12. If the recover command fails, set the status to unhealthy and close the healthchecker, otherwise set the status to healthy and continue with step 3.

KEYWORDS

nro

Identifier.

enabled

Shows 1 if the healthchecker is enabled, 0 if it is not.

health_cmd

See set -h.

health_type

See set -h.

interval

See set -i.

kill_after

See set -k.

name

See set -N.

recover_cmd

See set -r.

recover_kill_after

See set -k.

recover_timeout

See set -u.

recover_timeout_signal

See set -l.

recover_total

See set -T.

recover_type

See set -r.

retries

See set -R.

start_period

See set -s.

status

Healthchecker status.

There are four states:

• starting: Initial state. The healthchecker is initializing.
• healthy: The command specified by -h was executed successfully.
• failing: The command specified by -h was executed unsuccessfully.
• unhealthy: State set when the total recovery attempts have been reached or when the recovery command (specified by -r) has been executed without success.

timeout

See set -t.

timeout_signal

See set -S.

EXIT STATUS

The appjail healthcheck utility exits 0 on success, and >0 if an error occurs.

SEE ALSO

appjail-logs(1) appjail-startup(1) appjail-quick(1) sysexits(3)

AUTHORS

Jesús Daniel Colmenares Oviedo <DtxdF@disroot.org>