GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  KYUAFILE (5)

NAME

Kyuafile - Test suite description files

CONTENTS

Synopsis
Description
     File versioning
     Test suite definition
     Test program registration
     Recursion
     Top-level Kyuafile
Files
Examples
     Connecting disjoint test suites
See Also

SYNOPSIS

atf_test_program string name [string metadata] include string path plain_test_program string name [string metadata] syntax int version tap_test_program string name [string metadata] test_suite string name

DESCRIPTION

A test suite is a collection of test programs and is represented by a hierarchical layout of test binaries on the file system. Any subtree of the file system can represent a test suite, provided that it includes one or more s, which are the test suite definition files.

A Kyuafile is a Lua script whose purpose is to describe the structure of the test suite it belongs to. To do so, the script has access to a collection of special functions provided by kyua(1).

    File versioning

Every Kyuafile file starts with a call to syntax int version. This call determines the specific schema used by the file so that future backwards-incompatible modifications to the file can be introduced.

Any new Kyuafile file should set version to '2'.

    Test suite definition

If the Kyuafile registers any test programs, the Kyuafile must define the name of the test suite the test programs belong to by using the test_suite function at the very beginning of the file.

The test suite name provided in the test_suite call tells kyua(1) which set of configuration variables from kyua.conf(5) to pass to the test programs at run time.

    Test program registration

A Kyuafile can register test programs by means of a variety of *_test_program functions, all of which take the name of a test program and a set of optional metadata properties that describe such test program.

The test programs to be registered must live in the current directory; in other words, the various *_test_program calls cannot reference test programs in other directories. The rationale for this is to force all Kyuafile files to be self-contained, and to simplify their internal representation.

ATF test programs are those that use the atf(7) libraries. They can be registered with the atf_test_program table constructor. This function takes the name of the test program and a collection of optional metadata settings for all the test cases in the test program. Any metadata properties defined by the test cases themselves override the metadata values defined here.

Plain test programs are those that return 0 on success and non-0 on failure; in general, most test programs (even those that use fancy unit-testing libraries) behave this way and thus also qualify as plain test programs. They can be registered with the plain_test_program table constructor. This function takes the name of the test program, an optional test_suite name that overrides the global test suite name, and a collection of optional metadata settings for the test program.

TAP test programs are those that implement the Test Anything Protocol. They can be registered with the tap_test_program table constructor. This function takes the name of the test program and a collection of optional metadata settings for the test program.

The following metadata properties can be passed to any test program definition:
allowed_architectures Whitespace-separated list of machine architecture names allowed by the test. If empty or not defined, the test is allowed to run on any machine architecture.
allowed_platforms Whitespace-separated list of machine platform names allowed by the test. If empty or not defined, the test is allowed to run on any machine platform.
description Textual description of the test.
required_configs Whitespace-separated list of configuration variables that the test requires to be defined before it can run.
required_disk_space Amount of available disk space that the test needs to run successfully.
required_files Whitespace-separated list of paths that the test requires to exist before it can run.
required_memory Amount of physical memory that the test needs to run successfully.
required_programs Whitespace-separated list of basenames or absolute paths pointing to executable binaries that the test requires to exist before it can run.
required_user If empty, the test has no restrictions on the calling user for it to run. If set to 'unprivileged', the test needs to not run as root. If set to 'root', the test must run as root.
timeout Amount of seconds that the test is allowed to execute before being killed.
X-NAME Custom variable defined by the test, where 'NAME' denotes the name of the variable.

    Recursion

To reference test programs in another subdirectory, a different Kyuafile must be created in that directory and it must be included into the original Kyuafile by means of the include function.

include may only be called with a relative path and with at most one directory component. This is by design: Kyua uses the file system structure as the layout of the test suite definition. Therefore, each subdirectory in a test suite must include its own Kyuafile and each Kyuafile can only descend into the s of immediate subdirectories.

If you need to source a Kyuafile located in disjoint parts of your file system namespace, you will have to create a 'shadow tree' using symbolic links and possibly helper s to plug the various subdirectories together. See the EXAMPLES section below for details.

Note that each file is processed in its own Lua environment: there is no mechanism to pass state from one file to the other. The reason for this is that there is no such thing as a "top-level" Kyuafile in a test suite: the user has to be able to run the test suite from any directory in a given hierarchy, and this execution must not depend on files that live in parent directories.

    Top-level Kyuafile

Every system has a top directory into which test suites get installed. The default is /usr/local/tests. Within this directory live test suites, each of which is in an independent subdirectory. Each subdirectory can be provided separately by independent third-party packages.

Kyua allows running all the installed test suites at once in order to provide comprehensive cross-component reports. In order to do this, there is a special file in the top directory that knows how to inspect the subdirectories in search for other Kyuafiles and include them.

The FILES section includes more details on where this file lives.

FILES

/usr/local/tests/Kyuafile.
  Top-level Kyuafile for the current system.
/usr/local/share/examples/kyua/Kyuafile.top.
  Sample file to serve as a top-level Kyuafile.

EXAMPLES

The following Kyuafile is the simplest you can define. It provides a test suite definition and registers a couple of different test programs using different interfaces:
syntax(2)

test_suite(’first’)

atf_test_program{name=’integration_test’} plain_test_program{name=’legacy_test’}

The following example is a bit more elaborate. It introduces some metadata properties to the test program definitions and recurses into a couple of subdirectories:

syntax(2)

test_suite(’second’)

plain_test_program{name=’legacy_test’, allowed_architectures=’amd64 i386’, required_files=’/bin/ls’, timeout=30}

tap_test_program{name=’privileged_test’, required_user=’root’}

include(’module-1/Kyuafile’) include(’module-2/Kyuafile’)

    Connecting disjoint test suites

Now suppose you had various test suites on your file system and you would like to connect them together so that they could be executed and treated as a single unit. The test suites we would like to connect live under /usr/tests, /usr/local/tests and ~/local/tests.

We cannot create a Kyuafile that references these because the include directive does not support absolute paths. Instead, what we can do is create a shadow tree using symbolic links:

$ mkdir ~/everything
$ ln -s /usr/tests ~/everything/system-tests
$ ln -s /usr/local/tests ~/everything/local-tests
$ ln -s ~/local/tests ~/everything/home-tests

And then we create an ~/everything/Kyuafile file to drive the execution of the integrated test suite:

syntax(2)

test_suite(’test-all-the-things’)

include(’system-tests/Kyuafile’) include(’local-tests/Kyuafile’) include(’home-tests/Kyuafile’)

Or, simply, you could reuse the sample top-level Kyuafile to avoid having to manually craft the list of directories into which to recurse:

$ cp /usr/local/share/examples/kyua/Kyuafile.top ~/everything/Kyuafile

SEE ALSO

kyua(1)
Search for    or go to Top of page |  Section 5 |  Main Index


Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.