GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  TAP (3)

NAME

tap - write tests that implement the Test Anything Protocol

CONTENTS

Synopsis
Description
     Printf Strings
     Test Plans
     Simple Tests
     Skipping Tests
     MARKING TESTS AS Dq TODO
     SKIP vs. TODO
     Diagnostic Output
     Exit Status
Examples
Compatability
See Also
Standards
History
Authors
Bugs

SYNOPSIS


.In tap.h

DESCRIPTION

The tap library provides functions for writing test scripts that produce output consistent with the Test Anything Protocol. A test harness that parses this protocol can run these tests and produce useful reports indicating their success or failure.

    PRINTF STRINGS

In the descriptions that follow, for any function that takes as the last two parameters "char *, ..." it can be assumed that the char * is a printf -like format string, and the optional arguments are values to be placed in that string.

    TEST PLANS

int plan_tests unsigned int
 
int plan_no_plan void
 
int plan_skip_all char * ...
 

You must first specify a test plan. This indicates how many tests you intend to run, and allows the test harness to notice if any tests were missed, or if the test program exited prematurely.

To do this, use plan_tests, which always returns 0. The function will cause your program to exit prematurely if you specify 0 tests.

In some situations you may not know how many tests you will be running, or you are developing your test program, and do not want to update the plan_tests parameter every time you make a change. For those situations use plan_no_plan. It returns 0, and indicates to the test harness that an indeterminate number of tests will be run.

Both plan_tests and plan_no_plan will cause your test program to exit prematurely with a diagnostic message if they are called more than once.

If your test program detects at run time that some required functionality is missing (for example, it relies on a database connection which is not present, or a particular configuration option that has not been included in the running kernel) use plan_skip_all, passing as parameters a string to display indicating the reason for skipping the tests.

    SIMPLE TESTS

unsigned int ok expression char * ...
 
unsigned int ok1 expression
 
unsigned int pass char * ...
 
unsigned int fail char * ...
 

Tests are implemented as expressions checked by calls to the ok and ok1 macros. In both cases expression should evaluate to true if the test succeeded.

ok allows you to specify a name, or comment, describing the test which will be included in the output. ok1 is for those times when the expression to be tested is self explanatory and does not need an associated comment. In those cases the test expression becomes the comment.

These four calls are equivalent:

int i = 5;

ok(i == 5, "i equals 5"); /* Overly verbose */ ok(i == 5, "i equals %d", i); /* Just to demonstrate printf-like behaviour of the test name */ ok(i == 5, "i == 5"); /* Needless repetition */ ok1(i == 5); /* Just right */

It is good practice to ensure that the test name describes the meaning behind the test rather than what you are testing. Viz

ok(db != NULL, "db is not NULL");            /* Not bad, but */
ok(db != NULL, "Database conn. succeeded");  /* this is better */

ok and ok1 return 1 if the expression evaluated to true, and 0 if it evaluated to false. This lets you chain calls from ok to diag to only produce diagnostic output if the test failed. For example, this code will include diagnostic information about why the database connection failed, but only if the test failed.

ok(db != NULL, "Database conn. succeeded") ||
    diag("Database error code: %d", dberrno);

You also have pass and fail. From the Test::More documentation:

Sometimes you just want to say that the tests have passed.
Usually the case is you’ve got some complicated condition
that is difficult to wedge into an ok().  In this case,
you can simply use pass() (to declare the test ok) or fail
(for not ok).

Use these very, very, very sparingly.

These are synonyms for ok(1, ...) and ok(0, ...).

    SKIPPING TESTS

int skip unsigned int char * ...
 
skip_start expression unsigned int char * ...
 
skip_end
 

Sets of tests can be skipped. Ordinarily you would do this because the test can’t be run in this particular testing environment.

For example, suppose some tests should be run as root. If the test is not being run as root then the tests should be skipped. In this implementation, skipped tests are flagged as being ok, with a special message indicating that they were skipped. It is your responsibility to ensure that the number of tests skipped (the first parameter to skip) is correct for the number of tests to skip.

One way of implementing this is with a "do { } while(0);" loop, or an "if() { } else { }" construct, to ensure that there are no additional side effects from the skipped tests.

if(getuid() != 0) {
        skip(1, "because test only works as root");
} else {
        ok(do_something_as_root() == 0, "Did something as root");
}

Two macros are provided to assist with this. The previous example could be re-written as follows.

skip_start(getuid() != 0, 1, "because test only works as root");

ok(do_something_as_root() == 0, "Did something as root");

skip_end; /* It’s a macro, no parentheses */

    MARKING TESTS AS Dq TODO

void todo_start char * ...
 
void todo_end void
 

Sets of tests can be flagged as being "TODO". These are tests that you expect to fail, probably because you haven’t fixed a bug, or finished a new feature yet. These tests will still be run, but with additional output that indicates that they are expected to fail. Should a test start to succeed unexpectedly, tools like prove(1) will indicate this, and you can move the test out of the todo block. This is much more useful than simply commenting out (or "#ifdef 0 ... #endif") the tests.

todo_start("dwim() not returning true yet");

ok(dwim(), "Did what the user wanted");

todo_end();

Should dwim ever start succeeding you will know about it as soon as you run the tests. Note that unlike the skip_* family, additional code between todo_start and todo_end is executed.

    SKIP vs. TODO

From the Test::More documentation;
If it’s something the user might not be able to do, use SKIP.
This includes optional modules that aren’t installed, running
under an OS that doesn’t have some feature (like fork() or
symlinks), or maybe you need an Internet connection and one
isn’t available.

If it’s something the programmer hasn’t done yet, use TODO. This is for any code you haven’t written yet, or bugs you have yet to fix, but want to put tests in your testing script (always a good idea).

    DIAGNOSTIC OUTPUT


.Fr unsigned int diag char * ...
 

If your tests need to produce diagnostic output, use diag. It ensures that the output will not be considered by the TAP test harness. diag adds the necessary trailing "\n" for you.

diag("Expected return code 0, got return code %d", rcode);

diag always returns 0.

    EXIT STATUS


.Fr int exit_status void
 

For maximum compatability your test program should return a particular exit code. This is calculated by exit_status so it is sufficient to always return from main with either "return exit_status();" or "exit(exit_status());" as appropriate.

EXAMPLES

The tests directory in the source distribution contains numerous tests of tap functionality, written using tap. Examine them for examples of how to construct test suites.

COMPATABILITY

tap strives to be compatible with the Perl Test::More and Test::Harness modules. The test suite verifies that tap is bug-for-bug compatible with their behaviour. This is why some functions which would more naturally return nothing return constant values.

If the
.Lb libpthread is found at compile time, tap should be thread safe. Indications to the contrary (and test cases that expose incorrect behaviour) are very welcome.

SEE ALSO

Test::More(1), Test::Harness(1), prove(1)

STANDARDS

tap requires a -isoC-99 compiler. Some of the tap functionality is implemented as variadic macros, and that functionality was not formally codified until C99. Patches to use tap with earlier compilers that have their own implementation of variadic macros will be gratefully received.

HISTORY

tap was written to help improve the quality and coverage of the FreeBSD regression test suite, and released in the hope that others find it a useful tool to help improve the quality of their code.

AUTHORS


.An Nik Clayton Aq nik@ngo.org.uk , <nik@FreeBSD.org>

tap would not exist without the efforts of
.An Michael G Schwern Aq schqern@pobox.com ,
.An Andy Lester Aq andy@petdance.com , and the countless others who have worked on the Perl QA programme.

BUGS

Ideally, running the tests would have no side effects on the behaviour of the application you are testing. However, it is not always possible to avoid them. The following side effects of using tap are known.
  • stdout is set to unbuffered mode after calling any of the plan_* functions.
Search for    or go to Top of page |  Section 3 |  Main Index


Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.