Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Contact Us
Online Help
Domain Status
Man Pages

Virtual Servers

Topology Map

Server Agreement
Year 2038

USA Flag



Man Pages

Manual Reference Pages  -  CDB_FILE (3)

.ds Aq ’


CDB_File - Perl extension for access to cdb databases



    use CDB_File;
    $c = tie %h, CDB_File, file.cdb or die "tie failed: $!\n";

    $fh = $c->handle;
    sysseek $fh, $c->datapos, 0 or die ...;
    sysread $fh, $x, $c->datalen;
    undef $c;
    untie %h;

    $t = new CDB_File (t.cdb, "t.$$") or die ...;
    $t->insert(key, value);

    CDB_File::create %t, $file, "$file.$$";


    use CDB_File create;
    create %t, $file, "$file.$$";


<B>CDB_FileB> is a module which provides a Perl interface to Dan Bernstein’s <B>cdbB> package:

    cdb is a fast, reliable, lightweight package for creating and
    reading constant databases.

    Reading from a cdb

After the tie shown above, accesses to %h will refer to the <B>cdbB> file file.cdb, as described in tie in perlfunc.

Low level access to the database is provided by the three methods handle, datapos, and datalen. To use them, you must remember the CDB_File object returned by the tie call: $c in the example above. The datapos and datalen methods return the file offset position and length respectively of the most recently visited key (for example, via exists).

Beware that if you create an extra reference to the CDB_File object (like $c in the example above) you must destroy it (with undef) before calling untie on the hash. This ensures that the object’s DESTROY method is called. Note that perl -w will check this for you; see perltie for further details.

    Creating a cdb

A <B>cdbB> file is created in three steps. First call new CDB_File ($final, $tmp), where $final is the name of the database to be created, and $tmp is the name of a temporary file which can be atomically renamed to $final. Secondly, call the insert method once for each (key, value) pair. Finally, call the finish method to complete the creation and renaming of the <B>cdbB> file.

Alternatively, call the insert() method with multiple key/value pairs. This can be significantly faster because there is less crossing over the bridge from perl to C code. One simple way to do this is to pass in an entire hash, as in: $cdbmaker->insert(%hash);.

A simpler interface to <B>cdbB> file creation is provided by CDB_File::create %t, $final, $tmp. This creates a <B>cdbB> file named $final containing the contents of %t. As before, $tmp must name a temporary file which can be atomically renamed to $final. CDB_File::create may be imported.


These are all complete programs.

1. Convert a Berkeley DB (B-tree) database to <B>cdbB> format.

    use CDB_File;
    use DB_File;

    tie %h, DB_File, $ARGV[0], O_RDONLY, undef, $DB_BTREE or
            die "$0: cant tie to $ARGV[0]: $!\n";

    CDB_File::create %h, $ARGV[1], "$ARGV[1].$$" or
            die "$0: cant create cdb: $!\n";

2. Convert a flat file to <B>cdbB> format. In this example, the flat file consists of one key per line, separated by a colon from the value. Blank lines and lines beginning with <B>#B> are skipped.

    use CDB_File;

    $cdb = new CDB_File("data.cdb", "data.$$") or
            die "$0: new CDB_File failed: $!\n";
    while (<>) {
            next if /^$/ or /^#/;
            ($k, $v) = split /:/, $_, 2;
            if (defined $v) {
                    $cdb->insert($k, $v);
            } else {
                    warn "bogus line: $_\n";
    $cdb->finish or die "$0: CDB_File finish failed: $!\n";

3. Perl version of <B>cdbdumpB>.

    use CDB_File;

    tie %data, CDB_File, $ARGV[0] or
            die "$0: cant tie to $ARGV[0]: $!\n";
    while (($k, $v) = each %data) {
            print +, length $k, ,, length $v, ":$k->$v\n";
    print "\n";

4. For really enormous data values, you can use handle, datapos, and datalen, in combination with sysseek and sysread, to avoid reading the values into memory. Here is the script, which can extract uncompressed files and directories from a <B>bunB> file.

    use CDB_File;

    sub unnetstrings {
        my($netstrings) = @_;
        my @result;
        while ($netstrings =~ s/^([0-9]+)://) {
                push @result, substr($netstrings, 0, $1, );
                $netstrings =~ s/^,//;
        return @result;
    my $chunk = 8192;
    sub extract {
        my($file, $t, $b) = @_;
        my $head = $$b{"H$file"};
        my ($code, $type) = $head =~ m/^([0-9]+)(.)/;
        if ($type eq "/") {
                mkdir $file, 0777;
        } elsif ($type eq "_") {
                my ($total, $now, $got, $x);
                open OUT, ">$file" or die "open for output: $!\n";
                exists $$b{"D$code"} or die "corrupt bun file\n";
                my $fh = $t->handle;
                sysseek $fh, $t->datapos, 0;
                $total = $t->datalen;
                while ($total) {
                        $now = ($total > $chunk) ? $chunk : $total;
                        $got = sysread $fh, $x, $now;
                        if (not $got) { die "read error\n"; }
                        $total -= $got;
                        print OUT $x;
                close OUT;
        } else {
                print STDERR "warning: skipping unknown file type\n";

    die "usage\n" if @ARGV != 1;

    my (%b, $t);
    $t = tie %b, CDB_File, $ARGV[0] or die "tie: $!\n";
    map { extract $_, $t, \%b } unnetstrings $b{""};

5. Although a <B>cdbB> file is constant, you can simulate updating it in Perl. This is an expensive operation, as you have to create a new database, and copy into it everything that’s unchanged from the old database. (As compensation, the update does not affect database readers. The old database is available for them, till the moment the new one is finished.)

    use CDB_File;

    $file = data.cdb;
    $new = new CDB_File($file, "$file.$$") or
            die "$0: new CDB_File failed: $!\n";

    # Add the new values; remember which keys weve seen.
    while (<>) {
            ($k, $v) = split;
            $new->insert($k, $v);
            $seen{$k} = 1;

    # Add any old values that havent been replaced.
    tie %old, CDB_File, $file or die "$0: cant tie to $file: $!\n";
    while (($k, $v) = each %old) {
            $new->insert($k, $v) unless $seen{$k};

    $new->finish or die "$0: CDB_File finish failed: $!\n";


Most users can ignore this section.

A <B>cdbB> file can contain repeated keys. If the insert method is called more than once with the same key during the creation of a <B>cdbB> file, that key will be repeated.

Here’s an example.

    $cdb = new CDB_File ("$file.cdb", "$file.$$") or die ...;
    $cdb->insert(cat, gato);
    $cdb->insert(cat, chat);

Normally, any attempt to access a key retrieves the first value stored under that key. This code snippet always prints <B>gatoB>.

    $catref = tie %catalogue, CDB_File, "$file.cdb" or die ...;
    print "$catalogue{cat}";

However, all the usual ways of iterating over a hash---keys, values, and each---do the Right Thing, even in the presence of repeated keys. This code snippet prints <B>cat cat gato chatB>.

    print join( , keys %catalogue, values %catalogue);

And these two both print <B>cat:gato cat:chatB>, although the second is more efficient.

    foreach $key (keys %catalogue) {
            print "$key:$catalogue{$key} ";

    while (($key, $val) = each %catalogue) {
            print "$key:$val ";

The multi_get method retrieves all the values associated with a key. It returns a reference to an array containing all the values. This code prints <B>gato chatB>.

    print "@{$catref->multi_get(cat)}";

multi_get always returns an array reference. If the key was not found in the database, it will be a reference to an empty array. To test whether the key was found, you must test the array, and not the reference.

    $x = $catref->multiget($key);
    warn "$key not found\n" unless $x; # WRONG; message never printed
    warn "$key not found\n" unless @$x; # Correct


The routines tie, new, and finish return <B>undefB> if the attempted operation failed; $! contains the reason for failure.


The following fatal errors may occur. (See eval in perlfunc if you want to trap them.)
Modification of a CDB_File attempted You attempted to modify a hash tied to a <B>CDB_FileB>.
CDB database too large You attempted to create a <B>cdbB> file larger than 4 gigabytes.
[ Write to | Read of | Seek in ] CDB_File failed: <error string> If <B>error stringB> is <B>Protocol errorB>, you tried to use CDB_File to access something that isn’t a <B>cdbB> file. Otherwise a serious OS level problem occurred, for example, you have run out of disk space.


Sometimes you need to get the most performance possible out of a library. Rumour has it that perl’s tie() interface is slow. In order to get around that you can use CDB_File in an object oriented fashion, rather than via tie().

  my $cdb = CDB_File->TIEHASH(/path/to/cdbfile.cdb);

  if ($cdb->EXISTS(key)) {
      print "Key is: ", $cdb->FETCH(key), "\n";

For more information on the methods available on tied hashes see perltie.


The create() interface could be done with TIEHASH.




Tim Goodwin, <>. CDB_File began on 1997-01-08.

Now maintained by Matt Sergeant, <>

Search for    or go to Top of page |  Section 3 |  Main Index

perl v5.20.3 CDB_FILE (3) 2015-06-30

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.