Benchmark::Dumb - Benchmark.pm compatibility layer for Dumbbench
use Benchmark::Dumb qw(:all);
cmpthese(
0.05, # 5% precision
{
fast => 'fast code',
slow => 'slow code',
}
);
# etc
This module implements an interface that is
similar to the functional
interface of the Benchmark module. This module, however, uses the Dumbbench
benchmarking tool under the hood. For various reasons, the interface and the
output of the benchmark runs are
not exactly the same. Among other
reasons, you would lose out on some of "Dumbbench"'s advantages.
Understanding this documentation requires some familiarity of how
"Benchmark.pm" works since it mostly explains how this module is
different.
Please read the following section carefully to understand the most important
differences:
This is a list of differences to the interface and behaviour of the
"Benchmark" module. It may not be complete. If so, please let me
know.
- •
- The $count parameter is interpreted very differently!
With "Benchmark.pm", specifying a positive integer meant that the
benchmark should be run exactly $count times. A negative value indicated
that the code should be run until $count seconds of cumulated run-time
have elapsed.
With "Benchmark::Dumb", we can do better. A positive integer
specifies the minimum number of iterations. "Dumbbench"
may choose to run more iterations to arrive at the necessary precision.
Specifying a certain target run-time (via a negative number for $count) may
seem like a tempting idea, but if you care at all about the precision of
your result, it's quite useless. This usage is not supported by
"Benchmark::Dumb" !
Instead, if you pass a positive floating point number as $count, the
fractional part of the number willbe interpreted as the target relative
precision that you expect from the result.
Finally, supplying a 0 as $count means that "Dumbbench" will be
invoked with the default settings. This is good enough for most
cases.
- •
- There are no exported functions by default!
- •
- The ":hireswallclock" option is ignored. We always use the
hi-res wallclock! While on the topic: We also only use wallclock
times.
- •
- The cache-related functions aren't implemented because we don't use a
cache.
- •
- The original "Benchmark.pm" implementation provides a
rudimentary object-oriented interface. We do not faithfully copy that. See
"METHODS" below.
- •
- The benchmark code will be run in a special package. It will not be
run in the caller package (at this time). If you need access to
previously-set-up package variables, you will need to include a
"package" statement in your code.
- •
- The "debug" method is not implemented and neither is the
"countit" function.
- •
- Some things that were previously considered functions are now considered
primarily methods (see "METHODS" below). But they are all
importable and callable as functions.
These functions work mostly (see the $count gotcha above) like the equivalent
functions in the "Benchmark" module, but the textual output is
different in that it contains estimates of the uncertainties. Some of the
style and
format options of the original functions are ignored
for the time being.
I'm quoting the "Benchmark" documentation liberally.
Time
COUNT iterations of
CODE.
CODE may be a string to eval
or a code reference. Unlike with the original "Benchmark", the code
will
not run in the caller's package. Results will be printed to
"STDOUT" as
TITLE followed by the "timestr()".
TITLE defaults to "timethis COUNT" if none is provided.
STYLE determines the format of the output, as described for
"timestr()" below.
Please read the section on "Differences to Benchmark.pm" for a
discussion of how the
COUNT parameter is interpreted.
Returns a "Benchmark::Dumb" object.
Arguments:
COUNT is the number of times to run the loop (see discussion
above), and
CODE is the code to run.
CODE may be either a code
reference or a string to be eval'd. Unlike with the original
"Benchmark", the code will
not run in the caller's package.
Returns a "Benchmark::Dumb" object.
The
CODEHASHREF is a reference to a hash containing names as keys and
either a string to eval or a code reference for each value. For each (
KEY,
VALUE) pair in the
CODEHASHREF, this routine will
call
timethis(COUNT, VALUE, KEY, STYLE)
The routines are called in string comparison order of
KEY.
The
COUNT must be positive or zero. See discussion above.
Returns a hash reference of "Benchmark::Dumb" objects, keyed by name.
Optionally calls "timethese()", then outputs a comparison chart. This:
cmpthese( 500.01, { a => "++\$i", b => "\$i *= 2" } ) ;
outputs a chart like:
Rate b a
b 5.75e+06+-47000/s -- -70.1%
a 1.925e+07+-650000/s 235+-12% --
This chart is sorted from slowest to fastest, and shows the percent speed
difference between each pair of tests as well as the uncertainties on the
rates and the relative speed difference. The uncertainty on a speed difference
may be omitted if it is below one tenth of a percent.
c<cmpthese> can also be passed the data structure that
"timethese()" returns:
my $results = timethese( 100.01, { a => "++\$i", b => "\$i *= 2" } ) ;
cmpthese( $results );
in case you want to see both sets of results. If the first argument is an
unblessed hash reference, that is "RESULTSHASHREF"; otherwise that
is "COUNT".
Returns a reference to an ARRAY of rows, each row is an ARRAY of cells from the
above chart, including labels. This:
my $rows = cmpthese( 500.01, { a => '++$i', b => '$i *= 2' }, "none" );
returns a data structure like:
[
[ '', 'Rate', 'b', 'a' ],
[ 'b', '5.75e+06+-47000/s', '--', '-70.1%' ],
[ 'a', '1.925e+07+-650000/s', '235+-12%', '--' ],
]
Please note that while the original "Benchmark" objects practically
asked for manual introspection since the API didn't provide convenient access
to all information, that practice is frowned upon with
"Benchmark::Dumb" objects. You have been warned. If there's a piece
of API missing, let me know.
There's no public constructor for "Benchmark::Dumb" objects because it
doesn't do what the "Benchmark" constructor did: It's not running
"time()" for you.
Returns the name of the benchmark result if any. (Not in "Benchmark".)
Returns the number of samples taken.
Returns a new "Benchmark::Dumb" object that represents the sum of this
benchmark and $other_benchmark.
"timesum($b1, $b2)" was a function in the original
"Benchmark" module and may be called as a function on two
"Benchmark::Dumb" objects as well. It is available for importing
into your namespace.
Returns a new "Benchmark::Dumb" object that represents the difference
between this benchmark and $other_benchmark ("$this-$other").
"timediff($b1, $b2)" was a function in the original
"Benchmark" module and may be called as a function on two
"Benchmark::Dumb" objects as well. It is available for importing
into your namespace.
Returns a textual representation of this benchmark.
"timestr($b)" was a function in the original "Benchmark"
module and may be called as a function on a "Benchmark::Dumb" object
as well. It is available for importing into your namespace.
Dumbbench
Benchmark
Some of the documentation was taken from the documentation for
"Benchmark.pm"'s functions.
Steffen Mueller, <smueller@cpan.org>
Copyright (C) 2010, 2012 by Steffen Mueller
This library is free software; you can redistribute it and/or modify it under
the same terms as Perl itself, either Perl version 5.8.1 or, at your option,
any later version of Perl 5 you may have available.