NAME

Data::HexDump::Range - Hexadecimal Range Dumper with color, bitfields and skip ranges

SYNOPSIS

  my $hdr = Data::HexDump::Range->new() ;
  
  print $hdr->dump([['magic cookie', 12, 'red'],['image type', 2, 'green']],  $data) ;
  print $hdr->dump('magic cookie, 12, red :image type, 2, green',  $data) ;
  
  $hdr->gather(['magic cookie', 12, 'red'], $data) ; 
  $hdr->gather(['image type', 2, 'green'], $other_data) ;
  
  print $hdr->dump_gathered() ;
  $hdr->reset() ;

DESCRIPTION

Creates a dump from binary data and user defined range descriptions. The goal of this module is to create an easy to understand dump of binary data.

This achieved through:

Highlighted (colors) dump that is easier to understand than a monochrome blob of hex data
Multiple rendering modes with different output formats
Bitfield rendering
Skipping uninterresting data
The possibility to describe complex structures

DOCUMENTATION

The shortest perl dumper is "perl -ne 'BEGIN{$/=\16} printf "%07x0: @{[unpack q{(H2)*}]}\n", $.-1'", courtesy of a golfing session with Andrew Rodland <arodland@cpan.org> aka hobbs. priodev, tm604, Khisanth and other provided valuable insight particularely with the html output.

hexd from libma <http://www.ioplex.com/~miallen/libmba/> is nice tools that inspired me to write this module. This module offers many more options but hexd may be a better alternative If you need very fast dump generation.

Data::HexDump::Range splits binary data according to user defined ranges and renderes them as a hex or/and decimal data dump. The data dump can be rendered in ANSI, ASCII or HTML.

Rendered Columns

You can choose which columns are rendered by setting options when creating a Data::HexDump::Range object. The default rendering includes the following

  RANGE_NAME OFFSET CUMULATIVE_OFFSET HEX_DUMP ASCII_DUMP

which corresponds to the object below:

  Data::HexDump::Range->new
        (
        FORMAT => 'ANSI',
        COLOR => 'cycle',
        
        ORIENTATION => 'horizontal',
        
        DISPLAY_RANGE_NAME => 1 ,
        
        DISPLAY_OFFSET  => 1 ,
        OFFSET_FORMAT => 'hex',
        
        DISPLAY_HEX_DUMP => 1,
        DISPLAY_ASCII_DUMP => 1,
        
        DATA_WIDTH => 16,
        ) ;

If you decided that you wanted the binary data to be showed in decimal instead for hexadecimal, you' set DISPLAY_HEX_DUMP => 0 and DISPLAY_DEC_DUMP => 1. See new for all the possible options. Most option are also available from the command line utility hdr.

Orientation

The examples below show the output of the following command:

  $>hdr -r 'magic cookie,12:padding, 32:header,5:data, 20:extra data,#:header,5:data,40:footer,4' -col -o ver -display_ruler lib/Data/HexDump/Range.pm

Vertical orientation

In this orientation mode, each range displayed on a separate line.

  RANGE_NAME       OFFSET   CUMULATI HEX_DUMP                                         ASCII_DUMP        
                                     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345  
  magic cookie     00000000 00000000 0a 70 61 63 6b 61 67 65 20 44 61 74              .package Dat     
  padding          0000000c 00000000                                     61 3a 3a 48              a::H 
  padding          00000010 00000004 65 78 44 75 6d 70 3a 3a 52 61 6e 67 65 20 3b 0a  exDump::Range ;. 
  padding          00000020 00000014 0a 75 73 65 20 73 74 72 69 63 74 3b              .use strict;     
  header           0000002c 00000000                                     0a 75 73 65              .use 
  header           00000030 00000004 20                                                                
  data             00000031 00000000    77 61 72 6e 69 6e 67 73 20 3b 0a 75 73 65 20   warnings ;.use  
  data             00000040 0000000f 43 61 72 70 20                                   Carp             
  "extra data" 
  header           00000045 00000000                3b 0a 0a 42 45                         ;..BE       
  data             0000004a 00000000                               47 49 4e 20 0a 7b            GIN .{ 
  data             00000050 00000006 0a 0a 75 73 65 20 53 75 62 3a 3a 45 78 70 6f 72  ..use Sub::Expor 
  data             00000060 00000016 74 65 72 20 2d 73 65 74 75 70 20 3d 3e 20 0a 09  ter -setup => .. 
  data             00000070 00000026 7b 0a                                            {.               
  footer           00000072 00000000       09 65 78 70                                  .exp

In colors:

Horizontal orientation

In this mode, the data are packed together in the dump

 OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_NAME
          0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345
 00000000 0a 70 61 63 6b 61 67 65 20 44 61 74 61 3a 3a 48  .package Data::H magic cookie, padding,
 00000020 65 78 44 75 6d 70 3a 3a 52 61 6e 67 65 20 3b 0a  exDump::Range ;. padding,
 00000030 0a 75 73 65 20 73 74 72 69 63 74 3b 0a 75 73 65  .use strict;.use padding, header,
 00000050 20 77 61 72 6e 69 6e 67 73 20 3b 0a 75 73 65 20   warnings ;.use  header, data,
 00000070 43 61 72 70 20 3b 0a 0a 42 45 47 49 4e 20 0a 7b  Carp ;..BEGIN .{ data, "extra data", header, data,
 000000a0 0a 0a 75 73 65 20 53 75 62 3a 3a 45 78 70 6f 72  ..use Sub::Expor data,
 000000b0 74 65 72 20 2d 73 65 74 75 70 20 3d 3e 20 0a 09  ter -setup => .. data,
 000000c0 7b 0a 09 65 78 70                                {..exp           data, footer,

In colors:

Range definition

  my $simple_range = ['magic cookie', 12, 'red'] ;

Ranges are Array references containing two to four elements:

name - a string
size - an integer or a format - a string
color - a string or undef
user information - a very short string descibing the range or undef

Any of the elements can be replaced by a subroutine reference. See "Dynamic range definition" below.

You can also pass the ranges as a string. The hdr command line range dumper that was installed by this module uses the string format.

Example:

 $>hdr --col -display_ruler -o ver -r 'header,12:name,10:magic,2:offset,4:BITMAP,4,bright_yellow:ff,x2b2:fx,b32:f0,b16:field,x8b8:field2, b17:footer,20' my_binary

Size field format

The size field is used to defined if the range is a normal range, a comment, a bitfield or a skip range. The formats are a s follows:

                  format                          range example
                  
  normal range => integer                         header, 4, bright_blue
  comment      => #                               data section start, # 
  extra header => @                               header, @, red 
  bitfield     => [XInteger][xInteger]bInteger    bitfield, X2x4b4         # X is byte offset, x is bit offset
  skip range   => xInteger                        boring, X256,, comment

Note that the integer part can be a hexadecimal value starting with 0x

Coloring

Ranges and ranges names are displayed according to the color field in the range definition.

The color definition is one of:

A user defined color name found in COLOR_NAMES (see new)
An ansi color definition - 'blue on_yellow'
undef - will be repaced by a white color or picked from a cyclic color list (see COLOR in new).

Linear ranges

For simple data formats, your can put all the your range descriptions in a array:

  my $image_ranges =
        [
          ['magic cookie', 12, 'red'],
          ['size', 10, 'yellow'],
          ['data', 10, 'blue on_yellow'],
          ['timestamp', 5, 'green'],
        ] ;

Structured Ranges

  my $data_range = # definition to re-use
        [
          ['data header', 5, 'blue on_yellow'],
          ['data', 100, 'blue'],
        ] ;
                        
  my $structured_range = 
        [
          [
            ['magic cookie', 12, 'red'],
            ['padding', 88, 'yellow'],
            $data_range, 
          ],
                
          [
            ['extra data', 12, undef],
            [
              $data_range, 
              ['footer', 4, 'yellow on_red'],
            ]
          ],
        ]

Comment ranges

If the size of a range is the string '#', the whole range is considered a comment

  my $range_defintion_with_comments = 
        [
          ['comment text', '#', 'optional color for meta range'],
          ['magic cookie', 12, 'red'],
          ['padding', 88, 'yellow'],
            
          [
            ['another comment', '#'],
            ['data header', 5, 'blue on_yellow'],
            ['data', 100, 'blue'],
          ],
        ] ;

Extra header

If the size of a range is the string '@', and extra header is inserted in the output. This is useful when you have very long output and want an extra header.

Bitfields

Bitfields can be up to 32 bits long and can overlap each other. Bitfields are applied on the previously defined range.

In the example below, bitfields ff, fx, f0 are extracted form the data defined by the BITMAP range.

                 .------------.                      .--------------.
 .---.           | data range |                      | data hexdump |
 | b |           '------------'                      '--------------'
 | i |                  |                                    |
 | t |     BITMAP  <----'   00000000 00000000 0a 70 61 63 <--'                                 .pac           
 | f |   ^ .ff              02 .. 03          -- -- -- 00    ----------------------------00--  .bitfield: ---.
 | i |---> .fx              00 .. 31          0a 70 61 63    00001010011100000110000101100011  .bitfield: .pac
 | e |   v .f0              00 .. 15          -- -- 61 63    ----------------0110000101100011  .bitfield: --ac
 | l |                         ^                    ^                     ^                          ^
 | d |                         |                    |                     |                          |
 | s |             .----------------------.-------------------.----------------------.    .---------------------.
 '---'             | start bit .. end bit | bitfields hexdump | bitfield binary dump |    | bitfield ascci dump |
                   '----------------------'-------------------'----------------------'    '---------------------'

The the format definiton is: an optional "x (for offset) + offset" + "b (for bits) + number of bits". Eg: x8b8 second byte in MYDATA.

An example output containing normal data and bifields dumps using the comand below.

  $>hdr  -r 'header,12:BITMAP,4,bright_yellow:ff,x2b2:fx,b32:f0,b16:footer,16' -o ver file_name

By default bitfields are not displayed in horizontal mode.

Skip ranges

If the size format is 'X' + number, that number of bytes is skipped from the data. Data::HexDump::Range will display the skip range in the dump but not the data.

In the command below, the range 'skip' removes 32 bytes from the display. '>>' is prepended to the range name.

Command: hdr -r 'magic cookie, 5 :other,37 :bf,b8 :skip,X32,, I skipped :more, 20' -rul -col -o ver

  RANGE_NAME       OFFSET   CUMULATI HEX_DUMP                                         ASCII_DUMP        
                                     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345  
  magic cookie     00000000 00000000 44 61 74 61 3a                                   Data:            
  other            00000005 00000000                3a 48 65 78 44 75 6d 70 3a 3a 52       :HexDump::R 
  other            00000010 0000000b 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== 
  other            00000020 0000001b 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a                    =========.       
  .bf              00 .. 07          -- -- -- 0a    ------------------------00001010  .bitfield: ---.  
  >>skip           0000002a 00000049 00 00 00 20 bytes skipped                                         
  more             0000004a 00000000                               20 63 6f 6c 6f 72             color 
  more             00000050 00000006 2c 20 62 69 74 66 69 65 6c 64 73 20 61 6e        , bitfields an

in color:

Dynamic range definition

The whole range can be replaced by a subroutine reference or elements of the range can be replaced by a subroutine definition.

  my $dynamic_range =
        [
          [\&name, \&size, \&color, \&comment ],
          [\&define_range] # returns a range definition
        ] ;

  sub cloth_size
  {
  my ($self, $data, $used_data, $size, $range) = @_ ;
  my %types = (O => 'S', 1 => 'M', 2 => 'L',) ;
  return 'size:' . ($types{$data} // '?') ;
  }
  
  $hdr->dump([\&cloth_size, 1, 'yellow'], $data) ;

  sub cloth_size
  {
  my ($self, $data, $used_data, $size, $range) = @_ ;
  return unpack "a", $data ;
  }
  
  $hdr->dump(['data', \&get_size, 'yellow'], $data) ;

  my $flip_flop = 1 ;
  my @colors = ('green', 'red') ;
  
  sub alternate_color {$flip_flop ^= 1 ; return $colors[$flip_flop] }
  
  $hdr->dump(['data', 100, \&alternate_color], $data) ;

  sub define_range(['whole range', 5, 'on_yellow']}
  
  $hdr->dump([\&define_range], $data) ;

define_range($data, $offset)

Returns a range description for the next range to dump

Arguments - See gather

$self - A Data::HexDump::Range object
$data - Binary string - the data passed to the dump method
$offset - Integer - current offset in $data

Returns -

$range - An array reference containing a name, size and color and user_information
OR
undef - Ignore this range
$comment - String - an optional comment that will be displayed if DUMP_RANGE_DESCRIPTION is set.

Note this is, very, different from "User defined range generator" below.

User defined range generator

A subroutine reference can be passed as a range definition. The subroutine will be called repetitively till the data is exhausted or the subroutine returns undef.

  sub my_parser 
        {
        my ($data, $offset) = @_ ;
        
        my $first_byte = unpack ("x$offset C", $data) ;
        
        $offset < length($data)
                ?  $first_byte == ord(0)
                        ? ['from odd', 5, 'blue on_yellow']
                        : ['from even', 3, 'green']
                : undef ;
        }
  
  my $hdr = Data::HexDump::Range->new() ;
  print $hdr->dump(\&my_parser, '01' x 50) ;

my_parser($data, $offset)

Returns a range description for the next range to dump

Arguments - See gather

$self - A Data::HexDump::Range object
$data - Binary string - the data passed to the dump method
$offset - Integer - current offset in $data

Returns -

$range - An array reference containing a name, size and color
OR
undef - Done parsing

EXAMPLES

See hdr_examples.pod in the distribution.

SUBROUTINES/METHODS

new(NAMED_ARGUMENTS)

Create a Data::HexDump::Range object.

  my $hdr = Data::HexDump::Range->new() ; # use default setup
  
  my $hdr = Data::HexDump::Range->new
                (
                FORMAT => 'ANSI'|'ASCII'|'HTML',
                COLOR => 'bw' | 'cycle',
                OFFSET_FORMAT => 'hex' | 'dec',
                DATA_WIDTH => 16 | 20 | ... ,
                DISPLAY_RANGE_NAME => 1 ,
                MAXIMUM_RANGE_NAME_SIZE => 16,
                DISPLAY_COLUMN_NAMES => 0,
                DISPLAY_RULER => 0,
                DISPLAY_OFFSET  => 1 ,
                DISPLAY_CUMULATIVE_OFFSET  => 1 ,
                DISPLAY_ZERO_SIZE_RANGE_WARNING => 1,
                DISPLAY_ZERO_SIZE_RANGE => 1,
                DISPLAY_RANGE_SIZE => 1,
                DISPLAY_ASCII_DUMP => 1 ,
                DISPLAY_HEX_DUMP => 1,
                DISPLAY_DEC_DUMP => 1,
                COLOR_NAMES => {},
                ORIENTATION => 'horizontal',
                ) ;

Arguments - All arguments are optional. Default values are listed below.

NAME - String - Name of the Data::HexDump::Range object, set to 'Anonymous' by default
INTERACTION - Hash reference - Set of subs that are used to display information to the user
Useful if you use Data::HexDump::Range in an application without terminal.
VERBOSE - Boolean - Display information about the creation of the object. Default is false
DUMP_ORIGINAL_RANGE_DESCRIPTION - Boolean - Diplays the un-processed range descritption.
With DUMP_RANGE_DESCRIPTION, this fields can be used to peek into what range descriptions the module get and how they are transformed into the format that is internally used by the module. These are for debugging purpose and you should normally not need to used them.
```
 Original range description
 |- 0 = 'Data'
 |- 1 = '128'
 |- 2 = undef
 `- 3 = undef
 
 Original range description
 |- 0 = CODE(0x1dc5230)
 |- 1 = undef
 |- 2 = undef
 `- 3 = undef
    
```

DUMP_RANGE_DESCRIPTION - Boolean - Diplays the processed range descritption in plain text before the dump

 128->26:Data
 |- COLOR = undef
 |- DATA = '_blah_blah_blah_blah_blah[\n]'
 |- IS_BITFIELD = '0'
 |- IS_COMMENT = '0'
 |- IS_SKIP = '0'
 |- NAME = '128->26:Data'
 |- OFFSET = '20'
 |- unpack format = 'x20 a26'
 `- USER_INFORMATION = undef

FORMAT - String - format of the dump string generated by Data::HexDump::Range.
Default is ANSI which allows for colors. Other formats are 'ASCII' and 'HTML'.
COLOR - String 'bw' or 'cycle'.
Ranges for which no color has been defined, in 'ANSI' or 'HTML' format mode, will be rendered in black and white or with a color picked from a cyclic color list. Default is 'bw'.
OFFSET_FORMAT - String - 'hex' or 'dec'
If set to 'hex', the offset will be displayed in base 16. When set to 'dec' the offset is displayed in base 10. Default is 'hex'.
DATA_WIDTH - Integer - Number of elements displayed per line. Default is 16.
DISPLAY_RANGE_NAME - Boolean - If set, range names are displayed in the dump.
MAXIMUM_RANGE_NAME_SIZE - Integer - maximum size of a range name (horizontal mode). Default size is 16.
DISPLAY_COLUMN_NAMES - Boolean - If set, the column names are displayed. Default false
DISPLAY_RULER - Boolean - if set, a ruler is displayed above the dump, Default is false
DISPLAY_OFFSET - Boolean - If set, the offset column is displayed. Default true
DISPLAY_CUMULATIVE_OFFSET - Boolean - If set, the cumulative offset column is displayed in 'vertical' rendering mode. Default is true
OFFSET_START - Integer - value added to the offset.
DISPLAY_ZERO_SIZE_RANGE - Boolean - if set, ranges that do not consume data are displayed. default is true
DISPLAY_ZERO_SIZE_RANGE_WARNING - Boolean - if set, a warning is emitted if ranges that do not consume data. Default is true
DISPLAY_COMMENT_RANGE - Boolean - if set, comment ranges are displayed. default is true
DISPLAY_RANGE_SIZE - Bolean - if set the range size is prepended to the name. Default false
DISPLAY_ASCII_DUMP - Boolean - If set, the ASCII representation of the binary data is displayed. Default is true
DISPLAY_HEX_DUMP - Boolean - If set, the hexadecimal dump column is displayed. Default is true
DISPLAY_DEC_DUMP - Boolean - If set, the decimall dump column is displayed. Default is false
DISPLAY_BITFIELD_SOURCE - Boolean - if set an extra column indicating the source of bitfields is displayed
MAXIMUM_BITFIELD_SOURCE_SIZE - Integer - maximum size of the bifield source column
DISPLAY_USER_INFORMATION - Boolean - if set an extra column displaying user supplied information is shown
MAXIMUM_USER_INFORMATION_SIZE - Integer - maximum size of theuser information column
DISPLAY_BITFIELDS - Boolean - if set the bitfields are displayed
BIT_ZERO_ON_LEFT - Boolean - if set the bit of index 0 is on left growing to the right. Default false

COLOR_NAMES - A hash reference

  {
  ANSI =>
        {
        header => 'yellow on_blue',
        data => 'yellow on_black',
        },
        
  HTML =>
        {
        header => 'FFFF00 0000FF',
        data => 'FFFF00 000000',
        },
  }

ORIENTATION - String - 'vertical' or 'horizontal' (the default).

Returns - Nothing

Exceptions - Dies if an unsupported option is passed.

gather($range_description, $data, $offset, $size)

Dump the data, up to $size, according to the description. The data dump is kept in the object so you can merge multiple gathered dumps and get a single rendering.

  $hdr->gather($range_description, $data, $offset, $size)
  $hdr->gather($range_description, $more_data)
  
  print $hdr->dump_gathered() ;

Arguments

$range_description - See "Range definition"
$data - A string - binary data to dump
$offset - dump data from offset

•: undef - start from first byte

•: $size - amount of data to dump

undef - use range description
CONSUME_ALL_DATA - apply range descritption till all data is consumed

Returns - An integer - the number of processed bytes

Exceptions - See _gather

dump_gathered()

Returns the dump string for the gathered data.

  $hdr->gather($range_description, $data, $size)
  $hdr->gather($range_description, $data, $size)
  
  print $hdr->dump_gathered() ;

Arguments - None

Returns - A string - the binary data formated according to the rnage descriptions

Exceptions - None

dump($range_description, $data, $offset, $size)

Dump the data, up to $size, according to the description

Arguments - See gather

Returns - A string - the formated dump

Exceptions - dies if the range description is invalid

get_dump_and_consumed_data_size($range_description, $data, $offset, $size)

Dump the data, from $offset up to $size, according to the $range_description

Arguments - See gather

Returns -

A string - the formated dump
An integer - the number of bytes consumed by the range specification

Exceptions - dies if the range description is invalid

reset()

Clear the gathered dump

Arguments - None

Returns - Nothing

Exceptions - None

BUGS AND LIMITATIONS

None so far.

AUTHOR

        Nadim ibn hamouda el Khemir
        CPAN ID: NKH
        mailto: nadim@cpan.org

COPYRIGHT AND LICENSE

This program is free software; you can redistribute it and/or modify it under the terms of either:

the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version, or
the Artistic License version 2.0.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Data::HexDump::Range

You can also look for information at:

AnnoCPAN: Annotated CPAN documentation
<http://annocpan.org/dist/Data-HexDump-Range>
RT: CPAN's request tracker
Please report any bugs or feature requests to L <bug-data-hexdump-range@rt.cpan.org>.

We will be notified, and then you'll automatically be notified of progress on your bug as we make changes.
Search CPAN
<http://search.cpan.org/dist/Data-HexDump-Range>