CW%bom2encMaps Byte Order marks to their encodings.
The keys of this hash are strings which represent the BOMs, the values are their encodings, in a format which is understood by Encode
The encodings represented in this hash are: UTF-8, UTF-16BE, UTF-16LE, UTF-32BE and UTF-32LE
CW%enc2bomA reverse-lookup hash for bom2enc, with a few aliases used in Encode, namely utf8, iso-10646-1 and UCS-2.
$encoding = open_bom(HANDLE, $filename, $default_mode) ($encoding, $spill) = open_bom(HANDLE, $filename, $default_mode)
opens HANDLE for reading on $filename, setting the mode to the appropriate encoding for the BOM stored in the file.
On failure, a fatal error is raised, see the DIAGNOSTICS section for details on how to catch these. This is in order to allow the return value(s) to be used for other purposes.
If the file doesnt contain a BOM, $default_mode is used instead. Hence:
open_bom(FH, my_file.txt, :utf8)
In the absence of a $default_mode argument, the following 2 calls should be equivalent:
open_bom(FH, no_bom.txt); open(FH, <, no_bom.txt);
If an undefined value is passed as the handle, a symbol will be generated for it like open() does:
# create filehandle on the fly $enc = open_bom(my $fh, $filename, :utf8); $line = <$fh>;
The filehandle will be cued up to read after the BOM. Unseekable files (e.g. fifos) will cause croaking, unless called in list context to catch spillage from the handle. Any spillage will be automatically decoded from the encoding, if found.
$enc = defuse(FH); ($enc, $spill) = defuse(FH);
Any uncaptured spillage will be silently lost. If the handle is unseekable, use list context to avoid data loss.
If no BOM is found, the mode will be unaffected.
$unicode_string = decode_from_bom($string, $default, $check) ($unicode_string, $encoding) = decode_from_bom($string, $default, $check)
Reads a BOM from the beginning of $string, decodes $string (minus the BOM) and returns it to you as a perl unicode string.
if $string doesnt have a BOM, $default is used instead.
If theres no BOM and no default, the original string is returned and encoding is .
$encoding = get_encoding_from_filehandle(HANDLE) ($encoding, $spillage) = get_encoding_from_filehandle(HANDLE)
Returns the encoding found in the given filehandle.
The handle should be opened in a non-unicode way (e.g. mode <:bytes) so that the BOM can be read in its natural state.
After calling, the handle will be set to read at a point after the BOM (or at the beginning of the file if no BOM was found)
If called in scalar context, unseekable handles cause a croak().
If called in list context, unseekable handles will be read byte-by-byte and any spillage will be returned. See get_encoding_from_stream()
($encoding, $spillage) = get_encoding_from_stream(*FH);
Read a BOM from an unrewindable source. This means reading the stream one byte at a time until either a BOM is found or every possible BOM is ruled out. Any non-BOM bytes read from the handle will be returned in $spillage.
If a BOM is found and the spillage contains a partial character (judging by the expected character width for the encoding) more bytes will be read from the handle to ensure that a complete character is returned.
Spillage is always in bytes, not characters.
This function is less efficient than get_encoding_from_filehandle, but should work just as well on a seekable handle as on an unseekable one.
($encoding, $offset) = get_encoding_from_bom($string)
If there is no BOM, an empty string is returned and $offset is zero.
To get the data from the string, the following should work:
File::BOM can be used as a PerlIO::via interface.
This method is less prone to errors on non-seekable files as spillage is incorporated into an internal buffer, but it doesnt give you any information about the encoding being used, or indeed whether or not a BOM was present.
There are a few known problems with this interface, especially surrounding seek() and tell(), please see the BUGS section for more details about this.
The via(File::BOM) layer must be added before the handle is read from, otherwise any BOM will be missed. If there is no BOM, no decoding will be done.
Add the via(File::BOM) layer on top of a unicode encoding layer to print a BOM at the start of the output file. This needs to be done before any data is written. The BOM is written as part of the first print command on the handle, so if you dont print anything to the handle, you wont get a BOM.
There is a Wide character in print warning generated when the via(File::BOM) layer doesnt receive utf8 on writing. This glitch was resolved in perl version 5.8.7, but if your perl version is older than that, youll need to make sure that the via(File::BOM) layer receives utf8 like this:
Seeking with SEEK_SET results in an offset equal to the length of any detected BOM being applied to the position parameter. Thus:
# Seek to end of BOM (not start of file!) seek(FILE_BOM_HANDLE, 0, SEEK_SET)
In order to work correctly with seek(), tell() also returns a postion adjusted by the length of the BOM.
The following exceptions are raised via croak()
o Couldnt read <filename>: $!
open_bom() couldnt open the given file for reading
o Couldnt set binmode of handle opened on <filename> to <mode>: $!
open_bom() couldnt set the binmode of the handle
o No string
decode_from_bom called on an undefined value
o Unseekable handle: $!
get_encoding_from_filehandle() or open_bom() called on an unseekable file or handle in scalar context.
o Couldnt read from handle: $!
_get_encoding_seekable() couldnt read the handle. This function is called from get_encoding_from_filehandle(), defuse() and open_bom()
o Couldnt reset read position: $!
_get_encoding_seekable couldnt seek to the position after the BOM.
o Couldnt read byte: $!
get_encoding_from_stream couldnt read from the handle. This function is called from get_encoding_from_filehandle() and open_bom() when the handle or file is unseekable.
Older versions of PerlIO::via have a few problems with writing, see above.
The current version of PerlIO::via has limitations with regard to seek and tell, currently only line-wise seek and tell are supported by this module. If read() is used to read partial lines, tell() will still give the position of the end of the last line read.
Under windows, tell() seems to return erroneously when reading files with unix line endings.
Matt Lawrence <firstname.lastname@example.org>
With thanks to Mark Fowler and Steve Purkis for additional tests and advice.
Copyright 2005 Matt Lawrence, All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
|perl v5.20.3||FILE::BOM (3)||2016-04-03|