GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages
PAR::Tutorial(3) User Contributed Perl Documentation PAR::Tutorial(3)

PAR::Tutorial - Cross-Platform Packaging and Deployment with PAR

This is a tutorial on PAR, first appeared at the 7th Perl Conference. The HTML version of this tutorial is available online as <http://search.cpan.org/perldoc?PAR::Tutorial>

 % sshnuke.pl 10.2.2.2 -rootpw="Z1ON0101"
 Perl v5.6.1 required--this is only v5.6.0, stopped at sshnuke.pl line 1.
 BEGIN failed--compilation aborted at sshnuke.pl line 1.
  • Q: "Help! I can't run your program!"
  • A1: Install Perl & "perl -MCPAN -e'install(...)'"
  • How do we know which modules are needed?
  • New versions of CPAN modules may break "sshnuke.pl"
A2: Install Perl & "tar zxf my_perllib.tgz"
Possibly overwriting existing modules; not cross-platform at all
A3: Use the executable generated by "perlcc sshnuke.pl"
Impossible to debug; "perlcc" usually does not work anyway

Do what JAR (Java Archive) does for Perl
  • Aggregates modules, scripts and other files into a Zip file
  • Easy to generate, update and extract
  • Version consistency: solves forward-compatibility problems
  • Developed by community: "par@perl.org"
PAR files can be packed into self-contained scripts
  • Automatically scans perl script for dependencies
  • Bundles all necessary 3rd-party modules with it
  • Requires only core Perl to run on the target machine
  • PAR also comes with "pp", the Perl Packager:

     % pp -o sshnuke.exe sshnuke.pl # stand-alone executable!
        

  • PAR files are just Zip files with modules in it
  • Any Zip tools can generate them:

     % zip foo.par Hello.pm World.pm        # pack two modules
     % zip -r bar.par lib/          # grab all modules in lib/
        
  • To load modules from PAR files:

     use PAR;
     use lib "foo.par";             # the .par part is optional
     use Hello;
        
  • This also works:

     use PAR "/home/mylibs/*.par";  # put all of them into @INC
     use Hello;
        

  • Use "par.pl" to run files inside a PAR archive:

     % par.pl foo.par               # looks for 'main.pl' by default
     % par.pl foo.par test.pl       # runs script/test.pl in foo.par
        
  • Same thing, with the stand-alone "parl" or "parl.exe":

     % parl foo.par                 # no perl or PAR.pm needed!
     % parl foo.par test.pl         # ditto
        
  • The PAR loader can prepend itself to a PAR file:
  • "-b" bundles non-core modules needed by "PAR.pm":

     % par.pl -b -O./foo.pl foo.par # self-contained script
        
  • "-B" bundles core modules in addition to "-b":

     % parl -B -O./foo.exe foo.par  # self-contained binary
        

  • Recursively scan dependencies with "scandeps.pl":

     % scandeps.pl sshnuke.pl
     # Legend: [C]ore [X]ternal [S]ubmodule [?]NotOnCPAN
     'Crypt::SSLeay'       => '0', #  X   #
     'Net::HTTP'           => '0', #      #
     'Crypt::SSLeay::X509' => '0', # S    # Crypt::SSLeay
     'Net::HTTP::Methods'  => '0', # S    # Net::HTTP
     'Compress::Zlib'      => '0', #  X   # Net::HTTP::Methods
        
  • Scan an one-liner, list all involved files:

     % scandeps.pl -V -e "use Dynaloader;"
     ...
     # auto/DynaLoader/dl_findfile.al [autoload]
     # auto/DynaLoader/extralibs.ld [autoload]
     # auto/File/Glob/Glob.bs [data]
     # auto/File/Glob/Glob.so [shared]
     ...
        

  • Combines scanning, zipping and loader-embedding:

     % pp -o out.exe src.pl         # self-contained .exe
     % out.exe                      # runs anywhere on the same OS
        
  • Bundle additional modules:

     % pp -o out.exe -M CGI src.pl  # pack CGI + its dependencies, too
        
  • Pack one-liners:

     % pp -o out.exe -e 'print "Hi!"'   # turns one-liner into executable
        
  • Generate PAR files instead of executables:

     % pp -p src.pl                 # makes 'source.par'
     % pp -B -p src.pl              # include core modules
        

Command-line options are almost identical to "perlcc"'s
Also supports "gcc"-style long options:

 % pp --gui --verbose --output=out.exe src.pl
    
  • Small initial overhead; no runtime overhead
  • Dependencies are POD-stripped before packing
  • Loads modules directly into memory on demand
  • Shared libraries (DLLs) are extracted with File::Temp
  • Works on Perl 5.6.0 or above
  • Tested on Win32 (VC++ and MinGW), FreeBSD, NetBSD, Linux, MacOSX, Cygwin, AIX, Solaris, HP-UX, Tru64...

  • A common question:

     > I have used pp to make several standalone applications which work
     > great, the only problem is that for each executable that I make, I am
     > assuming the parl.exe is somehow bundled into the resulting exe.
        
  • The obvious workaround:

     You can ship parl.exe by itself, along with .par files built
     by "pp -p", and run those PAR files by associating them to parl.exe.
        
  • On platforms that have "ln", there is a better solution:

     % pp --output=a.out a.pl b.pl  # two scripts in one!
     % ln a.out b.out               # symlink also works
     % ./a.out                      # runs a.pl
     % ./b.out                      # runs b.pl
        

  • Of course, there is no cross-platform binary format
  • Pure-perl PAR packages are cross-platform by default
  • However, XS modules are specific to Perl version and platform
  • Multiple versions of a XS module can co-exist in a PAR file
  • Suppose we need "out.par" on both Win32 and Finix:

     C:\> pp --multiarch --output=out.par src.pl
     ...copy src.pl and out.par to a Finix machine...
     % pp --multiarch --output=out.par src.pl
        
  • Now it works on both platforms:

     % parl out.par                 # runs src.pl
     % perl -MPAR=out.par -e '...'  # uses modules inside out.par
        

  • Modules can reside in several directories:

     /                      # casual packaging only
     /lib/                  # standard location
     /arch/                 # for creating from blib/ 
     /i386-freebsd/         # i.e. $Config{archname}
     /5.8.0/                # i.e. Perl version number
     /5.8.0/i386-freebsd/   # combination of the two above
        
  • Scripts are stored in one of the two locations:

     /                      # casual packaging only
     /script/               # standard location
        
  • Shared libraries may be architecture- or perl-version-specific:

     /shlib/(5.8.0/)?(i386-freebsd/)?
        
  • PAR files may recursively contain other PAR files:

     /par/(5.8.0/)?(i386-freebsd/)?
        

MANIFEST
  • Index of all files inside PAR
  • Can be parsed with "ExtUtils::Manifest"
META.yml
  • Dependency, license, runtime options
  • Can be parsed with "YAML"
SIGNATURE
  • OpenPGP-signed digital signature
  • Can be parsed and verified with "Module::Signature"

This is not meant to be a flame
All three maintainers have contributed to PAR directly; I'm grateful
perlcc
  • "The code generated in this way is not guaranteed to work... Use for production purposes is strongly discouraged." (from perldoc perlcc)
  • Guaranteed to not work is more like it
PerlApp / Perl2exe
  • Expensive: Need to pay for each upgrade
  • Non-portable: Only available for limited platforms
  • Proprietary: Cannot extend its features or fix bugs
  • Obfuscated: Vendor and black-hats can see your code, but you can't
  • Inflexible: Does not work with existing Perl installations

  • The URL of "MANIFEST" inside "/home/autrijus/foo.par":

     jar:file:///home/autrijus/foo.par!/MANIFEST
        
  • Open it in a Gecko browser (e.g. Netscape 6+) with Javascript enabled:
  • No needed to unzip anything; just click on files to view them

Static, machine-readable distribution metadata
Supported by "Module::Build", "ExtUtils::MakeMaker", "Module::Install"
  • A typical "pp"-generated "META.yml" looks like this:

     build_requires: {}
     conflicts: {}
     dist_name: out.par
     distribution_type: par
     dynamic_config: 0
     generated_by: 'Perl Packager version 0.03'
     license: unknown
     par:
       clean: 0
       signature: ''
       verbatim: 0
       version: 0.68
        
  • The "par:" settings controls its runtime behavior

OpenPGP clear-signed manifest with SHA1 digests
Supported by "Module::Signature", "CPANPLUS" and "Module::Build"
  • A typical "SIGNATURE" looks like this:

     -----BEGIN PGP SIGNED MESSAGE-----
     Hash: SHA1
    
     SHA1 8a014cd6d0f6775552a01d1e6354a69eb6826046 AUTHORS
     ...
     -----BEGIN PGP SIGNATURE-----
     ...
     -----END PGP SIGNATURE-----
        
  • Use "pp" and "cpansign" to work with signatures:

     % pp -s -o foo.par bar.pl      # make and sign foo.par from bar.pl
     % cpansign -s foo.par  # sign this PAR file
     % cpansign -v foo.par  # verify this PAR file
        

Framework for self-contained Web applications
  • Similar to Java's "Web Application Archive" (WAR) files
  • Works with mod_perl 1.x or 2.x
A complete web application inside a ".par" file
  • Apache configuration, static files, Perl modules...
  • Supports Static, Registry and PerlRun handlers
  • Can also load all PARs under a directory
One additional special file: "web.conf"

 Alias /myapp/cgi-perl/ ##PARFILE##/
 <Location /myapp/cgi-perl>
     Options +ExecCGI
     SetHandler perl-script
     PerlHandler Apache::PAR::Registry
 </Location>
    

  • First, make a "hondah.par" from an one-liner:

     # use the "web.conf" from the previous slide
     % pp -p -o hondah.par -e 'print "Hon Dah!\n"' \
          --add web.conf
     % chmod a+x hondah.par
        
  • Add this to "httpd.conf", then restart apache:

     <IfDefine MODPERL2>
     PerlModule Apache2
     </IfDefine>
     PerlAddVar PARInclude /home/autrijus/hondah.par
     PerlModule Apache::PAR
        
  • Test it out:

     % GET http://localhost/myapp/cgi-perl/main.pl
     Hon Dah!
        
  • Instant one-liner web application that works!

  • With LWP installed, your can use remote PAR files:

     use PAR;
     use lib 'http://aut.dyndns.org/par/DBI-latest.par';
     use DBI;    # always up to date!
        
  • Modules are now cached under $ENV{PAR_GLOBAL_TEMP}
  • Auto-updates with "LWP::Simple::mirror"
  • Download only if modified
  • Safe for offline use after the first time
  • May use "SIGNATURE" to prevent DNS-spoofing
Makes large-scale deployment a breeze
  • Upgrades from a central location
  • No installers needed

Also known as source-hiding techniques
  • It is not encryption
  • Offered by PerlApp, Perl2Exe, Stunnix...
Usually easy to defeat
  • Take optree dump from memory, feed to "B::Deparse"
  • If you just want to stop a casual "grep", "deflate" already works
PAR now supports pluggable input filters with "pp -f"
  • Bundled examples: Bleach, PodStrip and PatchContent
  • True encryption using "Crypt::*"
  • Or even _product activation_ over the internet
Alternatively, just keep core logic in your server and use RPC

  • To get the host archive from a packed program:

     my $zip = PAR::par_handle($0); # an Archive::Zip object
     my $content = $zip->contents('MANIFEST');
        
  • Same thing, but with "read_file()":

     my $content = PAR::read_file('MANIFEST');
        
  • Loaded PAR files are stored in %PAR::LibCache:

     use PAR '/home/mylibs/*.par';
     while (my ($filename, $zip) = each %PAR::LibCache) {
         print "[$filename - MANIFEST]\n";
         print $zip->contents('MANIFEST');
     }
        

  • GUI toolkits often need to link with shared libraries:

     # search for libncurses under library paths and pack it
     % pp -l ncurses curses_app.pl  # same for Tk, Wx, Gtk, Qt...
        
  • Use "pp --gui" on Win32 to eliminate the console window:

     # pack 'src.pl' into a console-less 'out.exe' (Win32 only)
     % pp --gui -o out.exe src.pl
        
  • "Can't locate Foo/Widget/Bar.pm in @INC"?
  • Some toolkits (notably Tk) autoloads modules without "use" or "require"
  • Hence "pp" and "Module::ScanDeps" may fail to detect them
  • Tk problems mostly fixed by now, but other toolkits may still break
  • You can work around it with "pp -M" or an explicit "require"
  • Or better, send a short test-case to "par@perl.org" so we can fix it

Installing XS extensions from CPAN was difficult
  • Some platforms do not come with a compiler (Win32, MacOSX...)
  • Some headers or libraries may be missing
  • PAR.pm itself used to suffer from both problems
  • ...but not anymore -- "Module::Install" to the rescue!

     # same old Makefile.PL, with a few changes
     use inc::Module::Install;      # was "use ExtUtils::MakeMaker;"
     WriteMakefile( ... );          # same as the original
     check_nmake();                 # make sure the user have nmake
     par_base('AUTRIJUS');          # your CPAN ID or a URL
     fetch_par() unless can_cc();   # use precompiled PAR only if necessary
        
  • Users will not notice anything, except now it works
  • Of course, you still need to type "make par" and upload the precompiled package
  • PAR users can also install it directly with "parl -i"

Additional resources
  • Mailing list: "par@perl.org"
  • Subscribe: Send a blank email to "par-subscribe@perl.org"
  • List archive: <http://nntp.x.perl.org/group/perl.par>
  • PAR::Intro: <http://search.cpan.org/dist/PAR/lib/PAR/Intro.pod>
  • Apache::PAR: <http://search.cpan.org/dist/Apache-PAR/>
  • Module::Install: <http://search.cpan.org/dist/Module-Install/>
Any questions?

Here begins the scary part
  • Grues, Dragons and Jabberwocks abound...
  • You are going to learn weird things about Perl internals
PAR invokes four areas of Perl arcana:
  • @INC code references
  • On-the-fly source filtering
  • Overriding "DynaLoader::bootstrap()" to handle XS modules
  • Making self-bootstrapping binary executables
The first two only works on 5.6 or later
  • DynaLoader and %INC are there since Perl 5 was born
  • PAR currently needs 5.6, but a 5.005 port is possible

On 1999-07-19, Ken Fox submitted a patch to P5P
  • To _enable using remote modules_ by putting hooks in @INC
  • It's accepted to come in Perl 5.6, but undocumented until 5.8
  • Type "perldoc -f require" to read the nitty-gritty details
  • Coderefs in @INC may return a fh, or undef to 'pass':

     push @INC, sub {
         my ($coderef, $filename) = @_;  # $coderef is \&my_sub
         open my $fh, "wget ftp://example.com/$filename |";
         return $fh;        # using remote modules, indeed!
     };
        
  • Perl 5.8 let you open a file handle to a string, so we just use that:

            open my $fh, '<', \($zip->memberNamed($filename)->contents);
            return $fh;
        
  • But Perl 5.6 does not have that, and I don't want to use temp files...

... Undocumented features to the rescue!
  • It turns out that @INC hooks can return two values
  • The first is still the file handle
  • The second is a code reference for line-by-line source filtering!
This is how "Acme::use::strict::with::pride" works:

 # Force all modules used to use strict and warnings
 open my $fh, "<", $filename or return;
 my @lines = ("use strict; use warnings;\n", "#line 1 \"$full\"\n");
 return ($fh, sub {
     return 0 unless @lines;    
     push @lines, $_; $_ = shift @lines; return length $_;
 });
    

  • But we don't really have a filehandle for anything
  • Another undocumented feature saves the day!
  • We can actually omit the first return value altogether:

     # Return all contents line-by-line from the file inside PAR
     my @lines = split(
         /(?<=\n)/,
         $zip->memberNamed($filename)->contents
     );
     return (sub {
         $_ = shift(@lines);
         return length $_;
     });
        

XS modules have dynamically loaded libraries
  • They cannot be loaded as part of a zip file, so we extract them out
  • Must intercept DynaLoader's library-finding process
Module names are passed to "bootstrap" for XS loading
  • During the process, it calls "dl_findfile" to locate the file
  • So we install pre-hooks around both functions
Our "_bootstrap" just checks if the library is in PARs
If yes, extract it to a "File::Temp" temp file
The file will be automatically cleaned up when the program ends
  • It then pass the arguments to the original "bootstrap"
  • Finally, our "dl_findfile" intercepts known filenames and return it

The par script ($0) itself
May be in plain-text or native executable format
Any number of embedded files
  • Typically used to bootstrap PAR's various dependencies
  • Each section begins with the magic string "FILE"
  • Length of filename in pack('N') format and the filename (auto/.../)
  • File length in pack('N') and the file's content (not compressed)
One PAR file
Just a regular zip file with the magic string "PK\003\004"
Ending section
  • A pack('N') number of the total length of FILE and PAR sections
  • Finally, there must be a 8-bytes magic string: "\012PAR.pm\012"

All we can expect is a working perl interpreter
  • The self-contained script *must not* use any modules at all
  • But to process PAR files, we need XS modules like Compress::Zlib
Answer: bundle all modules + libraries used by PAR.pm
  • That's what the "FILE" section in the previous slide is for
  • Load modules to memory, and write object files to disk
  • Then use a local @INC hook to load them on demand
Minimizing the amount of temporary files
  • First, try to load PerlIO::scalar and File::Temp
  • Set up an END hook to unlink all temp files up to this point
  • Load other bundled files, and look in the compressed PAR section
  • This can be much easier with a pure-perl "inflate()"; patches welcome!

Any questions, please?

PAR, pp, par.pl, parl

ex::lib::zip, Acme::use::strict::with::pride

App::Packer, Apache::PAR, CPANPLUS, Module::Install

Audrey Tang <cpan@audreyt.org>

You can write to the mailing list at <par@perl.org>, or send an empty mail to <par-subscribe@perl.org> to participate in the discussion.

Please submit bug reports to <bug-par@rt.cpan.org>.

Copyright 2003, 2004, 2005, 2006 by Audrey Tang <cpan@audreyt.org>.

This document is free documentation; you can redistribute it and/or modify it under the same terms as Perl itself.

See LICENSE.

2019-07-26 perl v5.32.1

Search for    or go to Top of page |  Section 3 |  Main Index

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with ManDoc.