GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  PAR::TUTORIAL (3)

.ds Aq ’

NAME

PAR::Tutorial - Cross-Platform Packaging and Deployment with PAR

CONTENTS

SYNOPSIS

This is a tutorial on PAR, first appeared at the 7th Perl Conference. The HTML version of this tutorial is available online as <http://search.cpan.org/perldoc?PAR::Tutorial>

DESCRIPTION

    On Deploying Perl Applications



 % sshnuke.pl 10.2.2.2 -rootpw="Z1ON0101"
 Perl v5.6.1 required--this is only v5.6.0, stopped at sshnuke.pl line 1.
 BEGIN failed--compilation aborted at sshnuke.pl line 1.



o Q: Help! I can’t run your program!
o A1: Install Perl & perl -MCPAN -einstall(...)
o How do we know which modules are needed?
o New versions of CPAN modules may break sshnuke.pl
o A2: Install Perl & tar zxf my_perllib.tgz
o Possibly overwriting existing modules; not cross-platform at all
o A3: Use the executable generated by perlcc sshnuke.pl
o Impossible to debug; perlcc usually does not work anyway

    PAR, the Perl Archive Toolkit

o Do what JAR (Java Archive) does for Perl
o Aggregates modules, scripts and other files into a Zip file
o Easy to generate, update and extract
o Version consistency: solves forward-compatibility problems
o Developed by community: par@perl.org
o PAR files can be packed into self-contained scripts
o Automatically scans perl script for dependencies
o Bundles all necessary 3rd-party modules with it
o Requires only core Perl to run on the target machine
o PAR also comes with pp, the Perl Packager:



 % pp -o sshnuke.exe sshnuke.pl # stand-alone executable!



    Simple Packaging

o PAR files are just Zip files with modules in it
o Any Zip tools can generate them:



 % zip foo.par Hello.pm World.pm        # pack two modules
 % zip -r bar.par lib/          # grab all modules in lib/



o To load modules from PAR files:



 use PAR;
 use lib "foo.par";             # the .par part is optional
 use Hello;



o This also works:



 use PAR "/home/mylibs/*.par";  # put all of them into @INC
 use Hello;



    PAR Loaders

o Use par.pl to run files inside a PAR archive:



 % par.pl foo.par               # looks for main.pl by default
 % par.pl foo.par test.pl       # runs script/test.pl in foo.par



o Same thing, with the stand-alone parl or parl.exe:



 % parl foo.par                 # no perl or PAR.pm needed!
 % parl foo.par test.pl         # ditto



o The PAR loader can prepend itself to a PAR file:
o -b bundles non-core modules needed by PAR.pm:



 % par.pl -b -O./foo.pl foo.par # self-contained script



o -B bundles core modules in addition to -b:



 % parl -B -O./foo.exe foo.par  # self-contained binary



    Dependency Scanning

o Recursively scan dependencies with scandeps.pl:



 % scandeps.pl sshnuke.pl
 # Legend: [C]ore [X]ternal [S]ubmodule [?]NotOnCPAN
 Crypt::SSLeay       => 0, #  X   #
 Net::HTTP           => 0, #      #
 Crypt::SSLeay::X509 => 0, # S    # Crypt::SSLeay
 Net::HTTP::Methods  => 0, # S    # Net::HTTP
 Compress::Zlib      => 0, #  X   # Net::HTTP::Methods



o Scan an one-liner, list all involved files:



 % scandeps.pl -V -e "use Dynaloader;"
 ...
 # auto/DynaLoader/dl_findfile.al [autoload]
 # auto/DynaLoader/extralibs.ld [autoload]
 # auto/File/Glob/Glob.bs [data]
 # auto/File/Glob/Glob.so [shared]
 ...



Perl Packager: CWpp

o Combines scanning, zipping and loader-embedding:



 % pp -o out.exe src.pl         # self-contained .exe
 % out.exe                      # runs anywhere on the same OS



o Bundle additional modules:



 % pp -o out.exe -M CGI src.pl  # pack CGI + its dependencies, too



o Pack one-liners:



 % pp -o out.exe -e print "Hi!"   # turns one-liner into executable



o Generate PAR files instead of executables:



 % pp -p src.pl                 # makes source.par
 % pp -B -p src.pl              # include core modules



    How it works

o Command-line options are almost identical to perlcc’s
o Also supports gcc-style long options:



 % pp --gui --verbose --output=out.exe src.pl



o Small initial overhead; no runtime overhead
o Dependencies are POD-stripped before packing
o Loads modules directly into memory on demand
o Shared libraries (DLLs) are extracted with File::Temp
o Works on Perl 5.6.0 or above
o Tested on Win32 (VC++ and MinGW), FreeBSD, NetBSD, Linux, MacOSX, Cygwin, AIX, Solaris, HP-UX, Tru64...

    Aggregating multiple programs

o A common question:



 > I have used pp to make several standalone applications which work
 > great, the only problem is that for each executable that I make, I am
 > assuming the parl.exe is somehow bundled into the resulting exe.



o The obvious workaround:



 You can ship parl.exe by itself, along with .par files built
 by "pp -p", and run those PAR files by associating them to parl.exe.



o On platforms that have ln, there is a better solution:



 % pp --output=a.out a.pl b.pl  # two scripts in one!
 % ln a.out b.out               # symlink also works
 % ./a.out                      # runs a.pl
 % ./b.out                      # runs b.pl



    Cross-platform Packages

o Of course, there is no cross-platform binary format
o Pure-perl PAR packages are cross-platform by default
o However, XS modules are specific to Perl version and platform
o Multiple versions of a XS module can co-exist in a PAR file
o Suppose we need out.par on both Win32 and Finix:



 C:\> pp --multiarch --output=out.par src.pl
 ...copy src.pl and out.par to a Finix machine...
 % pp --multiarch --output=out.par src.pl



o Now it works on both platforms:



 % parl out.par                 # runs src.pl
 % perl -MPAR=out.par -e ...  # uses modules inside out.par



    The Anatomy of a PAR file

o Modules can reside in several directories:



 /                      # casual packaging only
 /lib/                  # standard location
 /arch/                 # for creating from blib/
 /i386-freebsd/         # i.e. $Config{archname}
 /5.8.0/                # i.e. Perl version number
 /5.8.0/i386-freebsd/   # combination of the two above



o Scripts are stored in one of the two locations:



 /                      # casual packaging only
 /script/               # standard location



o Shared libraries may be architecture- or perl-version-specific:



 /shlib/(5.8.0/)?(i386-freebsd/)?



o PAR files may recursively contain other PAR files:



 /par/(5.8.0/)?(i386-freebsd/)?



    Special files

o MANIFEST
o Index of all files inside PAR
o Can be parsed with ExtUtils::Manifest
o META.yml
o Dependency, license, runtime options
o Can be parsed with YAML
o SIGNATURE
o OpenPGP-signed digital signature
o Can be parsed and verified with Module::Signature

    Advantages over perlcc, PerlApp and Perl2exe

o This is not meant to be a flame
o All three maintainers have contributed to PAR directly; I’m grateful
o perlcc
o The code generated in this way is not guaranteed to work... Use for production purposes is strongly discouraged. (from perldoc perlcc)
o Guaranteed to not work is more like it
o PerlApp / Perl2exe
o Expensive: Need to pay for each upgrade
o Non-portable: Only available for limited platforms
o Proprietary: Cannot extend its features or fix bugs
o Obfuscated: Vendor and black-hats can see your code, but you can’t
o Inflexible: Does not work with existing Perl installations

    MANIFEST: Best viewed with Mozilla

o The URL of MANIFEST inside /home/autrijus/foo.par:



 jar:file:///home/autrijus/foo.par!/MANIFEST



o Open it in a Gecko browser (e.g. Netscape 6+) with Javascript enabled:
o No needed to unzip anything; just click on files to view them

    META.yml: Metadata galore

o Static, machine-readable distribution metadata
o Supported by Module::Build, ExtUtils::MakeMaker, Module::Install
o A typical pp-generated META.yml looks like this:



 build_requires: {}
 conflicts: {}
 dist_name: out.par
 distribution_type: par
 dynamic_config: 0
 generated_by: Perl Packager version 0.03
 license: unknown
 par:
   clean: 0
   signature: 
   verbatim: 0
   version: 0.68



o The par: settings controls its runtime behavior

    SIGNATURE: Signing and verifying packages

o OpenPGP clear-signed manifest with SHA1 digests
o Supported by Module::Signature, CPANPLUS and Module::Build
o A typical SIGNATURE looks like this:



 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1

 SHA1 8a014cd6d0f6775552a01d1e6354a69eb6826046 AUTHORS
 ...
 -----BEGIN PGP SIGNATURE-----
 ...
 -----END PGP SIGNATURE-----



o Use pp and cpansign to work with signatures:



 % pp -s -o foo.par bar.pl      # make and sign foo.par from bar.pl
 % cpansign -s foo.par  # sign this PAR file
 % cpansign -v foo.par  # verify this PAR file



    Perl Servlets with Apache::PAR

o Framework for self-contained Web applications
o Similar to Java’s Web Application Archive (WAR) files
o Works with mod_perl 1.x or 2.x
o A complete web application inside a .par file
o Apache configuration, static files, Perl modules...
o Supports Static, Registry and PerlRun handlers
o Can also load all PARs under a directory
o One additional special file: web.conf



 Alias /myapp/cgi-perl/ ##PARFILE##/
 <Location /myapp/cgi-perl>
     Options +ExecCGI
     SetHandler perl-script
     PerlHandler Apache::PAR::Registry
 </Location>



    Hon Dah, A-par-che!

o First, make a hondah.par from an one-liner:



 # use the "web.conf" from the previous slide
 % pp -p -o hondah.par -e print "Hon Dah!\n" \
      --add web.conf
 % chmod a+x hondah.par



o Add this to httpd.conf, then restart apache:



 <IfDefine MODPERL2>
 PerlModule Apache2
 </IfDefine>
 PerlAddVar PARInclude /home/autrijus/hondah.par
 PerlModule Apache::PAR



o Test it out:



 % GET http://localhost/myapp/cgi-perl/main.pl
 Hon Dah!



o Instant one-liner web application that works!

    On-demand library fetching

o With LWP installed, your can use remote PAR files:



 use PAR;
 use lib http://aut.dyndns.org/par/DBI-latest.par;
 use DBI;    # always up to date!



o Modules are now cached under $ENV{PAR_GLOBAL_TEMP}
o Auto-updates with LWP::Simple::mirror
o Download only if modified
o Safe for offline use after the first time
o May use SIGNATURE to prevent DNS-spoofing
o Makes large-scale deployment a breeze
o Upgrades from a central location
o No installers needed

    Code Obfuscation

o Also known as source-hiding techniques
o It is not encryption
o Offered by PerlApp, Perl2Exe, Stunnix...
o Usually easy to defeat
o Take optree dump from memory, feed to B::Deparse
o If you just want to stop a casual grep, deflate already works
o PAR now supports pluggable input filters with pp -f
o Bundled examples: Bleach, PodStrip and PatchContent
o True encryption using Crypt::*
o Or even _product activation_ over the internet
o Alternatively, just keep core logic in your server and use RPC

    Accessing packed files

o To get the host archive from a packed program:



 my $zip = PAR::par_handle($0); # an Archive::Zip object
 my $content = $zip->contents(MANIFEST);



o Same thing, but with read_file():



 my $content = PAR::read_file(MANIFEST);



o Loaded PAR files are stored in %PAR::LibCache:



 use PAR /home/mylibs/*.par;
 while (my ($filename, $zip) = each %PAR::LibCache) {
     print "[$filename - MANIFEST]\n";
     print $zip->contents(MANIFEST);
 }



    Packing GUI applications

o GUI toolkits often need to link with shared libraries:



 # search for libncurses under library paths and pack it
 % pp -l ncurses curses_app.pl  # same for Tk, Wx, Gtk, Qt...



o Use pp --gui on Win32 to eliminate the console window:



 # pack src.pl into a console-less out.exe (Win32 only)
 % pp --gui -o out.exe src.pl



o Can’t locate Foo/Widget/Bar.pm in @INC?
o Some toolkits (notably Tk) autoloads modules without use or require
o Hence pp and Module::ScanDeps may fail to detect them
o Tk problems mostly fixed by now, but other toolkits may still break
o You can work around it with pp -M or an explicit require
o Or better, send a short test-case to par@perl.org so we can fix it

    Precompiled CPAN distributions

o Installing XS extensions from CPAN was difficult
o Some platforms do not come with a compiler (Win32, MacOSX...)
o Some headers or libraries may be missing
o PAR.pm itself used to suffer from both problems
o ...but not anymore — Module::Install to the rescue!



 # same old Makefile.PL, with a few changes
 use inc::Module::Install;      # was "use ExtUtils::MakeMaker;"
 WriteMakefile( ... );          # same as the original
 check_nmake();                 # make sure the user have nmake
 par_base(AUTRIJUS);          # your CPAN ID or a URL
 fetch_par() unless can_cc();   # use precompiled PAR only if necessary



o Users will not notice anything, except now it works
o Of course, you still need to type make par and upload the precompiled package
o PAR users can also install it directly with parl -i

    Platform-specific Tips

o Win32 and other icon-savvy platforms
o Needs 3rd-party tools to add icons to pp-generated executables
o PE Header manipulation in Perl — volunteers wanted!
o Linux and other libc-based platforms
o Try to avoid running pp on a bleeding-edge version of the OS
o Older versions with an earlier libc won’t work with new ones
o Solaris and other zlib-lacking platforms (but not Win32)
o You need a static-linked Compress::Zlib before installing PAR
o In the future, PAR may depend on Compress::Zlib::Static instead
o Any platform with limited bandwidth or disk space
o Use UPX to minimize the executable size

    Thank you!

o Additional resources
o Mailing list: par@perl.org
o Subscribe: Send a blank email to par-subscribe@perl.org
o List archive: <http://nntp.x.perl.org/group/perl.par>
o PAR::Intro: <http://search.cpan.org/dist/PAR/lib/PAR/Intro.pod>
o Apache::PAR: <http://search.cpan.org/dist/Apache-PAR/>
o Module::Install: <http://search.cpan.org/dist/Module-Install/>
o Any questions?

    Bonus Slides: PAR Internals

    Overview of PAR.pm’s Implementation

o Here begins the scary part
o Grues, Dragons and Jabberwocks abound...
o You are going to learn weird things about Perl internals
o PAR invokes four areas of Perl arcana:
o @INC code references
o On-the-fly source filtering
o Overriding DynaLoader::bootstrap() to handle XS modules
o Making self-bootstrapping binary executables
o The first two only works on 5.6 or later
o DynaLoader and %INC are there since Perl 5 was born
o PAR currently needs 5.6, but a 5.005 port is possible

Code References in CW@INC

o On 1999-07-19, Ken Fox submitted a patch to P5P
o To _enable using remote modules_ by putting hooks in @INC
o It’s accepted to come in Perl 5.6, but undocumented until 5.8
o Type perldoc -f require to read the nitty-gritty details
o Coderefs in @INC may return a fh, or undef to ’pass’:



 push @INC, sub {
     my ($coderef, $filename) = @_;  # $coderef is \&my_sub
     open my $fh, "wget ftp://example.com/$filename |";
     return $fh;        # using remote modules, indeed!
 };



o Perl 5.8 let you open a file handle to a string, so we just use that:



        open my $fh, <, \($zip->memberNamed($filename)->contents);
        return $fh;



o But Perl 5.6 does not have that, and I don’t want to use temp files...

    Source Filtering without Filter::* Modules

o ... Undocumented features to the rescue!
o It turns out that @INC hooks can return <B>twoB> values
o The first is still the file handle
o The second is a code reference for line-by-line source filtering!
o This is how Acme::use::strict::with::pride works:



 # Force all modules used to use strict and warnings
 open my $fh, "<", $filename or return;
 my @lines = ("use strict; use warnings;\n", "#line 1 \"$full\"\n");
 return ($fh, sub {
     return 0 unless @lines;   
     push @lines, $_; $_ = shift @lines; return length $_;
 });



    Source Filtering without Filter::* Modules (cont.)

o But we don’t really have a filehandle for anything
o Another undocumented feature saves the day!
o We can actually omit the first return value altogether:



 # Return all contents line-by-line from the file inside PAR
 my @lines = split(
     /(?<=\n)/,
     $zip->memberNamed($filename)->contents
 );
 return (sub {
     $_ = shift(@lines);
     return length $_;
 });



    Overriding DynaLoader::bootstrap

o XS modules have dynamically loaded libraries
o They cannot be loaded as part of a zip file, so we extract them out
o Must intercept DynaLoader’s library-finding process
o Module names are passed to bootstrap for XS loading
o During the process, it calls dl_findfile to locate the file
o So we install pre-hooks around both functions
o Our _bootstrap just checks if the library is in PARs
o If yes, extract it to a File::Temp temp file
o The file will be automatically cleaned up when the program ends
o It then pass the arguments to the original bootstrap
o Finally, our dl_findfile intercepts known filenames and return it

    Anatomy of a Self-Contained PAR executable

o The par script ($0) itself
o May be in plain-text or native executable format
o Any number of embedded files
o Typically used to bootstrap PAR’s various dependencies
o Each section begins with the magic string FILE
o Length of filename in pack(’N’) format and the filename (auto/.../)
o File length in pack(’N’) and the file’s content (not compressed)
o One PAR file
o Just a regular zip file with the magic string "PK\003\004"
o Ending section
o A pack(’N’) number of the total length of FILE and PAR sections
o Finally, there must be a 8-bytes magic string: "\012PAR.pm\012"

    Self-Bootstrapping Tricks

o All we can expect is a working perl interpreter
o The self-contained script *must not* use any modules at all
o But to process PAR files, we need XS modules like Compress::Zlib
o Answer: bundle all modules + libraries used by PAR.pm
o That’s what the FILE section in the previous slide is for
o Load modules to memory, and write object files to disk
o Then use a local @INC hook to load them on demand
o Minimizing the amount of temporary files
o First, try to load PerlIO::scalar and File::Temp
o Set up an END hook to unlink all temp files up to this point
o Load other bundled files, and look in the compressed PAR section
o This can be much easier with a pure-perl inflate(); patches welcome!

    Thank you (again)!

o Any questions, please?

SEE ALSO

PAR, pp, par.pl, parl

ex::lib::zip, Acme::use::strict::with::pride

App::Packer, Apache::PAR, CPANPLUS, Module::Install

AUTHORS

Audrey Tang <cpan@audreyt.org>

<http://par.perl.org/> is the official PAR website. You can write to the mailing list at <par@perl.org>, or send an empty mail to <par-subscribe@perl.org> to participate in the discussion.

Please submit bug reports to <bug-par@rt.cpan.org>.

COPYRIGHT

Copyright 2003, 2004, 2005, 2006 by Audrey Tang <cpan@audreyt.org>.

This document is free documentation; you can redistribute it and/or modify it under the same terms as Perl itself.

See <http://www.perl.com/perl/misc/Artistic.html>

Search for    or go to Top of page |  Section 3 |  Main Index


perl v5.20.3 PAR::TUTORIAL (3) 2011-12-28

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.