NAME

docs/pdds/draft/pdd30_install.pod - Parrot Installation

ABSTRACT

This PDD outlines Parrot's installation system and support. Parrot's core installation system will provide support for binary packages, a working make install target, compiled installables, and FHS compliant search paths for the installables. This document also aims to solve the current problem of accessing installed source-only files, and to allow the optimization of config bootstrapping if a frozen config_hash is already linked.

VERSION

$Revision: 31181 $

SYNOPSIS

Short-term parrot core goal:

make install DESTDIR=/tmp/inst
make test-installable -C language/lang
make installable -C language/lang
make install -C language/lang DESTDIR=/tmp/inst

Long-term external goal:

cd module_name-1.00 && parrot Makefile.pir && make build test install

or:

cpan6 module_name ... --download --build --test --installable \
                      --test-installable --install

DESCRIPTION

Parrot's MANIFEST will contain the installation path of all files. The installation script tools/dev/install_files.pl is driven by this definition, which contains the filelist and the recommended package if being installed at all. The three runtime paths for "include", "library" for load_bytecode and "dynext" for loadlib should end up in the $prefix/lib/parrot/ paths. {{NOTE: See #56996-fhs-runtime.patch }} Accessing /usr/runtime is forbidden by the FHS.

The Parrot and language implementions on top of Parrot may be installed as self-hosting single-file executables, with the help of merged pbc's and pbc2exe --install. The destination path for the language libraries is not decided yet, two conflicting layouts 1. "a language pbc is a simple library" vs 2. "a language is special" are discussed.

Eg. ad 1. load_bytecode 'languages/WMLScript/src/WMLScript.pbc' should be replaced by load_bytecode 'WMLScript' and WMLScript.pbc should be installed to runtime/parrot/library/WMLScript.pbc which ends up in $prefix/lib/parrot/library/WMLScript.pbc.

1a. load_bytecode 'languages/WMLScript' => runtime/parrot/library/languages/WMLScript.pbc => $prefix/lib/parrot/library/languages/WMLScript.pbc.

1b. aka "existing practice": load_bytecode 'languages/WMLScript/WMLScript.pbc' => languages/WMLScript/WMLScript.pbc or runtime/parrot/library/languages/WMLScript/WMLScript.pbc => $prefix/lib/parrot/library/languages/WMLScript/WMLScript.pbc.

ad 2. load_bytecode 'languages/WMLScript/src/WMLScript.pbc' should be replaced by load_language 'WMLScript' and WMLScript.pbc should be installed to runtime/language/library/WMLScript.pbc which ends up in $prefix/lib/parrot/language/library/WMLScript.pbc. See below at "Layout 2 "Parallel - A language is special"". {{ ?? }}

Bootstrapping the configuration hash should not read a config file when the hash is already contained in the pmc or executable. {{ See #57418 [TODO] optimize _config to omit .include "library/config.pir" on installables. }} The same problem is for every .include, loadlib and load_bytecode statement in installed files where the target is not installed. If in doubt install the missing library by patching the installation code. For load_bytecode 'PGE/Text.pbc' in some lua src we should take care, that this compiler pbc is really installed or if not merge it into the main lua.pbc.

Test executables are binary different to installable executables because of this embedded configuration hash. Test executables contain configuration hash with the prefix to the build directory, installables to the given prefix from Configure.pl. The executables that are tested should always be the same as the ones that are installed. Otherwise, subtle bugs can leak into the installed executables which can never be caught by the tests.

There are's also a long-standing conflict in building Parrot with an already installed shared libparrot.so. See #39742-installed-conflict.patch which adds the blib/lib path to PATH resp. LD_RUN_PATH to all parrot executable invocations.

DEPENDENCIES

Before you can make installable -C language/lang or a simple make or make installable in the directory languages, you need to make parrot_utils and make installable in the parrot build directory.

The languages have no extra dependencies to the required parrot_utils and src/install_config.o and install_config.fpmc

DEFINITIONS

The build_dir is the full path where Parrot was built. It is defined in the configuration hash. When building from source the build_dir is also the PARROT_RUNTIME prefix.

An installable is a bytecode or executable file which must not access the build_dir paths. The build_dir path is not available in a binary package. This is solved by generating and linking a special install_config.fpmc. Custom Python modules have a similar packaging problem, which they haven't solved yet.

The destination directory is the path of the installed Parrot tree after the prefix (/usr, /usr/local, or some other platform-specific or custom location). Creating a virtual installation path like this simplifies packaging by installing into a separate install tree and creating a tarball from that tree.

The configuration hash is the return value of the global function _config(), generated in config_lib.pasm, and either defined in library/config.pir, or as frozen PMC embedded in the test executable (config.fpmc), the installable executable (install_config.fpmc) or empty for miniparrot (null_config.fpmc).

Layout 1 "A language pbc is simply a library"

Rationale: A language pbc is not much more than a normal library, so it should go to the normal parrot library path. The library searchpath needs no extension for languages.

load_bytecode 'languages/WMLScript/src/WMLScript.pbc' should be replaced by load_bytecode 'WMLScript' and WMLScript.pbc should be installed to runtime/parrot/library/WMLScript.pbc which ends up in $prefix/lib/parrot/library/WMLScript.pbc.

Usage: load_bytecode 'WMLScript' =gt runtime/parrot/library/WMLScript.pbc =gt $prefix/lib/parrot/library/WMLScript.pbc

Layout 1, Variant a "Don't mix the libraries with the languages"

Rationale: A language pbc is a library under the languages namespace. The library searchpath needs no extension for languages. load_bytecode 'languages/bla' makes it clear that bla is a language, not a library.

Usage: load_bytecode 'languages/WMLScript' =gt runtime/parrot/library/languages/WMLScript.pbc =gt $prefix/lib/parrot/library/languages/WMLScript.pbc.

Layout 1, Variant b "Below - Existing practice"

Rationale: A language pbc is a language pbc. Other languages assume the path from the build_dir as languages/WMLScript/WMLScript.pbc, the installation root begins at $prefix/lib/parrot/library.

Usage: load_bytecode 'languages/WMLScript/WMLScript.pbc' =gt languages/WMLScript/WMLScript.pbc =gt $prefix/lib/parrot/library/languages/WMLScript/WMLScript.pbc.

Layout 2 "Parallel - A language is special"

Rationale: Languages are more than simple libraries because they could be invoked by link $0 magic on platforms which do support that. Languages should have their distinct libraries seperate from parrot with seperate dynext, library, include. parrot is just another language, with perl6 sharing the same namespace.

This requires either a new opcode or method to able to override the library searchpath.

load_bytecode 'languages/WMLScript/src/WMLScript.pbc' should be replaced by load_language 'WMLScript' and WMLScript.pbc should be installed to runtime/languages/library/WMLScript.pbc which ends up in $prefix/lib/parrot/languages/library/WMLScript.pbc.

Usage: load_language 'WMLScript' =gt runtime/language/library/WMLScript.pbc =gt $prefix/lib/parrot/WMLScript/library/WMLScript.pbc or Variant 2a: $prefix/lib/parrot/languages/WMLScript/library/WMLScript.pbc.

Note: We will not be able to get rid of /usr/lib/parrot here, so the runtime/parrot <=> /usr/lib/parrot mapping is not symetric anymore.

make install

The Parrot build system is currently optimized for building and testing in the build directory, but not for building with an already installed Parrot. This is complicated by some simple build system bugs. It is also not optimized to build and test installables, which should not access libraries in the build directory, but in the destination directory.

The short-term goal of this document is to make installation work for Parrot, its libraries and tools, and its languages so that packagers can provide binary packages for parrot and its languages.

The longer-term goal is a framework so that external libraries and languages not within the current parrot source tree can also be properly built, tested and installed.

We do not only support GNU make or Win32 nmake but also other platform make variants, so we generate the Makefile at first from a generic template.

Currently for Parrot and its libraries the install actions are defined in the main Makefile. For the languages the install actions are defined in the language's Makefile. {{ See an implementation of this in #56554-make-install-lang.patch.}}

One general comment: We need to be careful not to install bytecode files in the same directories as executables. The C includes and the PIR includes also need to be separate.

make install currently does not work with an already installed shared libparrot.so on most platforms. {{See a patch for this in RT #39742.}}

The to-be-defined make install actions for a language named "mylanguage":

Different platforms will also support different layouts.

bin_dir: Copy installables to the destination directory's bin directory as parrot-mylanguage.
script_dir: Optionally copy the main language file mylanguage.pbc to the destination directory's script directory. (/usr/lib/parrot/bin/ ?)
dynext: Copy shared libraries (groups and ops) to the destination directory's lib directory under /parrot/dynext/.
lib_dir Variant 1: Copy the main language library mylanguage.pbc to the destination directory's lib directory under parrot/library/ for HLL interoperability. Also the other pbc's to a parrot/library/mylanguage subdirectory. load_bytecode "mylanguage" should succeed.
lib_dir Variant 1a: Copy the main language library mylanguage.pbc to the destination directory's lib directory under parrot/library/languages/ for HLL interoperability. Also the other pbc's to a parrot/library/languages/mylanguage subdirectory. load_bytecode "languages/mylanguage" should succeed.
lib_dir Variant 3 or 3a: Copy the main language library mylanguage.pbc to the destination directory's lib directory under parrot/language/library/ for HLL interoperability. Also the other pbc's to a parrot/language/library/mylanguage subdirectory. load_language "mylanguage" should succeed. The mylanguage.pbc is resonsible to load the other "mylanguage/mylangspecial" pbc's.

The subdirs are currently needed for forth and WMLScript, the other language pbc's are php_*.pbc, pipplib.pbc and tcllib.pbc.
lib_dir: Optionally copy include PASM and PIR files to the destination directory's lib directory under parrot/include/ (not yet).
doc_dir: Copy documentation files to a $doc_dir/languages/mylanguage/ subdirectory tree.
man_dir: Generate man(1) pages and copy to destination directory's $man_dir/man1/ directory. {{ what about man(2) and man(3) pages? info_dir ditto }}
html_dir: Optionally generate HTML and copy to destination directory's html directory, possibly under a language specific subdirectory. This should be selectable by a Configure or make install option.

make installable -C languages/mylanguage

This creates a bytecode or executable file linked to install_config.fpmc, and this executable should not access the build directory's runtime/parrot paths.

A bytecode file may be optionally merged with install_config.fpmc, an executable is just linked with pbc_to_exe --install.

make test-installable -C languages/mylanguage

Goal: Test if the generated installable does not access the build directory paths but does find older libraries, includes and dynext's in the $prefix path. The destination directory is not known and may be optional. A simple test from make test without any features from an external library is enough, because newer libraries are not installed yet at this stage.

Implementation: TODO

Problem: make test-installable should copy the make install files away, out of the build directory, should temporarily rename the build directory, run a simple test, and remake the build directory back. This will not be possible from a make run from within the build directory. So renaming the runtime directory during the test might be the best idea.

This is fragile and similar for every language target, so it should be simplified by a make framework, like include Makefile.common, extending the current automake-like framework or using a Parrot/Install library.

A core ExtUtils::MakeMaker alike installation library for every external library and HLL, because the languages will be removed from trunk before the Parrot 1.0 release.

Configuration bootstrapping

Bootstrapping the configuration hash should not read a config file when the hash is already contained in the PMC or executable. .include "library/config.pir" and load_bytecode "config.pbc" should be omitted on installables if possible.

{{NOTE: We need to step back and take a broader view here. Why is library/config.pir currently included in the installables? It sounds like a hack to work around earlier limitations in the build system. This is an ideal opportunity to eliminate the hack. -allison}}

Accessing not-installed files

Makefile and MANIFEST cleanup

Problem: Various PIR files load source-only PIR, PASM or compiler bytecode files, which are not installed in binary packages. This shows up when trying to run an installable with the build directory removed or renamed.

$ parrot-forth.exe xx
"load_bytecode" couldn't find file 'languages/forth/tokenstream.pbc'
current instr.: ' init' pc 942 (forth.pir:9)
called from Sub 'main' pc 994 (forth.pir:40)

$ parrot-pheme.exe
"load_bytecode" couldn't find file 'compilers/tge/TGE/Rule.pbc'
current instr.: 'parrot;TGE;__onload' pc 19 (TGE.pir:94)
called from Sub 'parrot;Pheme::AST::Grammar;__onload' pc 7175
   (languages/pheme/lib/ASTGrammar.pir:5)
called from Sub 'parrot;Pheme::Compiler;main' pc -1 ((unknown file):-1)

$ parrot-pipp
Parrot VM: Can't stat
   /usr/src/perl/parrot/parrot-0.7.0-1/
        build/languages/pipp/src/common/pipplib.pbc, code 2.
Unable to append PBC to the current directory
current instr.: 'parrot;Pipp;__onload' pc 47 (src/common/pipp.pir:92)
called from Sub 'parrot;Pipp;pipp' pc -1 ((unknown file):-1)

Fix 1: Install all deps and make sure that every HLL is installed at library/HLLNAME.pbc

{{NOTE: This may be a sign that we need to rethink our language build strategy. Trying to glom everything into a single C executable is less than ideal. Especially since it causes problems for language interoperability if every language is running off its own independent executable. -allison}}

The simple Forth and Pipp problem could be solved by merging the missing bytecode files to a single file forth.pbc and generate from this the installable.

The simple Pheme problem could be solved by installing also all TGE and other compiler bytecode files at the parrot/library/compilers path. Since TGE is not used elsewhere anymore, Pheme should be fixed to get rid of this. {{NOTE: commonly used libraries should be installed somewhere. -allison}}

The same problem is for every .include, loadlib and load_bytecode statement in installed files where the target is not installed.

Fix 2: Module system.

Avoid already loaded pbc files.

Source loading PIR statements like loadlib and load_bytecode should

a) cache the file name and skip the file if it has already been loaded (as in perl5)

b) add load*_once sisters as in php - load_bytecode_once and loadlib_once. This is not preferred. It might be added later on, but parrot core should not rely on it.

Fix 3: pbc_merge fixups

pbc_merge could patch up the bytecode (if possible) to omit loading load_bytecode pbc-files which are being merged, but hacking bytecode during pbc_merge is not desirable.

IMPLEMENTATION

A new language is generated by tools/dev/mk_language_shell.pl

The makefiles are generated from a config/makefiles/root.in template, which can make use of conditional platform and config logic, the forward slashes are automatically converted to backslashes for MSWin32 and \n is converted to \r\n for MSWin32 nmake. See Parrot::Configure::Compiler.

A new lib/Parrot/Install.pm or Parrot/Install.pir library should provide the same support as described above and simplify the Makefile and installation maintenance. The entry point could be a Makefile.pl or Makefile.pir then.

ATTACHMENTS

None.

FOOTNOTES

None.

REFERENCES

{{ Some of the mentioned single patches against SVN HEAD are at http://code.google.com/p/cygwin-rurban/source/browse/ trunk/release/parrot/patches

The cygwin070patches branch contains all the mentioned fixes and language lib_dir installation variant 1, so it is really a "make_install" branch. }}

#nnnnn references tickets in http://rt.perl.org/rt3/Ticket/Display.html?id=nnnnn

To install Parrot::PMC, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Parrot::PMC

CPAN shell

perl -MCPAN -e shell
install Parrot::PMC

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

ABSTRACT

VERSION

SYNOPSIS

DESCRIPTION

DEPENDENCIES

DEFINITIONS

Layout 1 "A language pbc is simply a library"

Layout 1, Variant a "Don't mix the libraries with the languages"

Layout 1, Variant b "Below - Existing practice"

Layout 2 "Parallel - A language is special"

make install

make installable -C languages/mylanguage

make test-installable -C languages/mylanguage

Configuration bootstrapping

Accessing not-installed files

IMPLEMENTATION

ATTACHMENTS

FOOTNOTES

REFERENCES

Module Install Instructions