NAME

Test::AutoBuild - Automated build engine

SYNOPSIS

use Test::AutoBuild;
use Config::Record;

my $config = new Config::Record (file => $filename);
my $builder = new Test::AutoBuild (config => $config [, verbose => 1]);

my $status = $builder->run;

exit $status;

DESCRIPTION

This module provides the engine of the automated build system. It hooks together all the other modules to provide a structured workflow process for running the build. In a future release of autobuild, this module will be re-written to allow the workflow to be defined through the configuration file.

SETUP

After installing the modules, the first setup step is to create an unprivileged user to run the build as. By convention the user is called 'builder', in a similarly named group and a home directory of /var/builder. So as root, execute the following commands:

$ groupadd builder
$ useradd -g builder -m -d /var/builder builder

NB, with the combined contents of the source checkout, the cache and the virtual installed root, and HTTP site, the disk space requirements can be pretty large for any non-trivial software. Based on the applications being built, anywhere between 100MB and many GB of disk space make be neccessary. For Linux, making /var/builder a dedicated partition with LVM (Logical Volume Manager) will enable additional space to be easily grafted on without requiring a re-build.

The next step is to create the basic directory structure within the user's home directory for storing the various files. This should all be done while logged in as the build user:

# Where source code will be checked out to  
$ mkdir /var/builder/build-home

# Where packages will install themselves
$ mkdir /var/builder/build-root

# Where the builder will cache installed files
$ mkdir /var/builder/build-cache

# Where HTML status pages will be generated
# and packages / logs copied
$ mkdir /var/builder/public_html

# Where FTPable packages will be copied
$ mkdir /var/builder/public_ftp

# Required output dirs when building various 
# different types of packages
$ mkdir /var/builder/packages
$ mkdir /var/builder/packages/rpm
$ mkdir /var/builder/packages/rpm/BUILD
$ mkdir /var/builder/packages/rpm/RPMS
$ mkdir /var/builder/packages/rpm/RPMS/noarch
$ mkdir /var/builder/packages/rpm/RPMS/i386
$ mkdir /var/builder/packages/rpm/RPMS/i486
$ mkdir /var/builder/packages/rpm/RPMS/i586
$ mkdir /var/builder/packages/rpm/RPMS/i686
$ mkdir /var/builder/packages/rpm/RPMS/i786
$ mkdir /var/builder/packages/rpm/RPMS/sparc
$ mkdir /var/builder/packages/rpm/SPECS
$ mkdir /var/builder/packages/rpm/SOURCES
$ mkdir /var/builder/packages/rpm/SRPMS
$ mkdir /var/builder/packages/zips
$ mkdir /var/builder/packages/tars
$ mkdir /var/builder/packages/debian

CONFIGURATION

The first level of parameters in the configuration file are processed by this module. The currently supported parameters are:

debug = 0

Boolean value to turns on/off debugging output of build workflow engine

checkout-source = 0

Boolean value to turn on/off checkout of source code. When debugging build configuration, it is sometimes useful to turn off checkout of the module source files. By setting this value to 1, checkout will be disabled, letting the build run against the previously checked out source.

nice-level = 20

Sets the schedular 'nice' priority. Software builds can impose a considerable load on a machine, so by setting the nice-level to +20, the operating system schedular is informed that the automatic build should be run with lowest priority on the system. This often helps interactivity of command line shells on the build server. Since builds should never be run as root, the values for this setting can range from 0 (normal) to 20 (low).

control-file = rollingbuild.sh

Sets name of the shell file to execute to perform the build on a module. There is generally no need to change this value.

abort-on-fail = 0

Boolean value determining whether the build cycle terminates immediately when a module fails to build. Since the build engine will automatically skip any dependant modules upon failure of their pre-requisite, there is generally no need to set this value to 1.

tmp-dir = /var/tmp

Occassionally the build engine needs to maintain temporary files. This setting determines which directory the files will be saved in.

Build sub-configuration

The build sub-configuration block defines options specifically related to the build process. These are all defined within a block:

build = {
   .. options ...
}

home = /var/builder/build-home

Build home refers to the directory into which modules are checked out from source control repository.

root = /var/builder/build-root

Build root refers to the virtual root directory into which module builds will install files. When the module build is invoked, this parameter will be provided in the AUTO_BUILD_ROOT environment variable, allowing it to be used when running make install (or equivalent action). eg 'make install DESTDIR=$AUTO_BUILD_ROOT'

cache-dir = /var/builder/build-cache

Build cache refers to the directory in which the build engine caches files between cycles. There are currently two sets of files which are cached, generated packages (ie RPMs, ZIPs, etc), and installed files from the virtual root directory.

cache = 1

Boolean flag to determine whether to use the build cache. The build engine detects whether there have been any changes in source control for a module & if none were made, then it will skip build of the module & install files from the cache into the build root & use previously detected packages. If you want to force all modules to be re-built even when there were no changes, then set this to 0.

cache-timestamp = 0

As an alternative to having the build engine use the source control tools to detect any changes, it is possible to have it scan the local build home directory, comparing last modification timestamp to the last build cycle time. If any files are newer, then it will assume there were changes to the module. This option is only needed when the source control repository module does not support detection of changes. At this time all source control modules support change detection so there is no need to set this parameter to 1.

Lock sub-configuration

The lock sub-configuration block defines options specifically related to the locking between build instances. These are all defined within a block:

lock = {
    ....options....
}

file = /var/builder/.build.mutex

The file to use to prevent multiple instances of the builder running concurrently. The typical cron configuration starts the builder every 5 minutes. The first action the builder will take is to try and lock the files, if it fails it will exit immediately.

use-flock = 1

The flock() system call is not safe on NFS partitions. If the lock file is on an NFS partition then set this value to zero to have the builder use an alternative lock mechanism, albeit one with a small race condition. This race condition generally does not matter, unless two instances are started at *exactly* the same time.

Environment sub-configuration

The env configuration block allows arbitrary environment variales to be defined for the duration of the build process. The options should all be within a block

env = {
    ...options...
}

key = value

The key is the name of the environment variable (eg PATH) while the value is an arbitrary string (eg /usr/bin:/bin).

Group sub-configuration

The group configuration block defines a set of module groups. These groups are typically used in the build status HTML pages for splitting up the display of a large number of modules. Refer to the Test::AutoBuild::Group module for details of the configuration options. The options should all be within a block

    groups = {
	....options...
    }

Package type sub-configuration

The package-types configuration block defines a set of package types to handle. The directory defined for each package type will be scanned before & after a module build to pick up any packages which were generated during the build. Refer to the Test::AutoBuild::PackageType module for details of the configuration options.The options should all be within a block

    package-types = {
	....options...
    }

Repository sub-configuration

The repositories configuration block defines a set of source control repositories from which modules will be checked out. Refer to the Test::AutoBuild::Repository module for details of the configuration options. The options should all be within a block

    repositories = {
	....options...
    }

Publisher sub-configuration

The publisher configuration block defines a set of publishers to use for copying module build artifacts to an output directory, typically within the HTTP site. Refer to the Test::AutoBuild::Publisher module for details of the configuration options. The options should all be within a block

    publishers = {
	....options...
    }

Module sub-configuration

The module configuration block defines a set of modules to build on each cycle. Refer to the Test::AutoBuild::Module module for details of the configuration options.The options should all be within a block

    modules = {
	....options...
    }

Output module sub-configuration

The output configuration block defines a set of output modules to run at the end of each build cycle.Refer to the Test::AutoBuild::Output module for details of the configuration options.The options should all be within a block

    output = {
	....options...
    }

METHODS

$builder = Test::AutoBuild->(config => $config [, verbose => 1]);

Creates a new autobuild runtime object. $config is a configuration file (instance of Config::Record).

$config = $builder->config([$name, [$default]]);

If invoked with no arguments returns the Config::Record object storing the builder configuration. If invoked with a single argument, returns the configuration value with the matching name. An optional default value can be provided in the second argument

$builder-run();

Executes the build process. This is the heart of the auto build engine. It performs the following actions:

* Reads the list of modules, source control repositories,
  package types and output modules from the configuration
  file
* Initializes the build cache
* Takes out an exclusive file lock to prevent > 1 builder 
  running at the same time.
* Changes the (nice) priority of the AutoBuild process
* Checks the code for each module out of its respective 
  source control repository.
* Does a topological sort to determine the build order
  for all modules
* For each module to be built:
   - Take a snapshot of the package & virtual root install 
     directories
   - Change to the top level source directory of the module
   - Run the rollingbuild.sh script
   - Take another snapshot & compare to determine which
     files were install in the virtual root & which packages
     were generated
   - Save the intsalled files and packages in the cache.
* Invoke each requested output module, for example, HTML
  status generator, package & log file copiers, email
  alerts

AUTHORS

Daniel P. Berrange, Dennis Gregorovic

COPYRIGHT

Copyright (C) 2002 Daniel Berrange <dan@berrange.com>

SEE ALSO

perl(1), http://www.autobuild.org

cpanm Test::AutoBuild

perl -MCPAN -e shell
install Test::AutoBuild