/ 
Tapper-Doc-3.000004
/ Tapper::Doc::Start::StepByStep

NAME

Tapper::Doc::Start::StepByStep - Step-by-step instructions how to set up a Tapper infrastructure

About

Tapper is a modular test infrastructure, inclusive Operating System and Virtualization support. See Tapper::Doc::FactSheet for a feature overview.

Intro

In this setup guide we will set up a central master host running the testrun and reports databases, the web frontend and the automation layer based on network boot, tftp and nfs.

The central host is named

  • plutonium

and we use 2 slave machines named

  • johnconnor

  • sarahconnor

We use symbolic variables for these machines:

$TAPPER_SERVER = 'plutonium'
$TESTMACHINE1  = 'johnconnor'
$TESTMACHINE2  = 'sarahconnor'

Install base OS

Install a Debian on $TAPPER_SERVER.

Other OSes will work, too. We ran it for years on SLES 10 but prepared this manual based on Debian Squeeze.

Get example starter packages

TODO: PREPARE THESE PACKAGE FILES

We provide some example files for first start. Download the package and unpack it in /tmp:

cd /tmp
wget http://www.amd64.org/tapper/first-start/tapper-example-configs.tgz
wget http://www.amd64.org/tapper/first-start/tapper-example-images.tgz
tar xzf tapper-example-configs.tgz
tar xzf tapper-example-images.tgz

This will give you those files:

/tmp/tapper-example-configs/etc/tapper.cfg
/tmp/tapper-example-configs/etc/init.d/tapper_reports_receiver_daemon
/tmp/tapper-example-configs/etc/init.d/tapper_reports_api_daemon
/tmp/tapper-example-configs/etc/apache2/conf.d/tapper_reports_web.conf
/tmp/tapper-example-configs/etc/dhcp.conf

/tmp/tapper-example-configs/utils/opt-tapper-package.sh
/tmp/tapper-example-configs/utils/kernel_boot_example_precondition.yml

/tmp/tapper-example-images/netboot/bzImage
/tmp/tapper-example-images/guest/debian_base.tgz
/tmp/tapper-example-images/nfsroot/debian_nfs_root.tgz

Prepare basic work environment

Config

Create a Tapper config in /etc/tapper.cfg. Copy an example config from /tmp/tapper-example-configs/etc/tapper.cfg and change it to fit your needs.

Prepare central workdir

Create sub directories that appear in the Tapper config. You don't need to understand them now. Just create the subdir hierarchy:

mkdir -p /data/tapper/live/metareports/                     # meta-reports
mkdir -p /data/tapper/live/output/                          # intermediate output
mkdir -p /data/tapper/live/repository/images/               # OS images
mkdir -p /data/tapper/live/repository/packages/             # packages
mkdir -p /data/tapper/live/repository/packages/tapperutils  # Tapper clientlib
mkdir -p /data/tapper/live/testprogram/                     # test programs
mkdir -p /data/tapper/live/configs                          # auto-generated configs
mkdir -p /data/tapper/live/configs/tftpboot/                # grub configs via TFTP
mkdir -p /data/tapper/live/sync/                            # sync'ing interdependent tests
mkdir -p /data/tapper/live/nfsroot/                         # network bootable OS image

Prepare network boot OS

Create an NFS root that is able to boot over network. It will later run the Installer on the test machines:

The manual prepare-netboot-image.pod tells you how to configure such an image.

Copy the content of the OS subdirectory into the central workdir:

rsync -a /tmp/NFSROOT/ /data/tapper/live/nfsroot/

Install an NFS server

  • Install NFS server

    apt-get install nfs-server

  • Export /data/tapper

    echo '/data/tapper 192.168.1.0/255.255.255.0(fsid=0,no_root_squash,no_subtree_check,rw)' >> /etc/exports

Set up Perl

We set up our own perl to not interfere with a system Perl.

Install perl using perlbrew

  • Install perlbrew

    cpan App::perlbrew

  • Initialize perlbrew

    PERLBREW_ROOT=/opt/tapper/perl perlbrew init

  • Add perlbrew to your .bashrc

    Add this line

    source /opt/tapper/perl/etc/bashrc

    to your .bashrc (as instructed by perlbrew).

  • Install perl-5.12.1 (or any version >= 5.10.1)

    perlbrew install perl-5.12.1

    Use -D, -U and -A to pass switches to Configure.

  • Switch to your new perl

    perlbrew switch perl-5.12.1

  • Configure your cpan

    For example:

    $ cpan
cpan[1]> o conf auto_commit 1
cpan[2]> o conf urllist unshift ftp://ftp.fu-berlin.de/unix/languages/perl/
# ... as you like it ...

Create a client-side "/opt/tapper" package

We package a subset of dependencies needed on installed machines in /opt/tapper without anything else and create an opt-tapper.tgz package from it.

That package is later installed on the test machine.

It consists of a precompiled perl and the tools to control test suite execution (Tapper::PRC, and for some special situations Tapper::Installer).

The different architecture packages are specified in this section of tapper.cfg:

files:
 tapper_package:
   linux64: tapperutils/opt-tapper64.tar.gz
   linux32: tapperutils/opt-tapper32.tar.gz
   windows: tapperutils/wintest.tar.gz

Now let's create them. First start creating a chroot, e.g. by using the debian image you got earlier:

mkdir /opt/chroot
tar -C /opt/chroot/ -xzf /tmp/tapper-example-images/guest/debian_base.tgz
chroot /opt/chroot
mount -t proc proc /proc

Install perl inside this chroot as described in chapter Install perl using perlbrew:

cpan App::perlbrew
PERLBREW_ROOT=/opt/tapper/perl perlbrew init
source /opt/tapper/perl/etc/bashrc
perlbrew install perl-5.12.1
perlbrew switch perl-5.12.1
cpan
  cpan[1]> o conf auto_commit 1
  cpan[2]> o conf urllist unshift ftp://ftp.fu-berlin.de/unix/languages/perl/

Install the PRC and Installer packages:

cpan Tapper::PRC
cpan Tapper::Installer

Leave the chroot and pack the package:

exit
cd /opt/chroot
tar -czf /data/tapper/live/repository/packages/tapperutils/opt-tapper64.tar.gz opt/ # for linux 64bit

Repeat this step for every architecture you need, e.g. Linux 32bit, Linux with different glibc version or whatever. This of course requires appropriate chroot images.

Prepare Central Server

  • apt-get install mysql-server

  • apt-get install libmysqlclient-dev # needed for DBD::mysql

  • cpan DBD::mysql

    (The particular driver is your decision. We run Tapper with mysql and do unit tests with SQLite. We recommend using the same first and then feel free to try out others.)

  • cpan Tapper::Schema

  • cpan Tapper::CLI

Prepare Database

  • echo "create user tapper identified by 'tapper';" | mysql -uroot

  • echo "create database reportsdb;" | mysql -uroot -p

  • echo "grant all on reportsdb.* to tapper@'%';" | mysql -uroot

  • echo "create database testrundb;" | mysql -uroot -p

  • echo "grant all on testrundb.* to tapper@'%';" | mysql -u root

  • yes | tapper-db-deploy init --db ReportsDB

  • yes | tapper-db-deploy init --db TestrunDB

  • make sure a scheduling queue named "AdHoc" exists

    • tapper-testrun listqueue | grep AdHoc

    • if it does not exists, create it with

    • tapper-testrun newqueue --name AdHoc --prio 1000 # the prio is not important, the name is

Put your test machines into the database. You can already use the command line interface for this task:

tapper-testrun newhost --name $TESTMACHINE1
tapper-testrun newhost --name $TESTMACHINE2

You need to insert host feature information into your database. This information is used to generate hardware reports which later allow you to group reports based on machine groups. Furthermore, this information is used for scheduling based on requested_features.

Currently, there is no public tool to generate the required database entries automatically.

For this tutorial we provide the following example SQL statements for host features:

$ mysql testrundb -utapper -ptapper

-- machine1
mysql> insert into host_feature(host_id, entry, value)  values ((select id from host where name = '$TESTMACHINE1'),     'mem',  4096);
mysql> insert into host_feature(host_id, entry, value)  values ((select id from host where name = '$TESTMACHINE1'),   'cores',     4);
mysql> insert into host_feature(host_id, entry, value)  values ((select id from host where name = '$TESTMACHINE1'),  'vendor', 'AMD');
mysql> insert into host_feature(host_id, entry, value)  values ((select id from host where name = '$TESTMACHINE1'), 'has_ecc',     1);

-- machine2
mysql> insert into host_feature(host_id, entry, value)  values ((select id from host where name = '$TESTMACHINE2'),     'mem',    2048);
mysql> insert into host_feature(host_id, entry, value)  values ((select id from host where name = '$TESTMACHINE2'),   'cores',       2);
mysql> insert into host_feature(host_id, entry, value)  values ((select id from host where name = '$TESTMACHINE2'),  'vendor', 'Intel');
mysql> insert into host_feature(host_id, entry, value)  values ((select id from host where name = '$TESTMACHINE2'), 'has_ecc',       0);

You now have an empty database ready for use.

Setup Reports Framework

The reports framework collects and evaluates test reports of test results. Even though other ways to get these results are possible using the reports framework is simple and highly integrated into the whole Tapper framework.

  • apt-get install libexpat-dev # needed for XML::Parser

  • cpan Tapper::Reports::Web

  • cpan Tapper::Reports::Receiver

  • cpan Tapper::Reports::API

  • install the start scripts

    ** cp doc/oss/etc/init.d/tapper_reports_receiver_daemon /etc/init.d/
    ** cp doc/oss/etc/init.d/tapper_reports_api_daemon /etc/init.d/
    ** update-rc.d tapper_reports_receiver_daemon defaults
    ** update-rc.d tapper_reports_api_daemon defaults

  • start reports receiver and API daemon with

    ** /etc/init.d/tapper_reports_receiver_daemon start
    ** /etc/init.d/tapper_reports_api_daemon start

  • send a first test report to $TAPPER_SERVER 7357

    # ----- example report command ---------
echo '
1..2
# Tapper-Suite-Name: example-hello-world
# Tapper-Suite-Version: 1.01
# Tapper-Machine-Name: hello-host
ok - Hello test world
ok - Just another description
' | netcat -q7 -w1 $TAPPER_SERVER 7357
# ----- end example report command ------

    Depending on your netcat flavour the options -q and -w need to be droppped or used differently.

Setup Web Server

The Tapper::Reports::Web server is the best way to get the test results. Therefore, we recommend to install it even though tests are possible without it.

  • install apache

    apt-get install apache2

  • enable fastcgi

    apt-get install libapache2-mod-fcgid

a2enmod fcgid

  • cpan Tapper::Reports::Web

  • start tapper_reports_web_server

    cp doc/oss/etc/apache2/conf.d/tapper_reports_web.conf /etc/apache2/conf.d/
/etc/init.d/apache2 restart

  • connect to $TAPPER_SERVER in your web browser

    http://plutonium/tapper

Set up test machines

As long as you don't plan interdependent (client-server) testruns, just one test machine would be enough. To demonstrate a few Tapper features, we still describe a setup of two test machines. Substitue $TESTMACHINE1 and $TESTMACHINE2 with the names of your test maschines.

Automatic reset

Usually Tapper assumes that test machines are to be reinstalled for every test (though there are exceptions).

We also assume that doing a reboot via logging into the testmachine and calling reboot is not always possible, e.g. on a kernel Oops.

Tapper will always try to login first and then call some hardware reset facility, as specified in the config file. One example used here as example is a power outlet with network support. (In reality we have our own reset solution.)

Configure Reset mechanism

The example config file has it already configured under the options reset_plugin and reset_plugin_options.

Configure it accordingly, usually the IP addresses of the power outlet and the numbers where each machine is connected accordingly:

reset_plugin: PM211MIP
reset_plugin_options:
  ip: 192.168.1.39
  user: admin
  passwd: secret
  outletnr:
    johnconnor: 2
    sarahconnor: 1

Connect machines to the power outlet

Do it now.

Install TFTP daemon

When a system is installed it starts a minimalistic system over network. The kernel is loaded via TFTP, the root is served via NFS.

Here we install a TFTP server, put the kernel image there, and create some symlinks to ease later relative and absolute filenames. First believe us and try it first before you ask. :-)

Install TFTP server
apt-get install atftpd
Copy a linux kernel image
cp /tmp/tapper-example-images/netboot/bzImage /tftpboot/
ln -s /data/tapper/live/configs/tftpboot /tftpboot/tftpboot

Prepare the Installer nfsroot

When a system is installed it starts a minimalistic system over network. The kernel is loaded via TFTP, the root is served via NFS.

Here we setup the NFS root to use:

cd /data/tapper/live/nfsroot/
tar xzf /tmp/tapper-example-images/nfsroot/debian_nfs_root.tgz

This image is basically the Debian base image with changes to use it readonly over NFS. Furthermore, it already contains the Tapper::Installer which needs to be updated on changes. Use the chroot approach described in Create opt-tapper package.. For a detailed description on how this image was prepared look into prepare-nfs-root.pod.

Configure your grub to boot the testmachine

As configured by default Tapper writes host specific grub configs dynamically into /data/tapper/live/configs/tftpboot/$hostname.lst. It will dynamically contain whether to boot the Tapper installer or the ready installed test system.

Configure your DHCP accordingly to use the same files.

Find an example DHCP config in /tmp/tapper-example-configs/etc/dhcp.conf.

Basically it says to use a PXE grub and points machines to their respective grub config which is dynamically provided by Tapper.

It looks similar to this:

 # example dhcp config with invalid ethernet addresses
 subnet 192.168.1.0 netmask 255.255.255.0 {
 group
 {
        filename '/tftpboot/pxegrub';

        # offer the host the here given name as host name
        option host-name = host-decl-name;
        option dhcp-parameter-request-list = concat(option dhcp-parameter-request-list,96);
        host sarahconnor
        {
                hardware ethernet 00:09:11:11:11:11;
                fixed-address 192.168.1.2;
                option configfile "/data/tapper/live/configs/tftpboot/sarahconnor.lst";
        }
	host johnconnor
        {
                hardware ethernet 00:09:22:22:22:22;
                fixed-address 192.168.1.3;
                option configfile "/data/tapper/live/configs/tftpboot/johnconnor.lst";
        }
 }

but you will definitely need to work out your own DHCP solution. The only requirement is to get machines booting using those /data/tapper/live/configs/tftpboot/$hostname.lst grub configs.

Set up Master Control Program

The master controller is the part that schedules and controls testruns. In Tapper notation it is usually called the MCP (short for Master Control Program).

MCP is the core of the Tapper automation part.

Install the MCP packet on the $TAPPER_SERVER

cpan Tapper::MCP

Start the Tapper master controller:

tapper-mcp-runloop

You can now start your first fully automatic Tapper test.

tapper-testrun new --macroprecon /tmp/tapper-example-configs/utils/kernel_boot_example_precondition.yml --requested_host johnconnor

Congratulation, you are now an Tapper user.