/ 
Slay-Maker-0.08
River stage one • 1 direct dependent • 4 total dependents
/ Slay::Maker

NAME

Slay::Maker - An perl make engine using perl code for rules

STATUS

Beta. Pretty stable, though underdocumented.

DESCRIPTION

Slay::Maker is a make engine that uses perl declaration syntax for rules, including regular expressions for targets and anonymous subs for targets, dependencies, and actions.

This allows you to tightly integrate a make engine in an application and to exercise a large amount of control over the make process, taking full advantage of Perl code at any point in the make cycle.

RULE SYNTAX

The rulebase syntax is:

[ @targets1, ':', @dependencies1, '=', @actions1, { option => val } ],
[ @targets2, ':', @dependencies2, '=', @actions2, { option => val } ],
[ @targets3, ':', @dependencies3, '=', @actions3, { option => val } ],
[ @targets4, ':', @dependencies4, '=', @actions4, { option => val } ],
...

Each item in any of the three arrays may be a literal string or a subroutine (CODE) reference. A literal string is pretty much the same as using a literal string in a regular makefile. You may also use regular expression ('Regexp') references (qr/.../) in @targets and the $1, $2, ... variables inside strings in @dependencies:

[ qr/(.*).pm/, ':', "$1.pm.in", '=', sub { ... } ],

Subroutine references are evaluated as lazily as possible when the make is being run, so any CODE refs in @targets will be called each time a make is run, CODE refs in @dependencies will only be called if a target matches, and CODE refs in @actions are only called if the rule is fired.

TARGET SUBS

** NOT IMPLEMENTED QUITE YET **. It's simple to do, just haven't needed it yet.

Aside from strings and Regexps, you will be able to use CODE refs in the target list. These are called each time the rule is evaluated, which will usually happen once per target or dependency being checked when the make is run.

A target sub declaration might look like:

sub {
   my ( $maker ) = @_ ;
   ...
   return @targets;
},

(if target subs were implemented already).

DEPENDENCIES

Dependencies may be strings or CODE references. Plain strings have $1, $2, ... interpolation done on them (remember to \ escape the $1, etc.).

CODE refs will be called if the target matches and must return a possibly empty list of strings containing the names of dependencies. Variable interpolation will not be done on the returned strings. That would be obscene.

A dependency sub declaration might look like:

sub {
   my ( $maker, $target, $matches ) = @_ ;
   ...
   return @dependencies ;
},

where

$maker    refers to the Slay::Maker (or subclass) being run
$target   is the target that matched (in the event of multiple targets)
$matches  is an ARRAY of the values extracted from $1, $2, etc.

.

ACTIONS

If an $action is a plain string, it's passed to "sh -c '$string'". If it's an ARRAY ref, it's run without interference from or benefit of a shell (see "run" in IPC::Run for details). If it's a CODE ref, it's called.

An action sub declaration might look like:

sub {
   my ( $maker, $target, $deps, $matches ) = @_ ;
   ...
   return @dependencies ;
},

where

   $maker    refers to the Slay::Maker (or subclass) being run
   $target   is the target that matched (in the event of multiple targets)
   $deps     is an ARRAY of the expanded dependencies.  There's no way
             of telling which are freshly rebuilt, but you can track that
	     yourself in the action rules of the dependencies, if you
	     like.
   $matches  is an ARRAY of the values extracted from $1, $2, etc.

TARGET BACKUPS

A target may be moved off to a backup location before it is rebuilt, so that it may be restored if rebuilding fails. This is also used for the optional restoration of modification times described below.

Restoration needs to be done manually by calling the "restore" method, and you can call the "backup" method, too.

The "backup" method will be called automatically if modification time restoration is enabled for a target.

MODIFICATION TIME RESTORATION

One unusual option is that a target file's modification time can be restored if it is unchanged after being updated. This can be useful when checking files out of a repository: the files' mod times won't be affected if their contents haven't changed.

This can be done by a (fast but possibly misleading) check for a change in file size or by executing 'diff --brief' between a target's backup and it's new version. Other methods, such as hashing or block-by-block binary comparison will be implemented in the future if needed.

This is controlled by the "detect_no_diffs" option passed to the base class constructor:

my $self = Slay::Maker->new( ..., options => { detect_no_diffs => 1 } ) ;

and can be passed as an option to any rule.

AN EXAMPLE

Here's a real example, which will have to stand in for documentation until further notice. If you need more, mail me (barries@slaysys.com) and get me to do something productive for a change.

This is a subclass that compiles a set of builtin rules at module compilation time. It declares a method for spawning the cvs command first, then builds some rules.

   package Safari::Cvs::Make ;

   use Class::Std ;
   use base qw( Slay::Maker ) ;

   use strict ;
   use IPC::Run ;

   sub cvs {
      my Safari::Cvs::Make $maker = shift ;

      my $stdout_to ;
      if ( $_[-1] =~ /^\s*>(.*)/ ) {
	 $stdout_to = $1 ;
	 pop ;
      }

      my $cvs_out ;
      run [ qw( cvs -q -l -z9 ), @_ ], \undef, \$cvs_out or die $! ;

      return $cvs_out ;
   }


   my $builtin_rules = Safari::Make->compile_rules(
      [  'meta/cvs_modules',
	 '=', sub {   ## The action that occurs when the rule fires.
	    ## We could just do the cvs co -c here, but many pservers don't
	    ## have that implemented.  so, check out the modules file and
	    ## parse it.
	    my ( $maker, $target ) = @_ ;
	    $maker->cvs( qw( checkout -p CVSROOT/modules ), ">$target" ) ;
	 },
      ],
      [ 'update',
	 ':' => sub {
	    my ( $maker ) = @_ ;

	    my $context = $maker->{CONTEXT} ;

	    my @modules ;

	    my %args = $context->request->args ;
	    if ( defined $args{modules} ) {
	       @modules = split( ',', $args{modules} ) ;
	    }
	    elsif ( defined $args{module} ) {
	       @modules = $args{module} ;
	    }
	    else {
	       eval {
	          ## A recursive make
		  $maker->make( 'meta/cvs_modules', { force => 1 } ) ;
	       } ;
	       if ( $@ ) {
		  warn $@ ;
	       }

	       if ( ! open( F, "<meta/cvs_modules" ) ) {
		  warn "$!: meta/cvs_modules, globbing" ;
		  @modules = map {
		     s{^meta/HEAD/}{}; $_
		  } grep {
		     -d $_
		  } glob( 'meta/HEAD/*' ) ;
	       }
	       else {
		  my $line ;
		  my %modules ;
		  while (<F>) {
		     next if /^\s*#|^\s*$/ ;
		     chomp ;
		     $line .= $_ ;
		     redo if $line =~ s{\\$}{} ;
		     $modules{$1} = 1 if $line =~ m{^\s*(\S+)} ;
		     $line = '' ;
		  }
		  close F ;
		  @modules = sort keys %modules ;
	       }
	    }

	    debug 'modules', \@modules ;
	    die "No modules found\n" unless @modules ;
	    return map { "source/HEAD/$_/CVS/" } @modules ;
	 },
	 '=' => sub {
	    my ( $maker, $target, $deps ) = @_ ;

	    my @dirs = map { s{/CVS/}{} ; $_ } @$deps ;

	    ## We go ahead and update after creating modules for a couple of
	    ## reasons:
	    ## 1. It's rare that we've just checked out a module
	    ## 2. It's simpler this way
	    ## 3. If we just created a *big* module, then we might need to
	    ## update anyway.

	    ## We set $logs{$filename} = 1 if we must replace the current log file,
	    ## or = 0 to just ensure that the log file is fetched.
	    my %logs ;
	    my $force_all ;

	    my $cwd = cwd() ;
	    for ( @dirs ) {
	       chdir $_ or die "$!: $_" ;
	       my $module = $_ ;
	       $module =~ s{.*/}{} ;
	       ## -P: Prune empty dirs
	       ## -d: Add new dirs that we don't have
	       for ( split /^/m, $maker->cvs( qw( update -d -P ) ) ) {
		  chomp ;
		  if ( /^[UP]\s+(.*)/ ) {
		     $logs{"meta/HEAD/$module/$1.log"} = 1 ;
		  }
		  elsif ( /^\?\s+(.*)/ ) {
		     my $log_file = "meta/HEAD/$module/$1.log" ;
		     eval {
			rmtree( [ $log_file ] ) ;
		     } ;
		     warn "Error removing ;$log_file'" if $@ ;
		  }
		  else {
		     warn "Unexpected CVS file mode: $_" ;
		  }
	       }
	       chdir $cwd or die "$!: $cwd" ;
	    }

	    for ( sort keys %logs ) {
	       $maker->make(
		  $_,
		  {
		     force => $force_all || $logs{$_}
		  }
	       ) ;
	    }

	 },
	 {
	    force => 1,   # Always remake this target
	 }
      ],
   ) ;

METHODS

new

Constructor.

my $rules = [
   # ...
] ;
my $maker = Slay::Maker->new( { } );
my $maker = Slay::Maker->new( { rules => $rules } ) ;
my $maker = Slay::Maker->new( { rules => $rules,
                                options => { option => 1 } } ) ;

options (which can also be defined on a per-rule basis) are:

auto_create_dirs

Creates directories that targets will be created in before executing a target's actions.

detect_no_diffs

Copy the target before executing a rule, then restore the original modification and access times if the contents of the target are the same after the rule.

detect_no_size_change

Look for changes in the size of a file after executing a rule and restore the original modification and access times if it changes.

force

Always remake target, even if it does not appear to be out of date

Warning: options are not checked for spelling errors.

Options may be passed to new(), make(), build_queue(), and to rules themselves. Options passed to make() or build_queue() take precedence over rules' options, and rules' options take precedence over those passed to new().

add_rules

Add rules (compiled or not) to the rule base.

atime

This returns the atime of a file, reading from the stat cache if possible.

build_queue

Builds a new queue of rules to be exec()ed to make a target

builtin_rules

Returns [] by default. This is provided so that subclasses may overload it to provide sets of rules. This is called by new() before adding any rules passed to new().

canonpath

Cleans up the path, much like File::Spec::Unix::canonpath(), but also removes name/.. constructs.

chdir

Calls system chdir(), die()s on failure, and uses the parameter as the current directory. The last is the main reason for this sub: if you chdir() to a symbolic link, then we want to know the symbolic directory, not the real one returned by cwd().

check_targets

Checks targets and adds them to queue if need be. Does not integrate Slay::Maker options: this is left to the caller (usually the original build_queue() call).

clear_caches

Clears the stat cache, so the filesystem must be reexamined. Only needed if Slay::Maker is being called repetitively.

clear_stat

Clears the stat cache for a given path, so the next stat() on that path will read the filesystem.

compile_rules

Returns a rulebase compiled from the arguments. Rules that are already compiled are passed through unchanged. This is a class method, so

Slay::Maker->compile_rules( 
   [ 'a', [qw( b c )], 'cat b c > a' ],
   ...
) ;

can be used to compile a rulebase once at startup

backup

Copies a file so that it can be restored later or checked for changes.

If the target will only ever be replaced by the make, then it will not be altered in-place, and the move option may be passed:

$maker->backup( $target, { move => 1 } ) ;

If the target is an file which always changes size when it is changed, you may pass the stat_only option:

$maker->backup( $target, { stat_only => 1 } ) ;

The return value can be passed to restore(), target_unchanged(), and remove_backup().

cwd

Returns the current working directory, from the cache if that is possible.

e

Returns true if the file exists, but uses the stat_cache if possible.

exec_queue

Executes the queued commands.

find_rule

Given a target, finds a rule.

get_rule_info(<target>)

Given a target that has already been processed with check_targets, either directly or indirectly, returns the rule that is used to produce the target, a reference to an array of dependencies of the target, and a reference to an array of the matches. Thus, you would call

($rule, $deps, $matches) = get_rule_info($target);

Returns an undefined rule if there is no processed rule to produce the target.

make

Makes one or more target(s) if it is out of date. Throws exceptions if the make fails. May partially make targets.

make_level

Returns 0 if make() has not been called (well, actually, if recurse_in() has not been called). Returns number of recursive calls otherwise, so this is equal to 1 when making something but not recursing.

mtime

This returns the mtime of a file, reading from the stat cache if possible.

options

Sets / gets a reference to the options hash.

push

Adds a ( target, rule ) tuple to the exec queue. Will not add the same target twice.

recurse_in

Sets up for a recursive make. Called automatically by make() if make() is already running.

recurse_out

Restored after a recursive make. Called automatically by make() if make() is already running.

queue_size

Number of rules that need to be made.

remove_backup
   my $backup = $maker->backup( $target ) ;
   ## ... 
   $maker->remove_backup(
      $backup,
      {
	 restore_if_unchanged => 1,
	 deps                 => \@deps,
      }
   ) ;

Removes a backup of the target created with backup_target().

replace_rules

Replaces the rule for a target (or targets). The targets passed in must exactly match those of the rule to be replaced.

restore
my $backup = $maker->backup( $target, { move => 1 } ) ;
## Try to recreate target, setting $error on error
$maker->restore( $backup )
   if $error ;
$maker->restore( $backup, { deps => \@deps } )
   if ! $error && $maker->target_unchanged( $backup ) ;
$maker->remove_backup( $backup ) ;

Note that you only need this in case of an error. You can pass the restore_if_unchanged => 1 and deps => \@deps options to remove_backup().

When backup() has been called, it's return value can be passed to restore_target() to restore the original target, timestamps and all.

NOTE: restoring a target that's not changed is likely to cuase it to be remade every time once a dependency's timestamp becomes more recent. The deps option allows the timestamps to be set to the newest of the original timestamps and the dependencies' timestamps. This should not be done if there was an error generating the file.

rules

Gets or replaces the rules list

size

This returns the size of a file, reading from the stat cache if possible.

stat

Looks in the stat cache for the stat() results for a path. If not found, fills the cache. The cache is shared between all instances of this class, and may be cleared using clear_stat_cache().

target_unchanged

Takes the result of backup_target() and checks to see if the target has been changed or removed.

TODO

  • Propagate effects of restored timestamps.

    If a target has it's timestamps restored as a result of detecting no change (see options detect_no_size_change and detect_no_diffs), then there may well be no need to actually execute later rules.

    One way to do this is to re-check the mtime dependencies when rebuilding. Another is to subscribe later items in the queue to earlier items and have the earlier items set a flag that tells the later items to go ahead and execute. Items could flag themselves to execute regardless, which we might want to do if a dependency is not present when make is run.

  • Don't really call diff(1) for detect_no_diffs

AUTHOR

Barrie Slaymaker <barries@slaysys.com>

LICENSE

Copyright 2000, R. Barrie Slaymaker, Jr., All Rights Reserved.

That being said, do what you will with this code, it's completely free.

Please let me know of any improvements so I can have the option of folding them back in to the original.