NAME

ppll

SYNOPSIS

ppll [OPTIONS] COMMAND [ARGS]
ppll [OPTIONS] -c COMMAND [PARAMETERS]
ppll --help
ppll --version

DESCRIPTION

PARAMETERS AND JOBS

‘Parameters’ is what makes ppll run. In one way or another ppll is given a number of parameters, and it will create and run one ‘job’ for each parameter.

If no parameter-related option is used ppll will read the parameters from STDIN. Ordinarily, each line read becomes one parameter, but there are options to control this.

The other important input to ppll is the ‘command’. The command-line arguments after all options make up the command and its arguments.

Each job will execute the command. All occurrences of {} in any part of the command (and its arguments) will be replaced with the job’s parameter. If there are no such occurrences the parameter will be appended as an additional argument.

For example:

ppll echo {} <<EOF
cow
pig
sheep
EOF

Will create jobs that execute echo cow, echo pig and so on. An other way to do the same thing would be:

ppll echo <<EOF
cow
pig
sheep
EOF

{} becomes useful when it’s not supposed to be the last argument, e.g:

ppll cp {} farm/ <<EOF
cow
pig
sheep
EOF

Or when it’s supposed to be just a part of an argument, or show up more than once, e.g:

ppll curl -o photo-of-a-{}.jpg http://example.com/img/{}.jpg <<EOF
cow
pig
sheep
EOF

The above would run curl -o photo-of-a-cow.jpg http://example.com/img/cow.jpg, etc.

JOB CONTROL

The default is to run several such jobs in parallel. Once a job is finished another one is started, until there are no more jobs to run.

The number of jobs run at the same time is controlled by the --jobs option. The default is one per CPU (but not necessarily one on each CPU).

If a job fails (exits with a code other than 0), no more jobs will be started. ppll attempts to stop other running jobs.

The --force option tells ppll to not care about failed jobs and always keep starting new ones. Any failed jobs are still noted and will still affect ppll’s own exit code.

SERIAL LINES, PARALLEL FIELDS

This feature deserves its own section.

It can be used to parallelise heavy tasks, while also making sure certain tasks are run in a particular order.

Each line of input will create a ‘group’ of parameters, that will be run before the next group starts. Each line is split up into individual parameters (using the IFS environment variable), that will be run at the same time.

An example:

ppll --slpf wake-up.sh <<EOF
rooster
hens chickens
cows pigs sheep
cat
EOF

The above would run wake-up.sh first for rooster. Then it would run it for hens and chickens, at the same time, but after the rooster job has completed. Then cows, pigs and sheep in parallel, and so on.

OPTIONS

GENERAL

--help

-h

-?

Print (this) help and exit.

--version

-V

Print ppll’s version and exit.

PARAMETERS AND JOBS

--command COMMAND

-C COMMAND

Run COMMAND as a shell command with each parameter passed as an argument. This allows for shell constructs, such as piping, redirecting, etc.

If used then the rest of the command-line arguments are used as parameters, instead of a command.

More or less the same as:

ppll $SHELL -c <COMMAND> -

($SHELL defaults to /bin/sh.)

--delimiter DELIMITER

-d DELIMITER

Use DELIMITER, as a regex, to split lines into fields. Implies --fields.

--empty

Include empty parameters. These are otherwise discarded.

--fields

-f

Alias for --mode fields.

--force

-k

Keep starting new jobs even if a job has failed.

--jobs N

-j N

Run (up to) this many jobs in parallel.

--lines

-l

Alias for --mode lines.

--mode MODE

How input should be turned into parameters. MODE is one of:

auto (default)

If there’s only one line of input, behave as if fields, otherwise as if lines.

fields

Split lines into fields and use each field as a parameter. Splitting is done using a delimiter set with --delimiter, or the IFS environment variable.

lines

Use each line as a parameter.

slpf

Split each line into parameters. All parameters from a single line will be run in parallel, but lines will be run serially.

See also: --fields, --lines, --serial-lines-parallel-fields

--random

-R

Shuffle the parameters.

--replace-string STRING

-replstr STRING

-I=s STRING

Use STRING instead of {} as a placeholder for the parameter.

--reverse

-r

Use the parameters in reverse order.

--sequence SPEC

-s SPEC

Use SPEC as parameters.

SPEC is in one of the forms:

N..M

Parameters are from N to M, inclusive.

M

Parameters are from 1 to M, inclusive.

--serial-lines-parallel-fields

-slpf

Alias for --mode slpf.

OUTPUT

--colour

--color

--no-colour

--no-color

Turn on or off ANSI colours in the output.

The default is to enable colours if STDOUT is a TTY.

--markers

--no-markers

Prefix (the default), or don’t prefix, jobs’ STDOUT and STDERR output with > or ≫, respectively.

--quiet

-q

Print less. Can be repeated for less and less.

--summary

-S

--no-summary

Print (the default if there’s more than one parameter), or don’t print, a summary of how many jobs ran and which of them failed.

--timestamp-format FORMAT

Format timestamps using FORMAT. See DateTime::strftime.

The default is %T.%3N.

Implies --timestamps.

--timestamps

-t

Prefix lines from jobs’ output with a timestamp. The time printed will be the approximate time the line was printed by the job.

--verbose

-v

Print more. Can be repeated for more and more.

DIAGNOSTICS

CONFIGURATION AND ENVIRONMENT

ppll uses some environment variables in some cases:

COLUMNS

If set then used when printing things across the whole width of the terminal. If not set then ppll will try to work out the terminal width in other ways, falling back to 80.

IFS

Used for splitting lines into fields, e.g. with --fields or --serial-lines-parallel-fields.

SHELL

Used with --commmand (-c).

DEPENDENCIES

INCOMPATIBILITIES

BUGS AND LIMITATIONS

AUTHOR

LICENSE AND COPYRIGHT

cpanm App::ppll

perl -MCPAN -e shell
install App::ppll