/ 
IPC-Run3-Shell-0.58
River stage one • 1 direct dependent • 1 total dependent
/ IPC::Run3::Shell

Name

IPC::Run3::Shell - Perl extension for calling system commands as if they were Perl functions

Synopsis

#!/usr/bin/env perl
use warnings;
use strict;

use IPC::Run3::Shell qw(echo cat ps cp);

my $foo = echo("howdy", "<funny>", "world");
print $foo;                              # prints "howdy <funny> world\n"

my @passwd = cat("/etc/passwd", {chomp=>1});
print "$_\n" for @passwd;                # prints lines of /etc/passwd

print ps -ww;                            # prints output of "ps -ww"

cp("/etc/passwd", "/tmp/passwd.orig");

The above is adapted from Larry Wall's 1994 email about Shell.

Reference / Cheat Sheet

use strict;
use warnings;             # any problems with system commands are warnings
use IPC::Run3::Shell                         # nothing exported by default
                     qw/ :FATAL /;           # fatal warnings (optional)

# ### Basic Usage (Functional Interface)
use IPC::Run3::Shell qw/ echo cat /;    # these functions call system cmds
echo "Hello, World!";                   # void context: display output
my $groups = cat "/etc/group";          # scalar context: get all output
my @passwd = cat "/etc/passwd";         # list context: get output lines
                  # Note: option "both" changes return values, see below
my $foo = echo ' "" \\ ; $HOME 1>&2 ';  # shell metachars are unaffected

# ### OO Interface - methods become system commands
my $sh = IPC::Run3::Shell->new( show_cmd=>1 );    # independent options
$sh->cp(-v, '/etc/passwd', '/tmp/passwd.copy');   # Perl note: -v eq "-v"

# ### Options
  # set default options in "use", or per command as first/last arguments
use IPC::Run3::Shell { chomp=>1 }, qw/ ls /;      # chomp the return value
my @files = ls -lA, '/etc', {chomp=>0};           # override an option
my $motd = cat "/etc/motd", {allow_exit=>[0,1]};  # allowed exit codes
my @words = echo {irs=>"-"}, -n, "A-B-C-D";       # input record separator
my ($out,$err,$exitcode) = echo "Foo", {both=>1}; # return both STDOUT/ERR
my $out_and_err = echo "FooBar", {both=>1};       # STDOUT+STDERR merged
echo "Foo", "Bar", {fail_on_stderr=>1};           # fail on any STDERR
my $outfilt = echo "Hello", { stdout_filter => sub { s/[eo]/X/g } };
  # provide a callback that modifies $_ to filter stdout return value
my $bla = echo {show_cmd=>1}, "blah", "blah";     # show cmd (debugging)
  # IPC::Run3 options binmode_* and append_* are also supported

# ### I/O Redirection with Options stdin, stdout, and stderr
use IPC::Run3::Shell qw/ :run /;        # and a demo of the run() function
my ($i,$o,$e) = ("Hello,");
run '/usr/bin/perl', '-pe', 'warn "World!\n"',    # a tricky example
    {stdin=>\$i, stdout=>\$o, stderr=>\$e};
  # Note use of "stdout" option means return value will be the exit code
print "$o $e";                                    # prints "Hello, World!"
# ## A *simplified* reference for stdin/stdout/stderr options:
  # undef = inherit filehandle        # \undef = /dev/null
  # $filename = use that filename     # \*FILEHANDLE = use that filehandle
  # \$scalarref = use that string     # \@arrayref = lines* to/from array
  # \&coderef = called for each line*     ( * no newlines added to stdin )

# ### Aliasing
use IPC::Run3::Shell [ psw => {chomp=>1}, '/bin/ps', -ww ];
my @ps = psw;        # function psw() is now an alias for "/bin/ps -ww"

Description

This module's goal is to provide a relatively transparent interface to system commands from regular-looking Perl functions. It is built on the module IPC::Run3 and is heavily inspired by the module Shell, but with several important differences and added features:

  • functions are only exported at the request of the user (no autoloading in the user's namespace - unless you explicitly ask for it)

  • system commands and their arguments are never* sent through the shell (so you don't have to worry about escaping shell metacharacters)

  • failures of the system commands are (optional) warnings, which can be made fatal (in the spirit of autodie or bash's set -e)

  • advanced capturing of STDIN, STDOUT and STDERR

* this should hold true for all *NIX systems, and on Windows, IPC::Run3 versions 0.047 and up use Win32::ShellQuote so that you shouldn't have to worry about shell metacharacters there either.

Read on for more information and examples!

Version

This document describes version 0.58 of IPC::Run3::Shell.

Security

Calling system commands can be dangerous. This module does not provide any security checks. All of the usual warnings apply, see perlsec!

General

This module provides Perl functions with the following basic pattern: The name of the function (or method, if you're using the OO interface) becomes the name of the system command that is executed when you call the function, and any arguments given to the Perl function become the arguments of the system command. Options may be given to the Perl function as a hashref, see "Options" for details. The return value of the Perl function depends on a few things, this is described in "Return Values", but normally it'll be either the output of the system command or its exit code. If something goes wrong, an optional and optionally fatal warning is raised, see "Warnings". Additional features are described in the rest of this documentation, such as "Additional Functions".

The command and its argument list are stringified and should only consist of plain scalars or objects that stringify, as well as option hashrefs. Other references, objects, or undefs will trigger a (optional/optionally fatal) warning. Objects that use overloading but don't support stringification will cause a fatal error as they normally would. (Note that before v0.53, objects that stringified also triggered a warning, and references were passed through unchanged to IPC::Run3 instead of being stringified. undefs have been stringified to "" since v0.51.)

A note on terminology: This documentation uses the term "system command" to refer to external executables / programs / scripts; you may prefer to use a different term and that's okay ;-)

Warnings

This module ties into Perl's warnings system and provides a warnings category with the same name as the module. Several possible error conditions generate optional warnings, which can also be made fatal.

Any "modern" Perl script should start with the following, which enables all warnings:

use warnings;
use strict;    # or "use v5.12;" or a newer version

In addition, you can turn the warnings into fatal errors like this (added in v0.56):

use IPC::Run3::Shell ':FATAL';

Which is shorthand for this (see perllexwarn):

use IPC::Run3::Shell;
use warnings FATAL=>"IPC::Run3::Shell";

Note there are a few error conditions that are currently always fatal: incorrect arguments to use, incompatible options, a completely empty command, and a failure of IPC::Run3. (In the future, some of these may be reduced into warnings, if a useful default behavior exists.)

(Hint: If you want to catch fatal errors, you can look into Try::Tiny or a similar module.)

Functional Interface

use IPC::Run3::Shell {show_cmd=>1},  # default options for the following
    qw/ echo cat /,       # these Perl functions will call system commands
    [ lsl => 'ls', '-l' ];     # alias function "lsl" to command "ls -l"
echo "Hello,", "World!";       # similar to system() (due to void context)
my @passwds = cat {chomp=>1}, "/etc/passwd";  # get output lines as list
my $filelist = lsl;            # get the full output of the command

Arguments to use

This section describes the arguments after the module's name and optional module version, i.e. the "LIST" part of Perl's use function.

The functions you name in use will be made available in your script, these Perl functions will be translated into the system commands of the same name following the usual pattern described in "General".

Default options may be specified as one or more hashrefs at the beginning of the list, see "Options". These default options apply only to the functions you specify in use, including "run", and with the exception of "make_cmd".

Aliasing

Passing an arrayref to use allows you to alias functions to commands, with a full path name of the executable, any number of arguments, as well as default options that only affect that command. The first item in the array must be the function name, followed by any default options, then the command name and any arguments.

use IPC::Run3::Shell [ 'e', {show_cmd=>1}, 'echo', '-n' ],
    [ d => '/bin/date' ];
e d '+%Z';      # calls "/bin/date +%Z" and passes its output to "echo -n"

OO Interface

use IPC::Run3::Shell;                        # nothing exported by default
# the following default options apply to all commands called on the object
my $sh = IPC::Run3::Shell->new( show_cmd => 1 );
$sh->echo("Hello, World!");
my $x = $sh->echo({show_cmd=>0}, "Test!");   # overriding a default option

The OO interface has the advantages that no functions need to be imported to your namespace at all, and that each object has its own independent set of options.

When you make an object with new, almost all of the methods you call on that object are translated into system commands following the usual pattern described in "General". Default options, which apply to all commands called via the object, may be given to new as a plain hash (not a hashref).

Methods are provided via Perl's autoloading feature. So, the only methods that will not be available as system commands are the built-ins of the UNIVERSAL class, as well as "AUTOLOAD" and "DESTROY". So if you want to run a command named, for example, "can", you can use "run" to do that (note that "run" is not available via the OO interface).

(Note on internals: The objects will actually not be of type IPC::Run3::Shell, they will be of type IPC::Run3::Shell::Autoload, so they have a clean namespace.)

Additional Functions

run

use IPC::Run3::Shell qw/ :run /;               # NOTE the colon
my $root = run "grep", "root", "/etc/passwd";  # run system command "grep"
# Perl's built-in "grep" function remains unaffected

run mostly works like the other functions provided by this module (see "General"), with the difference that the name of the system command is the first argument to the function. This is useful, for example, if you don't want to override Perl built-in functions, or if you want to call a command with the full path.

It can be accessed either by specifying :run (note the colon!) to use (in this case it is affected by default options, if you specified them), or by calling IPC::Run3::Shell::run() directly (in this case it is not affected by default options, only by those that you give to IPC::Run3::Shell::run() directly).

make_cmd

use IPC::Run3::Shell qw/ :make_cmd /;  # NOTE the colon
my $pl = make_cmd 'perl', '-le';    # $pl->(ARGS) will run "perl -le ARGS"
$pl->('print "Hello, World!"');

make_cmd is an advanced function which gives you full customizability over the generated Perl function and the executed system command. (It is actually the central function of this module.)

make_cmd takes optional options (hashref(s)), a command name (optionally with full pathname), optional command-line arguments, and returns a code ref which behaves as described in "General" and "Return Values" and, when called, executes the specified system command. Any additional arguments to the code ref become additional command-line arguments. The code ref may also be given additional options.

It can be accessed either by specifying :make_cmd (note the colon!) to use or by calling IPC::Run3::Shell::make_cmd() directly. Note that make_cmd and the functions it generates are only affected by the options passed directly to them, and never by default options given elsewhere (such as to use).

import_into

Lets you export functions from this module into any package of your choosing. The first argument is the package name, the rest are the usual arguments to import.

use IPC::Run3::Shell;
IPC::Run3::Shell->import_into('Foo::Bar', 'echo');
{
    package Foo::Bar;
    echo("Hello, World");
}

The import_into method was added in v0.52.

Options

Options can be set in several different places:

  • In the functional interface, default options can be given to use, when making aliases, or to "make_cmd". They are always specified as one or more hashrefs as the first argument(s). In the case of use, default options apply to all functions named in the use statement (with the exception of "make_cmd").

  • In the OO interface, the default options are passed to the constructor, and they then apply to all commands called on that object. They are not affected by use. Each object has its own independent set of default options. Note that the options to new are not specified as a hashref but as a plain hash.

  • In any case, options can always be specified or overridden by passing one or more hashrefs as the first and/or last argument(s) to any individual command.

Whenever multiple option sets are present, they are cumulative, with options specified later overriding the earlier ones. Note that there are a few options, such as "stdout", that cannot be un-set. Specifying unknown options triggers warnings and is strongly discouraged for forwards compatibility.

allow_exit

Either a single value or an arrayref of allowed exit values. Defaults to only zero (0). May also be set to the string "ANY" to mean any exit code is allowed. If set to undef, resets the option to its default value. Any other non-numeric values or an empty arrayref cause a warning. This option does not make much sense unless you have warnings enabled.

irs

Sets a local $/, see Perl's input record separator. Note that, like "stdout", irs => undef has a meaning, so the mere presence of this option in the options hash means the option is set.

chomp

Setting this boolean option causes the return value(s) of the function to be chomped. This option has no effect if you're using the option "stdout".

stdout_filter

Setting this option to a coderef causes this coderef to be called to filter the command's STDOUT. In list context, the filter is applied per line, while in scalar context, the filter is applied over the entire output. The coderef should modify $_ to modify the output string(s); its return value is ignored.

For example, say you've got a command that outputs JSON, you can turn the output from that command into a Perl data structure like this (this is just a demonstration using echo that passes its input through):

use JSON::PP qw/encode_json decode_json/;
use IPC::Run3::Shell [ foo => { fail_on_stderr => 1,
    stdout_filter => sub { $_=decode_json($_) } }, qw/ echo / ];

my $foo = foo( encode_json( { test=>'123' } ) );
print $foo->{test}, "\n"; # prints "123"

This option has no effect if you're using the option "stdout". It is applied after the chomp operation, if that option is set.

The stdout_filter option was added in v0.58.

both

Setting this boolean option has a different effect depending on whether the function is called in list context or in scalar context - see also "Return Values". You cannot use this option together with "stdout", "stderr" or "fail_on_stderr".

In scalar context, the function returns the entire STDOUT plus STDERR of the executed command merged as one string. Note that the order of output on these two streams is not always well-defined; they may be mixed together. This is still useful if, for example, you know the command outputs on STDERR only and you want an easier way to capture it than the "stderr" option.

In list context, the function returns three values: the entire STDOUT as one string, the entire STDERR as one string, and the exit code of the command. This is somewhat similar to Capture::Tiny's capture used together with system (except that returns $? and not the exit code).

# call an external perl process for this example
use IPC::Run3::Shell {both=>1}, "perl";
my ($stdout, $stderr, $exitcode) =  perl {allow_exit=>123}, '-e',
  ' print "Foo\n"; warn "Bar\n"; exit 123 ';
# $stdout gets "Foo\n", $stderr gets "Bar\n" and $exitcode==123

The both option was added in v0.52.

stdin

stdout

stderr

use IPC::Run3::Shell {chomp=>1}, qw/ cat find /;
cat { stdin=>\"Hello, World!\n" };
# there will probably be some "permission denied" errors in /etc,
# so we're allowing for an exit value of 1 and are capturing stderr
my $rv = find { allow_exit=>[0,1] }, '/etc', '-name', 'passwd',
    '-print0', { stdout=>\my @files, stderr=>\my $err, irs=>"\0" };
print "find exited with $rv\n";
s/\0\z// for @files;  # the "chomp" option doesn't work with "stdout"
print "Found files: ".join(', ',@files)."\n";
print "## Warnings ##\n$err##\n" if $err=~/\S/;

These options give you powerful redirection of STDIN, STDOUT and STDERR of the executed command. The possible values are the same as in IPC::Run3; the following is a brief summary of the possible values:

undef

inherit the filehandle from parent

\undef

\dev\null (or equivalent)

scalar

a filename

filehandle

use that filehandle

scalar reference

read/write from/into that scalar (string)

array reference

read/write from/into that array, splitting output (not input) "lines" on $/

code reference

For stdin, the sub is called repeatedly and its return values are fed to the process until the sub returns undef. For stdout and stderr, the sub is called with each "line" of output (split on $/ as usual).

Note that if you specify the option stdout, the function will always return the exit code of the command, and not its output - see "Return Values". stdin and stderr do not have this effect.

Since setting an option like stdout => undef has a meaning, the mere presence of any of the hash keys stdout or stderr in the options hash will be interpreted as the respective option being set. This also means that these options cannot be un-set by overriding them. So normally, you'll want to set these in the per-command options only, and not in the default options.

Note this module does not make any guarantees about the capabilities of IPC::Run3 to redirect streams! For example, at this time of writing, IPC::Run3 does not support swapping of STDOUT and STDERR. Also, you will get unexpected results if you attempt to redirect STDOUT to STDERR and capture STDERR at the same time. (It is at the moment possible to redirect STDOUT to STDERR or vice versa if you don't touch the other.) If IPC::Run3's behavior changes in the future, that behavior will simply be passed through by this module.

You can't use the stderr option together with "fail_on_stderr". You can't use stdout or stderr together with "both".

fail_on_stderr

If this boolean option is set, any output to STDERR (except for a single $/) is considered a fatal error. This option may not be specified together with "stderr" or "both" (this is currently a fatal error). Note that turning this option on means that STDERR is always captured and not displayed.

show_cmd

Setting this option to a true value causes the command which is about to be executed to be printed to STDERR. You may also specify a filehandle / glob reference here to have the command printed there (e.g. show_cmd => \*STDOUT).

This is meant for debugging and/or user information only, as the output may not be safely escaped, so don't try to parse this output!

Options from IPC::Run3

The following options may be specified and they will be passed through to IPC::Run3.

  • binmode_stdin

  • binmode_stdout

  • binmode_stderr

  • append_stdout

  • append_stderr

  • return_if_system_error

    (Note this module turns this option on by default. It's recommended not to turn it off or you may get unexpected results in case of errors.)

Return Values

In the following table, "OUT" and "ERR" refer to the STDOUT and STDERR of the executed command. The options "both", "stdout" and "stderr" are described in their respective sections.

          |    Void Context    |   Scalar Context   |    List Context    |
==========+====================+====================+====================+
 system() |                    | return undef       | return empty list  |
 fails    | also generates optional warning that can be made fatal[1]    |
----------+--------------------+--------------------+--------------------+
 Bad exit |                    | output as below, but may be incomplete  |
 code[2]  | also generates optional warning that can be made fatal[1]    |
==========+====================+====================+====================+
 Default  | OUT and ERR are    | OUT is returned    | OUT is returned as |
          | not redirected,    | as one big string  | a list of lines[4] |
          | they are passed    |    ERR not redirected by default[3]     |
==========+ thru to script's   +====================+====================+
 Option   | STDOUT and STDERR  | return merged OUT  | return a list of   |
 "both"   |                    | and ERR            | (OUT,ERR,exitcode) |
==========+====================+====================+====================+
 Option   | OUT is redirected as requested (ERR handled separately)      |
 "stdout" | Function always returns the exit code                        |
==========+==============================================================+
 Option   | ERR is redirected as requested (OUT handled separately)      |
 "stderr" |                                                              |
==========+==============================================================+

You should always be able to inspect the exit code of the system command in Perl's $? variable. See the documentation of Perl's system() for an example of how to interpret $?.

[1] This is described in "Warnings".

[2] If you're expecting a nonzero return code from a system command, see the option "allow_exit". Currently, a command being terminated due to a signal will generate a (optional/optionally fatal) warning and there is no equivalent of "allow_exit" for signals.

[3] STDERR can be handled via the options "stderr" or "fail_on_stderr".

[4] More exactly, the command's STDOUT output is split on the input record separator $/. Since that's the line separator by default, you'll normally get a list of the lines of the output, unless you change $/, which you can also do via the option "irs".

Portability

This module strives to be as portable as IPC::Run3. Before reporting bugs concerning portability, please first test if maybe it's a bug in IPC::Run3.

The tests for this module require that you have the perl executable in your PATH.

If you notice any other portability issues, please let me know!

Of course, scripts using this module to call system commands will often not be portable, but that's simply the nature of scripts that call system commands.

Caveats

A few things to be aware of:

As described in "Return Values", the functions provided by this module act differently depending on the context they are called in (void, scalar and list) and some of the options (e.g. "stdout" and "both"). While this is an attempt to provide a DWIM (Do What I Mean) interface, the difference in behavior might be subtle and lead to a bit of confusion.

Some functions are package methods, while others are not, i.e. don't mix up :: and ->. Examples: IPC::Run3::Shell->new(), IPC::Run3::Shell->import(), IPC::Run3::Shell->import_into(), IPC::Run3::Shell::run(), IPC::Run3::Shell::make_cmd(). (For the latter two, it's probably easier to use use IPC::Run3::Shell qw/:run :make_cmd/;.)

If you plan on subclassing this module, note that the OO interface doesn't follow traditional Perl OO design patterns (e.g. new isn't actually part of the package of the object).

Advanced Hints

Prototypes

Even though they're usually discouraged and often rightly so, there are still a few rare cases when prototypes are useful. One way to give the Perl functions prototypes is with Scalar::Util's set_prototype:

# a silly example for demonstration purposes only :-)
use IPC::Run3::Shell [ xuc => 'perl', '-e', 'print uc "@ARGV"', '--' ];
use Scalar::Util qw/set_prototype/;
BEGIN { set_prototype \&xuc, '$'; }     # make xuc() a unary operator
# normally xuc() would take all the arguments, now it only takes one
my $foo = join ",", xuc "a", "b", "c";
print $foo;                             # prints "A,b,c"

Importing at Run-Time

If you know the internals of Perl's use, you know that you can import functions at run-time too, as follows. For a description of the arguments of import please see "Arguments to use".

use IPC::Run3::Shell;
IPC::Run3::Shell->import('echo');
echo("foo");                       # NOTE parentheses are required here

If you like playing with globs, you could install the code refs created by "make_cmd" into your namespace manually. For example:

BEGIN {              # not required, could do this at run-time
    require IPC::Run3::Shell;
    *foo = IPC::Run3::Shell::make_cmd('echo','foo');
}
foo "bar";           # calls "echo foo bar"

Also, note the "import_into" function for more powerful exporting.

Calling the Shell

Warning: Make sure you understand the safety implications of this hint before you use it!

If you really want to invoke the shell and have your commands subject to its various interpolations and such, with all the possible safety implications (such as attackers injecting arbitrary commands into your shell!), you could do something like this:

use IPC::Run3::Shell [sh => 'sh', '-c'];
sh 'echo $HOME >&2';     # prints _the_shell's_ $HOME to _its_ STDERR

Autoloading

Warning: Make sure you understand the safety implications of this hint before you use it!

If you really, really wanted to have Shell's autoloading behavior back, with all the possible safety implications (such as any typos leading to the execution of system commands!), you can do this (added in v0.56):

use IPC::Run3::Shell ':AUTOLOAD';
echo("foo","bar");   # calls "echo foo bar" via autoloaded "echo"

Acknowledgements

Thank you to LanX for suggesting :AUTOLOAD and dolmen for a documentation suggestion.

See Also

IPC::Run3::Shell::CLIWrapper - makes it easier to build wrappers for command-line interfaces. This module was added in v0.58.

Larry Wall and others wrote the original Shell in 1994, with various contributions throughout the years. Shell is Copyright (c) 2005 by the Perl 5 Porters.

IPC::Run3 - I chose this module from the plethora of similar modules for several reasons: it handles all three STDOUT, STDERR and STDIN; it runs on Win32; it allows avoiding the shell; no non-core dependencies (except on Windows); and its test statistics are currently nearly perfect. While it doesn't support a few features that some of the more advanced modules do, like timeouts or interactively communicating with the subprocess, I hope it'll be enough for the majority of cases.

Shell::Tools - reduce boilerplate in Perl shell scripts, also includes easy access to this module!

Capture::Tiny - yet another capturing module. Its documentation includes a list of lots more such modules. There is also System::Sub, which does something quite similar to this module.

Env - tie environment variables to Perl variables, similar in spirit to this module

Author, Copyright, and License

Copyright (c) 2014-2020 Hauke Daempfling (haukex@zero-g.net).

This library is free software; you can redistribute it and/or modify it under the same terms as Perl 5 itself.

For more information see the Perl Artistic License, which should have been distributed with your copy of Perl. Try the command "perldoc perlartistic" or see http://perldoc.perl.org/perlartistic.html.