Proc-Spawner version 0.12 (EXPERIMENTAL)
========================================
Compiles and runs shell one-liners (or portable shell scripts) from Perl
without /bin/sh and "without" fork, matching dash/bash: the strings are
parsed by one of the Proc::ShellSubset parsers and run
by Proc::ShellSpawn, which starts programs with posix_spawn(3) and runs loops,
subshells, $( ) and pipelines of compound commands inside the Perl process.
(All the started programs are children of the initial Perl process, as opposed
to descendants.)
Proc::Spawner start children (every posix_spawn setting)
Proc::Spawner::Loop the event loop, one interface on POSIX and on Windows
Proc::Spawner::Graph the executable graph of children, pipes and files
Proc::ShellSpawn the shell layer
Proc::ShellSubset, Proc::ShellSubset::Small, Proc::ShellSubset::Large
parsers of three sizes of the shell language
WHAT IS OPEN: the file TODO lists every postponed thing in one place, first
the decisions waiting for the author (with where to read and how to act),
then the work, then what to watch.
HOW CLOSE TO DASH AND BASH (0.12; doc/large-subset/large-subset-state.md)
Snippet by snippet, across environments, with every difference of ours and
the ruling that allows it: doc/large-subset/large-subset-environment.md,
section 11 (post-0.11).
To redo any of these measurements: make corpus-help (after perl Makefile.PL;
Linux, root for the runs; doc/test-pools.md, section 0).
Of 41,952 shell strings from six public collections (tldr, NL2Bash,
ShellCheck's tests, the bash Cookbook, the pure sh and bash bibles), the
large parser accepts 40,410. The rest are mostly bash-only syntax, which the
subset rejects on purpose; 44 are forms both shells run alike, the work list
(section 4). Each accepted string was then run by dash, by bash --posix and
by Proc::ShellSpawn, isolated, with every program but the plain text tools
replaced by a recorder of its arguments, each run with a fresh /tmp, a
fixed /dev/urandom and fixed file times, and background jobs still running
when a shell ends given up to 5 seconds (section 7, the run of post-0.11;
the limits: known issue X27). Ours ran echo, printf, test, [, true, false
and pwd inside Perl, its default since 0.10:
run same only the dash and bash dash and bash
messages agree, ours differ
differ differs
coreutils involved 1,333 887 310 4 129
no coreutils 39,077 14,899 23,080 2 1,084
all 40,410 15,786 23,390 6 1,213
(and 15 strings that never end, or not in ours: section 7)
"Coreutils involved": the string calls echo, printf, test, [, kill or pwd,
which the subset runs as programs where the shells have builtins. "Only the
messages differ" is mostly dash's "x: not found" against bash's "x: command
not found". Where dash and bash differ, the subset follows dash in 965,
bash in 98, and neither in 150 (mostly $0, process IDs, times, and the
interactive builtins). The 6 where ours differs: printf -v (2, decided),
echo into a broken pipe in a subshell (1), exec command (1: known issue X56,
a bug of ours, open), a race (1), and ls -l of / (1: times outside the run).
Of the 40,410, 365+27=392 are shown not to depend on that environment at
all (N+M: N found by the programs, M by reading the snippets: section 1
there), and 34,304 more not under named assumptions (mostly that a program
does the same in each run), so their agreement is a real one; 2,140+14=2,154
more if also every value they take from the environment only passes through
the shell (664+7 unchanged, 1,476+7 split as well); the other 3,508+52=3,560
may depend on files, variables, programs' output, /tmp and the
like (known issue X27: doc/large-subset/large-subset-environment.md, and a
record by snippet ID,
doc/large-subset/large-subset-provenance.tsv). Runs with the environment varied
(a second case of POSIXLY_CORRECT, the locale, SIGPIPE, /proc; a fresh /tmp,
a fixed /dev/urandom) are footnotes to these figures: of the samples tried,
few came out otherwise (section 9 there).
Across environments (post-0.11; section 11 there: two trees of every
snippet, over the base run and the example environments of steps 2 and 3):
with messages counted nowhere, wherever a redirection sends them,
ours = dash = bash in every environment for 38,603; in 1,793 ours differs
from a shell somewhere: 1,620 allowed by the author's rulings (mostly A5,
ours with dash where the shells differ), 88 awaiting a decision, 40 chance,
45 not yet allowed. Each echo, printf, test, [, kill or pwd those snippets
ran (341 distinct calls) was also run alone by coreutils, dash's builtin,
bash's builtin and ours: ours = coreutils in all.
For the programs alone, each call the strings make was also run as dash's
builtin, as bash's builtin and as the program: of 650 calls (1,154 uses),
515 (995) agree, 41 (45) differ in messages only, and 94 (114) differ:
mostly echo -e and backslashes (dash's builtin against bash and the
program), unary -a -o -v in [ ] (bash's builtin against dash and the
program), and kill with a word that is no process ID (dash against bash and
the program). Section 7 has them all, with the decisions taken so far.
Tested on Linux with glibc 2.39 (also built as if some or all of the
extensions were missing: PROC_SPAWNER_WITHOUT="PIDFD_SPAWN ..." perl Makefile.PL),
and with /proc hidden, as on FreeBSD (RT #182479; FreeBSD itself is not tested
here): without /proc, t/graph.t and two batteries of t/dash.t are skipped,
and Perl's own writes into a pipe the caller gave may block (known issue X32). Parts that flip process-wide
state for a while (umask, signal actions) are bullet-proof only in
non-threaded execution.
INSTALLATION
To install this module type the following:
perl Makefile.PL
make
make test
make install
DEPENDENCIES
Only modules that come with Perl. Building needs a C compiler.
Makefile.PL finds whether the libc has posix_spawn, and which of its
extensions (chdir, fchdir, closefrom, tcsetpgrp, pidfd_spawn, cgroup), by
compiling a call to each. What the libc lacks is done in Perl's process
around the call where that can be undone (chdir, the fds, the signal mask,
...); the rest fails with ENOTSUP, or is dropped, as
%Proc::Spawner::UNSUPPORTED says. Without posix_spawn the primitive is fork
and exec. Resource limits, ignored signals and the nice value, which
posix_spawn has no option for, are set around the call too.
Tests: t/spawner.t (Proc::Spawner and the graph), t/graph.t (hand-written
graphs), t/parsers.t (1,117 strings through the parsers), and t/dash.t
(every battery string run by dash and by the shell layer; skipped without
dash; t/dash-known.txt lists the known differences). Their fixtures are
in t/: t/batteries/, t/parser/, t/sgraph/, t/try.pl (runs one string: perl t/try.pl 'STRING').
COPYRIGHT AND LICENCE
Copyright (C) 2026 by Ilya Zakharevich
This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself, either Perl version 5.22.2 or,
at your option, any later version of Perl 5 you may have available.
Keyboard Shortcuts
Global
s
Focus search bar
?
Bring up this help dialog
GitHub
gp
Go to pull requests
gi
Go to GitHub issues (only if GitHub is preferred repository)