ShellSpawn: running ShellSubset parse trees with posix_spawn (2026-11-01)

Code: claude/pspawn-tree.tar.gz-base64.txt (base64 -d, tar xzf): PosixSpawn/ (XS binding), lib/ShellSpawn.pm (the executor), try.pl, compare.pl, battery.txt, fdbattery.txt, fds, hyg.pl, traces.txt; since 2026-11-02 the executor runs on lib/Proc/Spawner.pm (starting children, the event loop) and lib/Proc/Spawner/Graph.pm (the executable spawn graph), both EXPERIMENTAL and described in pspawn-spawner.md; hand-written graphs: sgraph/, described in pspawn-graph.md; tests of the two modules: t/spawner.t; the large subset and the extension points (EXPERIMENTAL): lib/ShellSpawn/Large.pm, described in pspawn-extensions.md. The executor alone: claude/pspawn-ShellSpawn.pm. Build: cd PosixSpawn && perl Makefile.PL && make. Run: TRACE=1 perl try.pl 'COMMAND'.

1. The scenario

As decided on 2026-11-01: this module is the only code that starts children of the process; it may set SIGCHLD as it likes; posix_spawn is available. So: no fork in Perl, no wraparound, waitpid(-1, WNOHANG) is allowed (every child is ours).

2. The posix_spawn binding

Neither CPAN module exposes file actions: Proc::FastSpawn has spawn, spawnp and fd_inherit; POSIX::RT::Spawn has one spawn like system LIST. Hence PosixSpawn (XS, ~150 lines):

PosixSpawn::spawn(FILE, \@ARGV, \@ENV_OR_UNDEF, \@ACTIONS, \%ATTR) returns the pid, or the empty list with $!. Actions in order: [dup2 => FROM, TO], [close => FD], [open => FD, PATH, FLAGS, MODE], [chdir => DIR], [fchdir => FD], [closefrom => FD] (the last three are glibc extensions). [tcsetpgrp => FD] (glibc). Attributes: search (posix_spawnp), sigdefault => [SIGNUMS], sigmask => [SIGNUMS], pgroup => PGID, setsid => 1, resetids => 1, sched => [POLICY, PRIORITY], cgroup => FD (glibc), pidfd => 1. With pidfd, glibc 2.39's pidfd_spawn is used and spawn returns (PID, PIDFD).

PosixSpawn::fpe_at_exec() says what Perl's own exec would give a child for SIGFPE: 1 ignored, 0 default, -1 this Perl leaves SIGFPE alone. Perl ignores SIGFPE for itself at startup (PERL_FPU_INIT) and, around exec, restores the disposition it found then (PERL_FPU_PRE_EXEC, the global PL_sigfpe_saved).

Facts checked on glibc 2.39 (Linux):

3. Objects

4. From a command to posix_spawn

For a simple command:

  1. Expand the settings (NAME=value, left to right, each seeing the ones before), then the words. A $( … ) runs first, asynchronously.
  2. Apply the redirections to a copy of the context's map, in the order written, in the parent: n>&m copies entry m, n>&- sets undef, n>file opens the file in the parent (sysopen, close-on-exec) and puts the new end in entry n. So 2>&1 >f and >f 2>&1 come out as in sh, by construction.
  3. Builtins (:, exit, break, continue, read) run in the parent with that map. Otherwise: find the program in PATH, and spawn.
  4. File actions from the map (Proc::Spawner::actions() since 2026-11-02, on fd numbers; the graph leaves out a parent fd that a child gets under its own number): for every target fd t whose end has parent fd f ≠ t, dup2(f, t); for undef, close(t). A source fd f that is itself a target with another end would be overwritten before its turn: it is first copied to a scratch fd above every fd involved, and the scratch fds are closed at the end. A target whose end is already at the same number gets dup2(t, t) only if the parent's fd is close-on-exec (to clear the flag). Then chdir if the virtual current directory is not the real one; sigmask => []; sigdefault = the signals the Perl program has set to IGNORE since the module was loaded.
  5. Release the command's own opened files (their creator's holds), then the contexts in $tail.

Example from traces.txt: echo $x 3>&1 1>&2 2>&3 >/dev/null gives the map {1: '/dev/null' at parent fd 3, 2: our fd 1, 3: our fd 1}. Both parent fds 3 and 1 are targets with other ends, so: dup2 3->10, dup2 1->11, dup2 10->1, dup2 11->2, dup2 11->3, close 10, close 11.

5. When the parent forgets an end

Every construct is run as x_*(context, node, $tail, $k): $k is the continuation that gets the status; $tail lists the contexts to release right after the last process the construct can start has started. That is the rule of 2026-10-30: forget an end once all processes that could use it have started.

Example (traces.txt), { false || cat /etc/hostname; } | wc -l >/dev/null: the read end is closed right after wc starts; the write end stays open while false runs (because cat may follow) and is closed right after cat starts.

6. The loop

One loop, loop_until(DONE) (since 2026-11-02 Proc::Spawner::wait_until): runs queued continuations; reaps with waitpid(PID, WNOHANG) for each of its own children (until 2026-11-01 waitpid(-1, WNOHANG), which took other code's statuses); then selects on a self-pipe written by the SIGCHLD handler (installed with SA_RESTART while the loop runs, the previous handler restored after) and on the fds being read for $( … ) and read. Since 2026-11-01 the byte is written by a C handler in the binding (PosixSpawn::chld_to_fd; decision D1, EXPERIMENTAL, awaiting approval, pspawn-extensions.md), so select waits without limit. The fallback, used only without that function, is a deferred Perl handler with select waking every 50 ms ($Proc::Spawner::TICK): an immediate handler corrupted Perl's heap, and a deferred one can miss a child that ends just before select starts waiting (known issue X14; perl-signals-findings.md, S1–S3). The final fix is decision D1 on the page "Child Exit Wake-up". Every continuation is queued, never called from another one, so a loop of builtins (while :; do :; done) runs in constant memory. Each on_read is one read; the callback asks again. read reads one byte at a time, so later commands get the rest of the input, as in sh.

run(STRING) returns when the foreground commands have finished. A background chain started by it goes on at once, also between run calls (since post-0.04, the user's choice B of question K3, 2026-11-03): while background jobs remain, Proc::ShellSpawn keeps a deferred SIGCHLD handler installed, which calls Proc::Spawner::poll() (what is ready now, without waiting) when a child ends; it keeps the program's $? and $! as they were (X55). The costs, documented: while such jobs remain, the program's own sleep and select return early whenever a child ends (they are interrupted by the signal; doc/perl-signals-findings.md, the consequence of item L5); a program's own $SIG{CHLD} code is called after ours, and a $SIG{CHLD} set later replaces ours (the jobs then go on only in later run calls). A chain that waits for Perl to read a pipe ($( ), read) may still stall if the writing child fills the pipe and so never ends. An END block finishes the jobs before Perl exits. Until 0.04 the chains went on only inside run and at exit (known issue X1).

7. Tested (2026-11-01)

8. Not done, or different (the current list: pspawn-known-issues.md)

9. posix_spawn's features: what is used (2026-11-01)

| Feature | Used | Why, or why not | |---|---|---| | adddup2 | yes | every pipe end and redirection; scratch fds for conflicts (section 4) | | addclose | yes | n>&-, and the scratch fds | | addopen | no | the parent opens files itself: it can then say which redirection failed, and the commands of a group share one open file | | addchdir_np | yes | when the virtual current directory (after cd in a subshell) is not the real one | | addfchdir_np | no | would survive a renamed or removed directory (known issue X9) | | addclosefrom_np | no | sh passes fds it inherited without close-on-exec, and so do we | | addtcsetpgrp_np | no (bound) | job control only; fails with ENOTTY without a terminal | | setsigmask | yes | always empty | | setsigdefault | yes | SIGFPE unless it was ignored when Perl started (as Perl's own exec does); every signal Perl code has set to IGNORE since the module was loaded; SIGINT and SIGQUIT for foreground children only | | setpgroup | no (bound) | a non-interactive sh puts everything in one group; needed to kill a whole background job | | RESETIDS, setsid, scheduling, setcgroup_np | no (bound) | sh does not change them either; tested in the binding | | pidfd_spawn (glibc 2.39) | no (bound) | a pollable fd per child would replace the SIGCHLD self-pipe; but the child is still an ordinary child, so other code's waitpid(-1) can still take its status | | environment (envp) | yes | the exported variables (every name from %ENV that is an identifier, as dash; PWD kept up to date by cd) plus the command's own settings |

Signal dispositions, checked against dash with sigenv.txt (11 of 12 agree; the 12th is umask, outside the subset): a foreground child has nothing ignored or blocked; a background child, also inside a subshell or a pipeline, has SIGINT and SIGQUIT ignored; a signal ignored when Perl started stays ignored, SIGFPE included (checked both ways with trap '' FPE in the parent of Perl). While run() runs, Perl itself ignores SIGINT and SIGQUIT, as Perl's system does.

Not set through posix_spawn, inherited as in sh: umask, resource limits, nice value, process group and session, controlling terminal.