Proc::Spawner and its graph: the split of the executor (2026-11-02)
Status: EXPERIMENTAL. The user endorsed every choice experimentally (2026-11-02, 00:24) and asked for this split (00:32: "Please start"). The decisions taken while writing it, G1–G6 in section 6, are experimental too and wait for approval.
Until 2026-11-01 one module, ShellSpawn, did everything: the shell, the starting of children, the event loop, and a graph that only recorded and drew what happened. Now there are three layers, and the graph is what gets executed:
| Module | Lines (with POD, 2026-11-18) | What it does | Depends on |
|---|---|---|---|
| lib/Proc/Spawner.pm (+ Spawner.xs) | 686 (2026-11-02: 230) | starts a child with a given fd table and any posix_spawn setting; runs the one event loop that reports "a child ended", "an fd is readable/writable"; its XS part is the former binding PosixSpawn | – |
| lib/Proc/Spawner/Loop.pm (+ Loop/POSIX.pm, Loop/Win32.pm, win_loop.h) | 520 (since post-0.10) | the one event loop (W5): queued calls, timers, reads, writes, ended children, one interface on POSIX (select, SIGCHLD) and on Windows (an I/O completion port); Proc::Spawner's on_read, on_write, defer, wait_until, poll are its old names | Proc::Spawner's XS |
| lib/Proc/Spawner/Graph.pm | 299 | the graph: open files (ends) with hold counts, process nodes, Perl's own reads and writes; each process carries what to do when it ends; the notation, pictures, running hand-written graphs | Proc::Spawner |
| lib/Proc/ShellSpawn.pm (+ ShellSpawn/Large.pm, ShellSpawn/Builtins.pm, ShellSpawn/Programs.pm) | 981 (+319, +141, +376) | only the shell: words, redirections, contexts, builtins, traps, jobs | both |
lib/SpawnGraph.pm is gone; its notation, drawing and runner are in Proc::Spawner::Graph.
Renamed 2026-11-02, 07:30 (the user: "PosixSpawn should be subsumed by Proc::Spawner. The rest should be put into Proc::"; names option A). One distribution, Proc-Spawner, built at the top of the bundle with perl Makefile.PL && make. Older docs use the old names:
| Old name | New name |
|---|---|
| PosixSpawn (XS binding, its own directory and Makefile.PL) | part of Proc::Spawner: Spawner.xs at the top; the raw call PosixSpawn::spawn is now Proc::Spawner::spawn, an alias of the XSUB __posix_spawn (the user: the same interface should have a chance to work on POSIX and Windows, so no posix_ in its name), with a sixth, optional argument { OPTION => 'drop' | 'fail' }: what a port lacking an option (Windows) does with it, defaults in %Proc::Spawner::UNSUPPORTED ('*' => 'fail', EXPERIMENTAL); the call with the fd table and on_exit, formerly spawn, is now start; fpe_at_exec, chld_to_fd, chld_restore are in Proc::Spawner |
| ShellSpawn, ShellSpawn::Large | Proc::ShellSpawn, Proc::ShellSpawn::Large (lib/Proc/) |
| ShellSubset, ShellSubsetSmall, ShellSubsetLarge (in parser/) | Proc::ShellSubset, Proc::ShellSubset::Small, Proc::ShellSubset::Large (lib/Proc/); the test scripts are in t/parser/ (since 09:58) and find lib/ themselves |
Include paths are now lib and blib/arch. The A4 message says "Proc::ShellSubset::Large: … is not supported" since 0.02 (it said "ShellSubsetLarge: …" before).
1. Proc::Spawner: the small one
The functions below, and the raw spawn (the primitive with what it lacks done around it: section 8); nothing about pipes, files or the shell.
| Call | Meaning |
|---|---|
| start(PATH, ARGV, ENVP, FDS, %opt) (was spawn) | starts a child; returns its pid, or undef with $!. FDS = { CHILD_FD => PARENT_FD, or undef for "closed" }; fds not listed are inherited as they are (if not close-on-exec). FDS values may also be [PATH, FLAGS, MODE]: opened in the child. Options (2026-11-02, the user: "more or less all the fields of posix_spawn should be supported"): chdir, fchdir, sigdefault => [signals], sigmask => [signals], pgroup, setsid, tcsetpgrp, resetids, sched, cgroup, search, pidfd (then in list context the pid and a pidfd), umask, sigignore, rlimit, nice (M1), win_priority, win_job (W4), unsupported => HASH (passed to spawn), on_exit => CODE. Signals by name or number; an unknown name dies. The full list is in the module's header comment. |
| on_read(FD, N, CODE) | one read of at most N bytes when FD is readable: CODE->(BYTES), or CODE->(undef) at end of file |
| on_write(FD, BYTES, CODE) | writes BYTES as the reader takes them, never blocking Perl; CODE->() when done or the reader is gone |
| defer(CODE, ARGS) | queue CODE->(ARGS) |
| wait_until(CODE) | run the queue and wait for events until CODE->() is true |
Example: ls with its output in a pipe whose read end Perl reads, while ls's fd 2 is closed:
pipe my $r, my $w;
my ($out, $st, $eof) = ('');
Proc::Spawner::start('/bin/ls', ['ls'], undef, { 1 => fileno $w, 2 => undef }, on_exit => sub { $st = shift });
close $w;
Proc::Spawner::on_read($r, 4096, sub { defined $_[0] ? ($out .= $_[0], Proc::Spawner::on_read($r, 4096, __SUB__)) : ($eof = 1) });
Proc::Spawner::wait_until(sub { $eof && defined $st });
The rules, as decided on the page "Child Exit Wake-up":
- R1. Every callback runs from the queue, one at a time: never inside a signal handler, never inside another callback. A child that ends while a callback runs is noticed, and its
on_exitqueued, only after that callback returns. - D2. A callback never waits:
wait_untilcalled from inside one dies with "wait inside a reaction". - D1. While
wait_untilruns, SIGCHLD has a C handler (from the binding) that writes one byte into a pipe; the loopselects on that pipe. No Perl code runs inside the signal, and no wake-up is lost. Without the C function (another binding), a deferred Perl handler is used, andselectwakes every 50 ms. - D3. SIGCHLD is ours only while
wait_untilruns; the previous handler is put back afterwards. (Since post-0.04,Proc::ShellSpawninstalls a deferred SIGCHLD handler of its own betweenrun()calls while background jobs remain, the user's choice B of question K3; it callsProc::Spawner::poll().Proc::Spawneritself still keeps D3.)
2. Proc::Spawner::Graph: the executable graph
The graph says, for the current moment, which of our children, and which of Perl's own reads and writes, use which open file. Each process node carries what to do when it ends.
- Ends. An end is an open file the parent keeps: a pipe half (
r,w), a file, or one of the parent's own fds (parent). Whoever needs an end holds it (hold); the lastreleasecloses the parent's copy. Ends are close-on-exec, so a child gets exactly what its fd table gives it. Made by$g->new_pipe,$g->file_end(FH, MODE, PATH),$g->parent_end(N). - Processes.
$g->start(PATH, ARGV, ENVP, MAP, on_exit => HANDLER, …), whereMAP = { CHILD_FD => end, or undef }, starts the child and adds its node. The node records its edges (fd, how it is used, end), its pid and its state: running,exited N,signal N. - Perl's own reads and writes.
$g->perl_read(END, N, HANDLER)and$g->perl_write(END, BYTES, HANDLER)are nodes too (drawn as the process "Perl"), so the graph knows which ends the parent itself still uses, including ends the caller never sees. - Handlers. A handler is a code ref, or
[NAME, DATA…]with NAME a fully qualified sub name, its module loaded at first use (the user's choice for parse trees, 2026-11-01 16:08, carried over). When the process ends,NAME->(GRAPH, NODE, WAIT_STATUS, DATA…)runs from the queue. It may start new processes (each with its own handler), release ends, read and write; it never waits. So the graph grows and shrinks as it runs. - No reference cycles. The graph holds its processes; a process holds its ends and its handler's data; ends point back only weakly (
peer,g). Handlers get the graph as an argument, so their data never needs to hold it. A handler is dropped from its node once called. An ended process leaves the graph once it shares no end, or pipe, with a running process or with one of Perl's own reads and writes.
Example (from t/spawner.t): echo hello writes into pipe 1. When it ends, its handler starts cat reading pipe 1 and writing pipe 2, which Perl reads.
my $g = Proc::Spawner::Graph->new;
my ($r, $w) = $g->new_pipe; my ($pr, $pw) = $g->new_pipe;
sub then_cat { my ($g, $node, $st, $r, $pw) = @_;
$g->start('/bin/cat', ['cat'], undef, { 0 => $r, 1 => $pw });
Proc::Spawner::Graph::release($_) for $r, $pw }
$g->start('/bin/sh', ['sh', '-c', 'echo hello'], undef, { 1 => $w }, on_exit => ['main::then_cat', $r, $pw]);
Proc::Spawner::Graph::release($w);
Hand-written graphs in the notation (sgraph/*.sg) now run on this engine. after P1 fail becomes a named handler on P1 (Proc::Spawner::Graph::then) that starts the processes waiting for P1, or marks them never and releases their ends. sgraph/run-graph.pl checks every process's fds with the reporter program: 17 of 17 processes as drawn (6 examples).
Pictures. Proc::Spawner::Graph->new(dot => DIR) writes the graph after every event (start, exit, close) as DIR/trace-001.dot, …; the executor does this with TRACE=dot.
3. How ShellSpawn uses them
- One graph for the whole program,
$Proc::ShellSpawn::G, so a background job that outlivesrun()stays in it. - A context's map (child fd → end) is the
MAPgiven to$G->start. Contexts hold and release ends as before. - Each child's node carries the shell's continuation:
on_exit => ['Proc::ShellSpawn::resume', K], andresumecallsKwith sh's$?. $( … ),readand here-documents areperl_readandperl_writenodes.- What stays shell-specific is in
Proc::ShellSpawn::spawn: the PATH search and its messages, the environment, which signals children get back as default, the ENOEXEC fallback to/bin/sh.
4. Behaviour changes
- Only our own children are reaped (G2). The loop calls
waitpid(PID, WNOHANG)for each child it started, instead ofwaitpid(-1, …). So it no longer takes the statuses of the caller's own children (finding C2). If other code takes a status of ours first, the child is reported with status undef instead of being waited for forever; the shell gives$?= 1 for it (G3). On Windows other code cannot take it: the exit code stays with the process handle the module keeps until it has reported the child, so other code'swaitpidand ours both get it (checked under Wine, post-0.10,t/spawner.t). - The trace says
reaped PID: wait status W(the raw status) instead of sh's$?. - In pictures, an ended process shows
exited Norsignal N, and the Perl node also shows Perl's writes (here-documents), not only its reads.
Nothing else changed: on every battery the old and the new executor print the same output, messages and status for all 234 command lines (only pid lengths differ).
5. Tested at the split (0.01 to 0.04)
The counts of those releases; today's runs: section 8 and test-results/ (t/dash.t passes, every other line being listed in t/dash-known.txt). hyg.pl and mem.pl below were probes of that time.
In 0.03 (2026-11-03), t/dash.t gives: largebattery.txt 111 of 114 agree with dash, battery.txt 42/43, fdbattery.txt 14/14, sigenv.txt 12/12, t1.txt 39/45, t2.txt 42/44 (one line's order varies, as in dash), t3.txt 16/17, pspawn-d2.txt 9/10, pspawn-l10.txt 6/9, large-subset-b13.txt 37/38; every other line is a known difference (t/dash-known.txt). Post-0.03 adds large-subset-p6.txt: 19 of 19 (21 of 21 in post2-0.03); the other batteries, run again on post-0.03, give the same counts as 0.03. In 0.04: largebattery.txt 108 of 114 (three lines more now differ from dash on purpose: getopts leaves OPTARG unset, rule E), large-subset-p6.txt 25 of 25, and the new large-subset-e.txt 2 of 10 (the other 8 are rule-E choices listed in t/dash-known.txt); the rest as in 0.03. The counts below are pre-0.01 (2026-11-02).
t/spawner.t: 20 tests (32 since 2026-11-02, 08:00: every posix_spawn setting, and the rawspawn) ofProc::Spawnerand the graph: fd tables (including colliding fd numbers), closed fds,chdir,umask, ignored signals andsigdefault, exit and signal statuses, a 1 MB write by Perl without blocking, R1 (reactions never interleave), D2, "nothing left to wait for", a status taken by other code, a handler starting the next process. All pass, with the C handler and with the fallback.- Against dash, as before the split:
largebattery.txt99/103 (106/110 after the 7 lines added for L6–L8, see the known issues),battery.txt42/43,fdbattery.txt14/14,sigenv.txt12/12,t1.txt39/45,t2.txt43/44,t3.txt16/17. The same numbers as before also withPARSER=smallandPARSER=base. hyg.pl: parent fds unchanged, no zombies.mem.plon four commands andsigprobe/mem-large.pl, now t/memory.t: RSS flat over 1,500 runs. After 6,500 runs the graph holds 1 process node.sigprobe/exec-stress.pl, now t/stress.t: 9,750 children in 15 s, finished normally.large-subset-difffuzz.pl, seeds 7, 11, 23 × 200: one difference, the order of lines from parallel pipeline elements, which dash varies too.TRACE=dot: 18 pictures for a three-pipeline string; all render.
6. Decisions taken while writing it (EXPERIMENTAL, waiting for approval)
- G1.
Proc::Spawnertakes code refs only. Sub names (with data) are the graph's business, so the small module knows nothing of naming. - G2. Reap per pid, not with
waitpid(-1)(section 4). - G3. A status taken by other code:
on_exitgets undef; the graph's state islost; the shell's$?is 1. - G4. The graph is an object; ShellSpawn uses one for the whole program.
- G5. A handler is removed from its node when it is called, so its data (the shell's continuation, with its contexts) is freed even while the ended node stays in the graph for a while.
- G6.
Proc::Spawnerworks with fd numbers. The graph leaves out of the fd table an fd of the parent's that a child gets under its own number (it is inherited anyway); every other entry becomes adup2, also onto itself, which clears close-on-exec.
7. Not done
-
Whatever flips process-wide state for a while is bullet-proof only in non-threaded execution (documented limitation, the user, 2026-11-02: "Same for other stuff which needs temporary flipping around the call"). The places:
Proc::Spawner::start: Perl's umask, just around the call (posix_spawn has no umask setting);Proc::Spawner::wait_until: the SIGCHLD action, while it waits (decision D3);Proc::ShellSpawn::open_end: Perl's umask, set to the shell's while a redirection's file is opened;Proc::ShellSpawn::run: SIGINT, SIGQUIT and SIGPIPE ignored while it runs; traps set in%SIGwhile it runs (trap, inProc::ShellSpawn::Large), put back after it.
-
posix_spawn also cannot make a child ignore a signal (it inherits what Perl ignores), nor set resource limits or the nice value. Since post-0.08 these are set around the call too (decision M1, below).
-
Windows: section 8.
-
Known issue X15: the shell layer still breaks some cycles by hand (contexts in their scopes). The two new modules have none.
8. Windows (decided 2026-11-02, written since post-0.04; in the distribution since post-0.04: no CPAN release had it, it was only in the Project's later copy of 0.01; checked against CPAN's 0.01 to 0.04 on 2026-11-05)
The user's rule: what Windows can do fully and natively is implemented; everything else is reported as not implemented, so the caller's unsupported hash (or %Proc::Spawner::UNSUPPORTED) decides between drop and fail, and a caller may use a workaround that is safe only in single-threaded execution (as the umask is set around the call on Linux). No "approximate" category.
- Implemented (per-call parameters of
CreateProcessW, or preparation that touches nothing shared, or nothing to do):- the child's fds 0–2 (
STARTUPINFOwithSTARTF_USESTDHANDLES, as Perl's owndo_spawnvp_handles()inwin32/win32.c); - "only these handles", and so
close,closefrom(PROC_THREAD_ATTRIBUTE_HANDLE_LIST); chdir(lpCurrentDirectory);fchdir(the directory handle's path);- a file "opened in the child" (opened by the parent, its relative path taken from the child's directory);
search(our own lookup through PATH and PATHEXT);pidfd(the process handle thatCreateProcessreturns);pgroup => 0(CREATE_NEW_PROCESS_GROUP);sigdefault,resetids(nothing is inherited, there is no set-user-id).- fds 3 and up through the C runtime's
lpReserved2: per call and thread-safe, but only children built on that runtime read it; decided earlier as W1–W3 (Go), with programs known to ignore it (java,javaw) refused such fds.
- the child's fds 0–2 (
- Not implemented:
umask, a non-emptysigmask, ignoring signals in the child (butPIPE, andINTin a new group: nothing to do, post-0.08),pgroup => PGID(joining a group),setsid,tcsetpgrp,sched,cgroup. - Already defined by Perl on Windows, so used as they are: the command line built from the argument list (
create_command_line(), the quoting of multi-argsystem), the status (((U8) exitcode) << 8, fromwin32_waitpid()), and waiting for children (win32_msgwait()over the process handles). - What spawn does when the primitive lacks a setting (post-0.06; the user, 2026-11-05).
%Proc::Spawner::MISSINGhas as its keys the settings the primitive does not do itself (a hash since post-0.12, item JC3 of the review of the executor's code; until then a bitmap,$Proc::Spawner::MISSING): what this build's libc lacks (Proc::Spawner::__native()lists what it has), plusPROC_SPAWNER_MISSING(names, for tests).Proc::Spawner::spawn(which wraps the XSUB__posix_spawn) does each missing one in this process, around the call, and undoes it afterwards; this is bullet-proof only with one thread, as the umask always was. A setting that cannot be done so is unsupported: dropped or failed, asUNSUPPORTEDsays.
| Setting | posix_spawn (glibc 2.39) | Windows, natively (CreateProcessW) | When missing: done around the call |
|---|---|---|---|
| dup2, close, open, closefrom (the child's fds) | yes (closefrom: 2.34) | fds 0-2 (STARTUPINFO), the handle list (close, closefrom), a file opened by the parent; fds 3 and up through the C runtime's lpReserved2 | this process arranges its own fds as the child should have them (close: close-on-exec), saving and putting back what it changes; one of the four missing means all four, and chdir, fchdir too (the actions keep their order) |
| chdir, fchdir | 2.29, 2.29 | lpCurrentDirectory (fchdir: the directory handle's path) | this process changes directory and comes back (through an fd of .) |
| search | posix_spawnp | our own lookup through PATH and PATHEXT | PATH looked up here |
| pidfd | pidfd_spawn (2.39) | the process handle | pidfd_open right after the call (Linux 5.3); none: the pid alone |
| sigdefault | yes | nothing to do (nothing is inherited) | an ignored signal set back to its default around the call |
| sigmask | yes | no signals | this process's mask set around the call |
| pgroup => PGID (joining a group) | yes | no | this process joins the group around the call |
| pgroup => 0 (a new group) | yes | CREATE_NEW_PROCESS_GROUP | not possible (the child's pid is known only after): unsupported |
| resetids | yes | nothing to do (no set-user-id) | the effective ids set to the real ones around the call |
| setsid | 2.26 | no | not possible (no way back): unsupported |
| tcsetpgrp | 2.35 | no | not possible (the child's group is known only after): unsupported |
| sched, cgroup | yes, 2.35 | no | not undone reliably: unsupported |
| umask | (none) | (none) | always set around the call |
| sigignore, rlimit, nice (post-0.08, M1) | (none) | no | always set around the call, in C by the XSUB (below); unsupported where it cannot be put back |
| posix_spawn itself (post-0.08, M2) | yes | (win_spawn.h) | fork and exec; without fork either, Perl's own system { FILE } 1, ARGV; everything else around the call |
What posix_spawn has no option for (post-0.08; the user, 2026-11-06, decision M1: "do the around-the-call settings where they can be undone, and (temporarily) fail for the rest"). start(..., sigignore => [SIGNALS], rlimit => { NAME => SOFT | [SOFT, HARD] }, nice => INCREMENT); in spawn's attributes sigignore => [SIGNUMS], rlimit => [[RESOURCE, SOFT, HARD], ...] (undef: as ours; 'unlimited'), nice => INCREMENT. The XSUB sets them in this process right around the call, in C (nothing is allocated and no Perl code runs in between, Spawner.xs, around_parse), with every signal blocked meanwhile, and puts them back right after; the child gets the signal mask from before. Unsupported (dropped or failed, as UNSUPPORTED says), because it could not be put back or would harm this process meanwhile:
- a hard limit lower than ours without
CAP_SYS_RESOURCE(raising it back needs it; as root in a container without that capability too: checked here, where root lacks it); - a higher nice value unless
CAP_SYS_NICEorRLIMIT_NICEallows lowering it back (checked as an unprivileged user:nice => 5ENOTSUP,dropdrops it); - a CPU limit this process has used up already (it would get
SIGXCPU); - ignoring
SIGCHLD(our other children ending meanwhile would be reaped by the kernel, their statuses lost); - on Windows,
rlimitof resources with no counterpart there (below), andsigignorebut for what a Windows child does anyway:PIPE(there is no SIGPIPE: a write into a closed pipe fails, as with SIGPIPE ignored) andINTwhen the child is in a new process group (new_group, since post-0.12 by the child's SIGINT, W7: sosigignoreofINTitself gives the child a new group; Windows turns Ctrl-C off in a new group, and Perl'skill INT,GenerateConsoleCtrlEvent(CTRL_C_EVENT, pid), does not reach it). Other signals reach a Windows child only as Perl'skill:TERMandBREAKas Ctrl-Break (the child's default ends it), anything else asTerminateProcess(win32/win32.c,terminate_process): none can be ignored there.
Failing as the child would: a lower nice value without privilege (EACCES), a limit above our hard one without privilege (EPERM), an unknown resource (EINVAL); a signal both in sigignore and sigdefault, SIGKILL, SIGSTOP: EINVAL. A memory limit (AS) lower than what this process uses makes glibc's posix_spawn fail (ENOMEM: it maps the child's stack in our address space), where a child setting it before exec would not (checked: AS 10 MB fails, ours is put back, later calls work; with fork and exec, M2, the same call succeeds). On Linux nice and the signal mask are per thread, so only the calling thread sees them; the signal actions and the limits are the process's (bullet-proof only in non-threaded execution, as the umask). A signal arriving meanwhile stays pending (Linux keeps a blocked signal pending although it is ignored) and is delivered after.
nice and rlimit on Windows (post-0.09; the user, 2026-11-08, decision W4: "(c) and (b); (c) beats (b) if present"). Windows' own settings, exact, per child: win_priority (a priority class: idle, below_normal, normal, above_normal, high, realtime), set with SetPriorityClass on the child while it is still suspended (Wine ignores the class given as a creation flag, Windows honours both), and win_job (limits of a job of the child's own: ProcessMemoryLimit, JobMemoryLimit in bytes of committed memory, PerProcessUserTimeLimit, PerJobUserTimeLimit in 100-ns ticks of user time, ActiveProcessLimit; the job is nested in the kill_with_parent one when that is on, which needs Windows 8, and closed with the child's handle). nice and rlimit are mapped to them, approximately: nice from our class's nice value (idle 16, below_normal 8, normal 0, above_normal -8, high -16, realtime -20) plus the increment, to idle (12 and up), below_normal (4 to 11), normal (-4 to 3), above_normal (-5 to -12) or high (-13 and below), never realtime (the values and ranges are Cygwin's, winprio_to_nice and nice_to_winprio in its winsup/cygwin/miscfuncs.cc, since post-0.12; before, a table of our own: idle 19, below_normal 5, above_normal -5, high -10, and only 0 as normal); rlimit AS and DATA to ProcessMemoryLimit (committed memory, not address space; the smaller wins), CPU to PerProcessUserTimeLimit, NPROC to ActiveProcessLimit (the processes in the child's job, not the user's), each from the soft value if set, else the hard one (Windows has one limit and ends the child at it, where Unix sends SIGXCPU at the soft CPU limit); NOFILE, CORE, FSIZE, STACK and the rest have no counterpart: unsupported. win_priority and win_job beat the mapping where both are given. Elsewhere win_job is unsupported, and win_priority (post-0.12, the user, 2026-11-15) is mapped the other way: the child gets the class's nice value above, as the increment from ours (win_priority => 'idle' from nice 0 is nice => 16), with what nice can and cannot do (lowering the nice value needs CAP_SYS_NICE or RLIMIT_NICE; a higher one is unsupported where it could not be put back, as for nice); nice, if given, beats it there; an unknown class fails with EINVAL. Checked under Wine with a child that prints its priority class: win_priority, nice 5, 15, -12, and nice with win_priority give the classes above; a resource with no counterpart fails with ENOTSUP or is dropped; an unknown class fails with EINVAL. Wine stores no job limits (a plain C program setting them reads back none), so their effect is not checked here, only that the child starts with them; the mapping is tested on Linux (t/spawner.t).
No posix_spawn at all (post-0.08; the user, 2026-11-06, decision M2: as Run 0.06's forkless mode, branch project-2026-11-03, claude/Run-0.06/). Makefile.PL now probes posix_spawn itself (POSIX_SPAWN); without it, or with PROC_SPAWNER_MISSING=posix_spawn (tests), the primitive is the XSUB __spawn_nowait: fork and exec, in C, with nothing in between but the signal mask put back (as spawnvp(P_NOWAIT) on a system with a plain spawn; exec's errno comes back through a close-on-exec pipe). Every other setting is then missing and done around the call as above (setsid, pgroup => 0, tcsetpgrp, sched, cgroup unsupported); rlimit, sigignore, nice as M1. argv reaches the child exactly (execve). Checked: the whole suite passes so (58 tests), and a build without posix_spawn (PROC_SPAWNER_WITHOUT=POSIX_SPAWN) passes t/spawner.t and t/dash.t. Where there is no fork either (Windows when posix_spawn is masked, OS/2, VMS; on Linux with PROC_SPAWNER_MISSING="posix_spawn fork": $Proc::Spawner::PRIMITIVE is system1), the redirections are delegated to Run (the user, 2026-11-07: "one of the major points of the exercise"; "it must delegate the whole code of redirection to Run"): Run 0.07, in repository ilyaz-ai/run (0.02 to 0.06 as released, then 0.07; on CPAN since 2026-11-10). Run::start(1, ARGV, REDIRS, ERRS, FILE) applies the redirections in this process around Perl's own spawn, system { FILE } 1, ARGV (P_NOWAIT; on Unix Run emulates it by fork and exec), and puts our fds back; the fd actions become Run's [FD => {fd => M}], {close => 1}, and a file opened here as {filehandle => FH, pin => 1}; chdir and fchdir are done here, in order, so an open's relative path is taken from the directory of that moment. What Perl's spawn does on Windows (win32/win32.c of 5.38.2, do_spawnvp_handles, create_command_line):
- argv: Perl builds the command line itself: an argument with blanks is put in quotes, but quotes and backslashes in it are not escaped, so a child sees other arguments (the table above). ❰BUG❱ in Perl (
doc/perl-win32-system-bugs.md, to be reported); the arguments are passed as they are, and the tests of exact argv and batch files are TODO on this branch until Perl is fixed (the user: no bug-for-bug workaround). - the program: looked up here (PATH and PATHEXT as
win_spawn.h, withsearch; otherwise as given, relative to the directory then): posix_spawn's meaning of FILE, not Perl's (which searches PATH always). - fds: the child gets the standard handles of the moment (
GetStdHandle), which the C runtime'sdup2andcloseset for fds 0-2; the child sees no fds 3 and up:dup2oropenonto them is unsupported,closeandclosefromof them need nothing. But Perl's spawn passes the child every inheritable handle (bInheritHandles, no list), and Perl makes its handles inheritable (no close-on-exec by$^Fon Windows), so a child could hold, unseen, the write end of its own input pipe and never see end of file (X38). Since post-0.10 our fds from 3 up are made not inheritable for the call and put back after, as POSIX has it (the user, 2026-11-09: "do as on POSIX"). Not thread-safe: a spawn in another thread meanwhile does not pass them either. (Windows' own thread-safe way is the list of handles a child gets,PROC_THREAD_ATTRIBUTE_HANDLE_LIST, whichwin_spawn.huses; Perl's spawn does not: ❰BUG❱ in Perl,perl-win32-system-bugs.md, 4; its section 5 lists every workaround of ours.) - the rest: a new process group always (Perl gives one to every child it does not wait for, as
system 1, ...; a child thatsystemwaits for stays in Perl's group;pgroup => 0needs nothing, a group to join is unsupported);sigdefault,resetids, an emptysigmaskneed nothing (nothing is inherited); a non-emptysigmask,pidfd,fchdirare unsupported; the environment is%ENV, set around the call; at most 64 children at once (Perl's table;EAGAINafter); the child is Perl's, so Perl'swaitpidtakes its status. - Checked again (2026-11-17, post-0.12), and recorded: only what did not pass, so a record of just its header means every one passed (
t/flavors.plandt/missing-matrix.plwith-o, at each release): on Linux the four levels (test-results/linux-flavors.txt) and the matrix's 23 entries (test-results/linux-missing-matrix.txt); under Wine the three levels (test-results/wine-flavors.txt) and the matrix's 14 entries (test-results/wine-missing-matrix.txt): all pass. The counts per test: below. - Per test (2026-11-18, the user: a failing TODO test, and a test skipped because the flavor cannot do a setting, count apart, where prove counts them as passed):
t/flavors-outcomes.plrecords each test's outcome in one flavor,t/flavors-compare.plputs the flavors side by side (test-results/linux-flavors-by-test.txt,test-results/wine-flavors-by-test.txt; the test files only). No test failed in any flavor. On Linux no TODO test applies; ok and skipped: as built 206 and 4, the redirections only 194 and 16, fork 204 and 6, system1 194 and 16. fork skipspgroup => 0andsetsid(refused: the child does nothing between fork and exec, F1 inTODO); system1 skips those andsigignore,rlimit,nice(also refused: their wrappers in C, M1, are aroundposix_spawnand fork only), as does the level with the redirections only, which makes every other setting missing. Under Wine: as built and the redirections only alike, 161 ok and 20 skipped; system1 156 ok, 24 skipped and 5 TODO failing, all by Perl's own spawn: exact argv and a batch file (t/windows.t), three lines oft/portable.twith"in an argument ofsh(perl-win32-system-bugs.md, 1); its skips:kill_with_parent(no job) andnew_group(always a new group); and its other tests: fd 3 unsupported, at most 60 children at once. - Under Wine the tests must run with Windows' own standard handles (
cmd /c "perl t\windows.t > out.txt"): Wine's handles for a Linux stdout cannot be fdopened (EINVAL), which Run's saving of fd 1 needs. The earlier counts (2026-11-14: 179, 179 and 181 under Wine, 156 per level on Linux; post-0.08: the suite through Run, andt/windows.tthrough both primitives) are inChanges. OS/2 and VMS are not tested.
The Windows primitive (post-0.06, EXPERIMENTAL; built into the module since 0.07, through Makefile.PL since post-0.09). win_spawn.h is posix_spawn on CreateProcessW, in plain C; Spawner.xs includes it under _WIN32 and #defines the posix_spawn names to it, so __posix_spawn stays one source. It does the Windows column of the table above: the child's fds 0-2 through STARTUPINFO, fds 3 and up through the C runtime's lpReserved2 table, exactly the handles the child should have through PROC_THREAD_ATTRIBUTE_HANDLE_LIST (inheritable copies made for the call, so the parent's handles keep their flags); the file actions in order on a table of the child's fds (an open by the parent, relative to the child's directory); argv quoted as the C runtime parses it back; posix_spawnp through PATH and PATHEXT (a relative program name is taken from the child's directory, after chdir, as glibc does); pgroup => 0 as CREATE_NEW_PROCESS_GROUP; ENOTSUP for the rest. The process handle is kept, so the exit code survives until the child is waited for (__win_spawn_handle, __win_spawn_forget). Checked with MinGW-w64 under Wine; compiled for x86, x64 and ARM64.
Compared with Perl's own spawn on Windows (do_spawnvp_handles(), create_command_line(), qualified_path(), win32_waitpid(), win32_kill() in win32/win32.c of perl 5.38.2; 2026-11-05). Fixed in win_spawn.h and Spawner.xs after the comparison:
- the program without an extension: PATHEXT's extensions first, the bare name last (Perl tries
.exefirst; a bareprog, say an MSYS script besideprog.exe, cannot be run); a quoted PATH entry ("C:\a;b") is one entry, as Perl reads it; cmd.exe /c(or/x/c) with two or more words after it: the rest gets one more pair of quotes, which cmd strips by its own rules, as Perl does;- the environment block sorted by name, ignoring case, with the parent's hidden
=C:entries (each drive's directory) kept; - a command line over 32766 characters:
E2BIG; - strings: Perl's own
CreateProcessAtakes bytes in the ANSI code page; so a string Perl does not hold as UTF-8 is read as ANSI, one it holds as UTF-8 as UTF-8; - the child's default environment and directory are Perl's (
PerlEnv_get_childenv,PerlEnv_get_childdir), which with pseudo-forks can differ from the process's; - memory: compiled against Strawberry Perl 5.38.2 (
PERL_IMPLICIT_SYS),mallocandfreeare Perl's macros for the interpreter's allocator;win_spawn.hnow allocates on the process heap itself (it mixed allocators before).Spawner.dllthen compiles and links against Strawberry Perl's headers andlibperl538.awith MinGW-w64 (it loads: below).
Run under Wine with Strawberry Perl 5.38.2 (the user's approval, 2026-11-05): Spawner.dll built with MinGW-w64 against its headers and libperl538.a loads; Proc::Spawner::spawn starts Windows' perl.exe, an open action gives it a file as stdout, Perl's own waitpid reaps it (status 3), and %Proc::Spawner::WINDOWS works. This found two faults, fixed: spawn gave the count of its results, not the pid, in scalar context (its wrapper; on Linux only when a setting is missing); Proc::Spawner used fcntl and SIGCHLD at load time. Then (until post-0.10) the loop polled for ended children every $TICK, and pipes could not be read or written in it (Perl's select there works on sockets only); since W5 it is an I/O completion port (below).
How the child sees its arguments (a Windows perl.exe printing @ARGV; given: a b, q"x, c\\, the empty string, x "y" z, tail\, d\"e, %PATH%, a&b):
| Started by | The child's @ARGV |
|---|---|
| Perl's own system LIST | two arguments: [a b] [qx c\\ " x y z tail\ d"e %PATH% a&b] (quotes taken as quoting, the empty one gone; a -e program with " in it does not even compile) |
| Proc::Spawner::spawn | all nine, as given |
Choices (the user, 2026-11-05), in %Proc::Spawner::WINDOWS (win_spawn.h: WIN_OPT_*), every child:
new_group(default on until 0.12;'auto'since post-0.12, the user, 2026-11-14, decision W7: "Of course" to Claude's proposal;Proc::Spawner::win_new_group):CREATE_NEW_PROCESS_GROUP, where Windows turns Ctrl-C off.'auto': only for a child that would have SIGINT ignored on POSIX (sigignorehasINT, or$SIG{INT}is'IGNORE'andsigdefaultdoes not haveINT:Proc::ShellSpawn's background jobs, not its foreground commands); such a child goes on after a Ctrl-C at the console, as a POSIX asynchronous list ignores SIGINT; every other child stays in Perl's group and gets the Ctrl-C itself, as a POSIX child in the foreground process group gets SIGINT. Perl's own spawn does the same (5.38.2'swin32/win32.c: a new group only forP_NOWAIT,system 1, ...).1: every child (the old default: "the parent is the one to be killed, and it is to kill its children", the user, 2026-11-05);0: none. Checked under Wine (t/win_spawn.t): the flag Windows records for a child in a new group (ConsoleFlagsin its process parameters), for each case; not with a real Ctrl-C.kill_with_parent(default on until 0.12; off since post-0.12, the user, 2026-11-14, decision W6: as Perl's own spawn, which makes no job object;win_spawn.h'sWIN_OPT_DEFAULTno longer has it either, so Perl's copy inherits none of it): on, the child is in a job object (JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE) whose only handle is the parent's, so Windows ends the children when the parent ends, however it ends. The child is started suspended, put in the job, then resumed, so nothing it starts escapes. Checked under Wine (a child that would write a file after 3 seconds: killed when the parent exits; with the option off, it writes it). Before Windows 8, a parent that is in a job already cannot put its children in another one: they then run outside it. Off, a child still running when Perl ends keeps running, as with Perl's own spawn and on POSIX; what the defaults cost on Windows (files kept open cannot be deleted, the console, Ctrl-C only on this console, Perl'skill INT/TERM/BREAKand a child in Perl's group, the primitivesystem1; and Ctrl-Break, which reaches every process on the console whatever the settings) is listed in the POD of%Proc::Spawner::WINDOWS, for users. On, the other way round: nothing started here outlives the parent, and the children end byTerminateProcess, with no chance to clean up. Checked under Wine:t/windows.t(a child that writes a file after 3 seconds, its parent exiting at once: the file is written by default, and not with the option on),t/win_spawn.t(the defaults make no job).new_console(default on, as Perl): a child with none of fds 0-2 gets a console of its own (CREATE_NEW_CONSOLE); off: it shares ours with no standard handles.
Batch files (.bat, .cmd) run (the user: most arguments can be passed safely, and the others can be recognized): through the system's cmd.exe /e:ON /v:OFF /d /c ""SCRIPT" ARGS" (not a cmd.exe found on PATH), each argument quoted for cmd as Rust's standard library does since BatBadBut (CVE-2024-24576): quoted unless it has only letters, digits and #$*+-./:?@\_; " doubled; % written %%cd:~,%, which cmd expands to a single %, so no %NAME% is expanded; a trailing \ quotes it too. Refused (EINVAL): an argument with CR or LF (cmd ends the line there), a script path with " or ending in \. The script sees each argument as cmd shows it (%1 with the quotes, %~1 without). Under Wine, & | < > and quotes arrive literally and nothing else runs; % does not (Wine's cmd reads %% on a command line as one %, real cmd does not); not checked on real Windows.
Unicode: the child is started with CreateProcessW (UTF-16 command line, environment and paths), Perl's own with CreateProcessA (the ANSI code page). A string Perl holds as UTF-8 therefore reaches the child as it is, whatever the code page; one it holds as bytes is read in the ANSI code page, as Perl's spawn reads it.
Windows versions: the explicit handle list needs Vista; Windows 7's console handles are pseudo-handles that the list refuses, so they are left out of it (the child reaches the console through STARTUPINFO); nested job objects need Windows 8. Not tested before Windows 10 (Wine 9 imitates 10).
The kept process handle is released once the loop has the child's status (post-0.08: reap calls Proc::Spawner::reaped, which calls __win_spawn_forget; since post-0.10 start's child watch does; a caller of the raw spawn who waits itself calls reaped(PID)). Checked under Wine with Strawberry Perl 5.38.2: one child's handle is gone after its status (0.08: still open), and 1,100 children in a row all give their status. With 1,100 children started before any is waited for, all ended by then, 0.09's fixed table of 1,024 kept handles lost 76 statuses (the children whose handles did not fit were closed at once; known issue X34); since post-0.09 the table grows, and none is lost (checked under Wine both ways). rlimit, sigignore, nice failed there with ENOTSUP, or were dropped, until post-0.09; since W4 they are mapped (nice to a priority class, rlimit to win_job, sigignore of PIPE, and of INT in a new group, accepted), and what has no counterpart fails or is dropped.
Done post-0.09: Win32::SetChildShowWindow is honoured as by Perl's own spawn (STARTF_USESHOWWINDOW and wShowWindow from Perl's w32_use_showwindow, w32_showwindow; checked under Wine with a child that prints its startup info: the same through both primitives, before, after SetChildShowWindow(0) and (7), and after resetting it); more than 64 children at once (Windows' limit of 64 handles per wait does not apply: post-0.09 the loop polled each child, since W5 it waits with RegisterWaitForSingleObject; 100 at once checked, 80 in t/loop.t; Perl's own spawn, system1, still stops at 64, Perl's table); building through Makefile.PL (no POSIX probes there; checked under Wine with Strawberry Perl's own gcc and gmake: perl Makefile.PL, gmake without a warning, gmake test of t/windows.t and t/parsers.t).
Done post-0.10, W5: the loop is Proc::Spawner::Loop, with a Windows backend on an I/O completion port; how it works and what was checked: pspawn-loop.md. The shell layer on top runs there too since post-0.10: pspawn-shell-windows.md.
Done post-0.12, win_spawn.h per call, with hooks (asked for so that Perl's core can take the file as its own posix_spawn on Windows; a patch to Perl with a copy of it is being drafted in another session; nothing in what Proc::Spawner does changes but the three fixes below). What was the file's, for the whole process, is now each call's: the WIN_OPT_* options, wShowWindow and the job that ends the children with the parent are attributes (win_attr_setoptions, win_attr_setshowwindow, win_attr_setkilljob; defaults as before; Spawner.xs sets %Proc::Spawner::WINDOWS over the defaults in each call, where before it changed the file's variables, so a key deleted from the hash meant its last value, now its default). The code page of every string is an attribute too (win_attr_setcodepage: UTF-8 by default, CP_ACP for Perl's own byte strings). Where the file used the process's own state, a caller may plug in its own (win_spawn_hooks_t, win_attr_sethooks; a member left NULL: the file's way): keep (the child's process handle, and its own job if any, go to the caller's table instead of the file's: Perl's waitpid, kill and $? know only its own), var (PATH and PATHEXT), env (the child's environment when envp is NULL), cwd (the child's directory and the base of relative names), fd_handle (_get_osfhandle), errno_of (a Windows error as an errno). The allocator is chosen when the file is included (WIN_SPAWN_MALLOC, WIN_SPAWN_REALLOC, WIN_SPAWN_FREE; the process heap by default). The file's own job and child table are safe in several threads now (InterlockedCompareExchangePointer, an SRWLOCK). From the patch to Perl, taken with changes: three options of the command line that Proc::Spawner does not set, WIN_OPT_CMD_RAW (one word after cmd.exe's /c: a command line for cmd, passed in one pair of quotes), WIN_OPT_ARGV_RAW (the caller's own command line; no batch-file path) and WIN_OPT_KEEP_QUOTED (a word in quotes copied as it is, also inside the quotes after /c: the patch's version lost the opening quote there). The patch's way of handing over the process handle closed the child's own job, which ended the child at once when it had win_job limits and kill_with_parent; the hook keep gets the job too (both checked under Wine against the patch's file). Spawner.xs plugs in Perl's own environment and directory (PerlEnv_getenv, PerlEnv_get_childenv, PerlEnv_get_childdir, as Perl's own spawn takes them), in place of the chdir action it added before. Found on the way, in a pseudo-process (a child of Perl's fork emulation) under Wine, and fixed: X57 (the program was looked up with the process's PATH, and a relative name from the process's directory), X58 (a name given twice in the environment: either value, now the first), X59 (reaped was gone since W5); pspawn-known-issues.md. The child's directory and changed variables were right; a variable deleted in a pseudo-process still reaches the child under Wine, through Perl's own environment for a child, as with Perl's own spawn (a fault of Perl's where the process's environment is not sorted, as under Wine). Checked: under Wine with Strawberry Perl 5.38.2.2, t/loop.t, t/windows.t, t/parsers.t, t/programs.t, t/portable.t pass, t/windows.t through both primitives; t/win_spawn.t checks the file alone, through a C program (t/win/thooks.c, its child t/win/clchild.c): every attribute and hook, the code page, the options of the command line (39 checks; on Windows built with Perl's own C compiler, elsewhere skipped unless PROC_SPAWNER_WINE=1 with MinGW-w64 and Wine; passes both ways). Not checked: real Windows; several threads at once.
Proc::ShellSpawn on Windows, before its port (the port came later on 2026-11-09: pspawn-shell-windows.md).** What runs there: Proc::Spawner (the primitive, win_spawn.h), the loop (pspawn-loop.md), the parsers, and the programs Proc::ShellSpawn runs inside Perl (X21: echo, printf, test, [, true, false). The shell layer itself did not, before the port: found under Wine on 2026-11-09, when the snippet library was to run there (large-subset/large-subset-state.md, section 7, "W5"), every run() died at once, first at getppid (for $PPID; Windows Perl has none), then, past that in a scratch copy, in Proc::Spawner::Graph at fcntl with F_GETFL (a handle's mode), and the graph's pipes were made as FIFOs and reopened through /proc, which Windows lacks too (the port takes $PPID from NtQueryInformationProcess and does without fcntl and /proc there). Under Wine with Strawberry Perl 5.38.2.2, on the code of post-0.10, these pass: t/loop.t, t/windows.t, t/parsers.t, t/programs.t (skipped there: Ctrl-C without a console, the comparisons with dash, bash and coreutils), and the large parser over all 45,501 extracted corpus samples gives the same trees and rejections as on Linux.
Still not done: the inheritable copies of handles exist during the call, so another thread's CreateProcess with inheritance could take them (one thread assumed).
While this process's fds or directory are changed, every signal is blocked, so no handler of ours sees them; not on Windows, where Perl has no such signals to block (X64); the child's mask is set by the primitive. If sigmask is missing too, the signals cannot be blocked without the child inheriting that: then a handler running in that moment would see the changed fds. Tests: t/missing-matrix.pl runs the whole suite once per setting the primitive does natively on this platform (19 on Linux with glibc 2.39, 11 on Windows), then with all emulable settings missing, with the Windows column's gaps, without posix_spawn, and with everything missing; an entry that would be the same run as the tests as built, or as an entry before it, is skipped, and said so (14 entries under Wine, 23 on Linux).
- Testing on Linux "as on Windows": done:
PROC_SPAWNER_MISSINGandt/missing-matrix.pl's entry with the Windows column's gaps (above).