Proc-Spawner: what is open, in one place
========================================
0.12, 2026-11-13. Every postponed thing, by who has to act. The full
records behind each line: doc/pspawn-known-issues.md (X..., section "Open"),
doc/large-subset/large-subset-state.md (sections 4-6),
doc/large-subset/large-subset-environment.md (X27), doc/pspawn-spawner.md
(section 8: Windows), Changes. Keep this file current: when an item is
decided or done, remove it here and record it where it belongs.
A. WAITING FOR THE USER'S DECISION
1. X27: which samples of the behaviour run (large-subset-state.md,
section 7) compare meaningfully, in the user's four steps (2026-11-10):
(1) find the samples the environment cannot affect; (2) for the
others, describe exactly how they are affected, and run them in a few
example environments (where only one command of a snippet is
affected); (3) recognize more such samples by simple programmatic
tests; (4) leave the rest to deep AI analysis.
Step 1, done (2026-11-07): the levels from the parse trees alone
(doc/large-subset/large-subset-environment.md, sections 1-4): strict
365, held 2, almost strict 34,291 (if programs behave alike in each
run), undecided 5,752 (before step 3). Record per sample:
large-subset-provenance.tsv.
Step 2, done in part (2026-11-08; section 9): each undecided sample has
codes naming what kind of thing it depends on (files, vars, path-tmp,
...: section 3, "At a glance"), not exactly how (files does not say
which file). Example environments were run where one switch of the
whole environment flips the case (-env of large-subset-runcompare.pl):
POSIXLY_CORRECT, locale, SIGPIPE, /proc, a fresh /tmp, a fixed
/dev/urandom, empty /home, /mnt, /var, a private /dev, every logged
program failing (STANDIN_STATUS) or printing a chosen text
(STANDIN_OUTPUT), fixed file times (FIXTURE_MTIME); each on every
sample with its code, not only those with one affected command. The
samples that changed: tools/large-subset-env-variants.tsv. Decided:
such runs are footnotes to the level totals (EV1); the published table
is the run with a fresh /tmp, a fixed /dev/urandom and fixed file
times (A1(b)).
Step 2 for the codes inside the run, done (2026-11-10, 11; branch
x27-step2; large-subset-environment.md, section 10):
large-subset-blocks.pl -d names each undecided sample's dependences
exactly; the 2,901 samples that depend only on files, variables,
standard input, arguments, cd or HOME by exact names (the partially
decided) ran in example setups: 3 each, every combination (9) for two
dependences, 3 + 3m for more (the user's "3^m tries ... 3+3*m"):
1,547 kept their class, 1,350 changed it with ours still on the side
of a shell, 4 moved to where ours agrees with neither (explained).
Nine bugs of ours found on the way (X39-X47), all fixed. Record:
tools/large-subset-env-setups.tsv.
The match across environments (the user, 2026-11-10 and 11; section
11): two trees (the user's layout): every snippet by whether dash and
bash agree and with which ours sides; columns by builtin-vs-coreutils,
checked command by command (every command ours ran, in every
environment, run alone by coreutils, dash's builtin, bash's builtin
and ours: ours = coreutils in all 155); the second tree splits each
box by legalization. Messages count nowhere, wherever a redirection
sends them (the user: compensated before the analysis; the former
comparison: large-subset-match-raw.md). Run again post-0.11
(2026-11-13: the code with X39 to X56 found, the runner waiting for
background jobs, step 3's environments counted too): of the 1,793
snippets where ours is not as both shells: legalized by rulings 1,620,
awaiting A12 88, chance 40, not legalized 45 (items 13 and 17; X56,
and cd -). large-subset-matchreport.pl prints it all;
tools/large-subset-match.tsv per snippet.
Step 3, done (2026-11-12; section 12): large-subset-blocks.pl follows
every value from the environment to where it lands; of the 5,741 left
undecided by the codes: pass 664 and split 1,476 (only passing through
the shell, some split and globbed into a program's words) are almost
strict now (the user's (a): assumptions passthrough and splitglob,
probed by make probe-values), so almost strict 36,442, undecided
3,601; control 2,525 (only through what the run can set: a switch whose
other state a ruling covers, a program's answer, /tmp); affects-shell
1,076 (a value from the environment affects what the shell itself
does).
Checked against step 2's runs: no counterexample left unexplained.
Step 2 for the control class, done (2026-11-12; section 12; the user:
with any program a stand-in): 2,441 samples in 8,862 setups, each run
in a root of its own (-env cowroot), the programs' answer one more
dependence; where the environment changed the class, only into a
difference a ruling legalizes (the one not legalized, cd with extra
words, is the same in every environment). Partially decided now
2,838 of the 3,601 undecided. Record: large-subset-env-control.tsv.
Step 2 for the class affects-shell, done (2026-11-12; section 12): 948
samples, with the same $0 for the three and a PID namespace per run;
of the 410 where ours is not as both shells: legalized 301, awaiting
A12 67, chance 39 (33 races; 6 persistent results, item 17), not legalized 3
([^...], item 13). X53 found and fixed. Names with an expansion
(the user: choose the value and make the file of that name together;
2026-11-12): 119 of 149 samples got setups (control 34, affects-shell
85); not legalized among them 31 (cd with extra words 30, a two-digit
descriptor 1: item 13); X54 found and fixed. The 33 races, run 20 times
each (make corpus-races): 14 were a background job that ours finishes
before it ends and the run killed in the shells; decided (A18, the
user, 2026-11-12): the runner waits for them too, up to 5 s (-after).
Then 27 the same in every run, 6 races (chance); X55 found and fixed. Partially decided now
3,508 of the 3,601 undecided; the other 93, which the setups left out,
are classified by Claude reading them (item 19).
3. P2 and P4 (large-subset-state.md, sections 4-5): ${...} forms both
shells refuse; messages with our name and line, and $-.
5. X2: $! of a background job that starts no process (design question K4).
6. X7: run() ignores SIGINT and SIGQUIT while it runs, as Perl's system
does: say if not wanted.
7. Perl's Windows bugs drafted in doc/perl-win32-system-bugs.md (argv
quoting, the cmd.exe retry, batch files, inherited handles): file them, or ask.
8. Delete the branch run of this repository on GitHub (the proxy refuses
deletions here); its contents are in ilyaz-ai/run, whose 0.07 is on
CPAN (the user, 2026-11-10).
9. X22 (bash's own bug): report it to bug-bash, if wanted.
10. M2: where there is neither posix_spawn nor fork, a missing Run makes
spawn die (at its first call, not at load). Act: keep it, or fail
with $! as an unsupported setting does.
11. Perl's own writes into the graph's own pipes reopen /proc/self/fd/N for
every write. Keeping that handle for the pipe's life (closed with the
end) is safe for the graph's pipes only (not the caller's fds, which
the module never closes); measured: at most 2.5% even for a loop of
builtins writing into a pipeline. Act: do it, or leave it.
12. X27, section 11: ~ with HOME empty. dash 0.5.12 drops the word (and
~/x loses its prefix): ❰BUG❱ dash, as POSIX treats the result of ~ as
quoted; bash and ours give an empty field. The rule table marks ours'
way "proposed" (62 samples). Act: confirm (then
also report it to dash, if wanted), or decide otherwise.
13. X27, section 11: 7 snippets where bash = ours != dash and no ruling
legalizes it (and 2 with cd -, in B): [^...] in a pattern (2) and
return outside a function (1), as rule E but not yet listed under it;
echo foo 42>/dev/null (dash: one-digit fd numbers only; also, in
step 3, head -<$LINENUM + 10> filename, NB:6724); echo $LINENO
after a #! line (dash: empty); cd with extra words (dash ignores them;
in step 3 also 30 samples where a program's answer gives cd its
operands, cd $(find . -name config): bash and ours, since X54, "too
many arguments", status 1); [^...] also in step 3 (mv
/mnt/usbdisk/[^l]* ...);
races between the elements of a pipeline: ls -a | tee output.file,
and since the runs of post-0.11 cat lol | sed ... > lol (sed's
redirection empties lol before or after cat reads it) and NB:12523,
find $DiskName ... | sort ... >> $TMPFILE given "two words *"
(find's pattern sees the file sort's redirection makes, or not); each
came out otherwise in another run; and, found by the fuzzer, not a
snippet (the user, 2026-11-12: join it to this list): return || shift
at the top level, where ours follows bash for return (X48) and dash
for the failed shift (A5) and so ends as neither shell (status 2; both
give 0). Act: decide them one by one, with the rules (the user,
2026-11-11: later).
15. MetaCPAN shows the .md files rendered (/view/doc/...; relative links
between them work, checked by the author 2026-11-11) but not their
tables; the POD's links to them (Proc::Spawner::Docs, to /source/:
the raw text) did not work. The experiments for 0.12 (the user,
2026-11-11 and 12), written post-0.11: Proc::Spawner::Docs links
each document twice, its name to the rendered file of the newest
release (/dist/Proc-Spawner/view/doc/...) and, in an =for html
paragraph, the same file of the same release (../../../doc/... from
/release/.../view/lib/Proc/Spawner/Docs.pod; it works from
/release/.../view/ and /dist/.../view/, not from /pod/...); the test
page doc/metacpan-tables.md has one table as a pipe table, raw HTML,
preformatted text indented and fenced, and a list. Act, after 0.12
is on MetaCPAN: look at Docs and the test page, check both kinds of
link, choose the table form.
16. The documents: large-subset-state.md is a log grown by appending
(paragraphs of 2,000 characters, its history inside), hardly usable
(the user, 2026-11-11). Rewrite it as the current state, its
history to Changes; links into the private repository (item 14) with
it. Postponed (the user: not critical).
17. X27, step 3: persistent results (the same in every run, which the
environment does not change), found in the class affects-shell,
that no ruling covers yet (section 12): the
environment env and printenv list (5 snippets, since IN1 with v=5 env |
less: bash adds SHLVL and _; dash and ours list the same variables,
in another order); /proc/$$/cmdline (1: each shell's own command line,
ours a Perl one); times (1: ours, Perl, has used CPU time where dash
shows 0); and, found post-0.11 (20 runs: ours' own result in each, so
not a race), (echo ...; sleep 1; echo ...) | zenity with zenity
missing: the shells' builtin echo dies of SIGPIPE and ends the
subshell, ours' echo (inside Perl, as the program before IP1) fails
alone and the subshell goes on (one sleep 1 more). Act: rule on them
(they are built into the shells and into ours; the environment does
not change them).
19. X27, step 3: the 93 undecided samples the setups left out, classified
by Claude (an AI) reading them (2026-11-13; the user's categories: AI- and a category used
before, and AI-PATH-twister; large-subset-environment.md, section 12;
record doc/large-subset/tools/large-subset-ai-classes.tsv): AI-strict
27 (an unmovable stone 18, a device working as expected 9), AI-names
with an expansion 26, AI-values passed on unchanged 7, AI-values passed
on split 7, AI-tilde 6, AI-A5 5, AI-glob 4, AI-shells' own variables 4,
AI-PATH-twister 4, AI-noise 3. Act: accept each category, or move a
sample (the work each takes: B).
20. win_spawn.h for Perl's core (post-0.12; another session, prhx, is
drafting a patch that takes a copy of it into Perl as its posix_spawn
on Windows; doc/pspawn-spawner.md, section 8, "Done post-0.12").
Taken from that patch, with changes: the options WIN_OPT_CMD_RAW,
WIN_OPT_ARGV_RAW, WIN_OPT_KEEP_QUOTED, which Proc::Spawner does not
set. Act: keep them, or leave them to Perl's copy.
Also, a batch file: cmd.exe takes the first word of its command line
both as what it runs and as %0, so posix_spawn's two names (FILE, what
runs; argv[0], what the program is called) cannot both reach a batch
file. With system { 'rm' } 'swissarmy', 'foo' and rm.cmd found as
C:\bin\rm.cmd, that first word can be: (1) the path found,
"C:\bin\rm.cmd" (now; %0 shows the path); (2) FILE as written, rm
(cmd.exe looks it up again, in the current directory first, then PATH:
another file may run than the one found and checked); (3) argv[0],
swissarmy (cmd.exe runs whatever swissarmy names, not FILE at all).
Perl's own spawn: (3), as far as Claude knows (its command line starts
with argv[0], and Windows hands a batch file's command line to cmd.exe
as it is; not checked on real Windows, and under Wine Perl's batch
files fail); its t/win32/system.t, which expects %0 as written, has
FILE and argv[0] the same, where (2) and (3) agree. Act: (1), (2), or
(2) as an option (Perl's copy, for its old %0). Claude's view
(2026-11-14): (1), (2) only as an option, never (3).
21. X61's rest: `read` with backslashes in the line, where dash and bash
differ from each other (found 2026-11-13 with X61's random cases):
for the user to decide case by case (pspawn-known-issues.md, X61).
B. WORK (no decision needed)
- FIRST IN THE NEXT CYCLE (the user, 2026-11-18: after the release, which
is made in the configuration as tested; all found reading the code, none
of it run):
1. X67 (ours): Proc::Spawner::spawn_unsupported (Spawner.pm, the line
"if (grep $m{$_}, qw(dup2 close open closefrom))"): where the libc
lacks one fd action, every fd action of every spawn is done in Perl's
own process around the call, even when the call asks for none of the
missing kind. Hit: a libc without addclosefrom (glibc before 2.34,
musl), and t/flavors.pl's level 2, which so tests posix_spawn with
every fd action done in the parent, not "the redirections only".
Fix: decide per call (only an action of a missing kind asked for).
2. A test of closefrom: no test file calls it, and nothing in the
distribution uses it (only Proc::Spawner's interface).
3. Where closefrom alone is missing, a wrapper around posix_spawn: the
closefrom replaced by a close for each fd from N up open in the child
at that point (Perl's inheritable fds, and the earlier dup2 and open
targets, minus the earlier closes), as system1 does already
(run_redirs); the fds from /proc/self/fd where present, not one fcntl
per fd up to the open-files limit (which may be 1,048,576). Low
value until something uses closefrom.
4. F1, the user's decision: the fork primitive (__spawn_nowait, M2) does
nothing between fork and exec but the signal mask, as a plain spawn
would, so setsid, pgroup => 0, tcsetpgrp, sched and cgroup are
refused, and rlimit, sigignore, nice are flipped in the parent around
the call (not thread-safe), though the child could do them all
itself. (a) keep it; (b) every setting in the child, between fork
and exec, errors through the exec pipe; (c) (b), and a test-only
level keeping today's plain spawn (Claude's view). Only the fork
level reaches __spawn_nowait; its other paths (emulate_actions, the
settings around the call) are reached by level 2 and the matrix too.
5. system1 on Linux refuses rlimit, sigignore and nice, which fork does
(found 2026-11-18 by t/flavors-outcomes.pl: 6 tests skipped, and 2
of win_priority): their wrappers in C (M1, around_begin and
around_end in Spawner.xs) are called around posix_spawn and fork,
not around Run's start. Calling them there too would make system1
as capable (not thread-safe, as for the other two).
- W6 and W7 (post-0.12; doc/pspawn-spawner.md, section 8): try them with a
real Ctrl-C and Ctrl-Break (under Wine with a pseudo-terminal, as the
tricks topic on Wine says, or on Windows): the foreground child gets the
Ctrl-C, a background job does not; and what Perl's kill TERM does to a
child in Perl's group (it leads no process group).
- win_spawn.h and Perl (post-0.12; A20): once the patch to Perl exists,
use Perl's posix_spawn where Perl has it (by its %Config entry) and keep
win_spawn.h for older Perls; section 5 of doc/perl-win32-system-bugs.md
then becomes per Perl version. Where Perl has system(1, LIST) without
fork, M2's fallback (system1, through Run) can use it. Until then,
after any change to win_spawn.h tell the user, so that Perl's copy
follows (the interface and names there are not settled).
- The parser's work list (large-subset-state.md, section 4, 44 samples):
the real gaps alias :=, export ${foo?}, command -, unset -f ls /bin/ls.
- cd - with OLDPWD unset (dash prints the directory; ours prints nothing);
exec foo's message lacks "exec: " (X5's kind).
- X30: $LINENO makes parsing quadratic (count the lines once).
- X1's rest (K5, post-0.12: at exit ours waits only while it has
something left to start for a job, then leaves the running processes,
as the shells do; PROC_SHELLSPAWN_END): a job whose next command ours
still has to start keeps ours from exiting until then (the shells leave
a copy of themselves for it). To exit at once as they do, ours would
run such a job in a Perl process of its own from the start (sh as Perl,
SH1), handing it the state it needs (variables, functions, $1..., the
options, the open descriptors); on Linux only the parent gets a child's
status, so taking a job over at exit is not possible; on Windows a
helper could take over the job's process handles (DuplicateHandle).
- The choices now set by environment variables (PROC_SHELLSPAWN_END,
_ECHO, _SH, _BIN, _INPROC, _ENVNAMES; PROC_SPAWNER_MISSING): a stopgap (the user,
2026-11-15); eventually each is an argument of the API, with no action
at a distance.
- X3 brace expansion; X4 redir_strict; X6 no default PATH; X9 the virtual
directory passed as a path; X15 reference cycles left in the shell layer;
X17 messages inside eval; X24 a trap that re-sends its own signal.
- Settings posix_spawn lacks (M1, M2; pspawn-spawner.md, section 8): a
helper program that sets in the child what cannot be done around the
call (setsid, pgroup 0, tcsetpgrp, sched, cgroup).
- Windows (doc/pspawn-shell-windows.md): kill of another process and time
(X21's rest); ulimit; a test on real Windows. The snippet library under
Wine, now that the shell layer runs there: ours alone on the samples
where it calls no program but those done inside Perl (23,898 by the
logging run of post-0.09), against ours on Linux (large-subset-state.md,
section 7, "W5": the driver was a scratch script, not kept). Of the
user's follow-ups to WS1 (2026-11-09) one remains: cat, sort, grep and
the like have so far only BIN1, a temporary workaround (the user,
2026-11-10; PROC_SHELLSPAWN_BIN, else Git for Windows' or MSYS2's
programs; doc/pspawn-shell-windows.md): what replaces it is open.
- X10: only Linux is tested (and Windows under Wine).
- "Through Run" (large-subset-state.md, section 7) was run before IP1,
with the coreutils programs; the table itself is rebuilt for 0.10's
in-process default, and W5 checked over it on Linux (2026-11-09: no
change).
- X32: without /proc, Perl's own writes into a pipe the caller gave can
block if a child fills the pipe between select and the write (possible,
not observed; no test yet).
- Messages: section 11 of large-subset-environment.md counts them nowhere
(post-0.11), section 7 of large-subset-state.md and README.md's run table
still count a message on standard output or in a file as a difference.
Bring section 7 in line: large-subset-runreport.pl as matchreport.pl
(drop the shells' message lines from standard output; files by
files_nomsg, which needs a new full run); then make docs-check.
- X56: exec with a builtin's name runs ours' builtin, where dash and bash
look only for a program (not found, 127). Fix in the base grammar
(refuse after exec what the large one refuses); the parse results are
then measured again.
- large-subset-difffuzz.pl, what is left after seed 3 (-j 4, two hours,
315,187 strings, 285,676 accepted; it found X50 and X51): of the 826
where dash and bash agree and ours not, 683 have a background job's
EXIT trap (its output comes after the shell ended, or not at all:
chance), 119 other strings with & (X2's effects: wait $! gives 0 for a
job started without a process), 24 without & (X50 and X51, fixed, and
return || shift, A13); a background job that sends a signal to $$
ends with the main shell in ours, while in the shells it is another
process and goes on (X2's kind). Not looked at: the 456 where all three differ.
- X27, step 3, the 93 classified by Claude reading them (A19), what each category
takes in the setups (large-subset-setups.pl): AI-names with an
expansion, a value the text itself sets (a for or read variable, an
assignment: for f in */; do cd "$f"; find | while read d; do cd "$d";
tmpdir=$(mktemp -d) && cd $tmpdir; 19): follow the value through the
text (a glob's matches, a stand-in's answer, the fixed word); a program
at a fixed path not on PATH (2): a stand-in placed there; $HOME/... (5):
as ~/...; AI-tilde: users added to the run's own /etc/passwd; AI-A5:
names with .. in the run's own root; AI-shells' own variables: OLDPWD
varied; AI-PATH-twister: the inherited PATH varied (unset, empty, a
missing directory); AI-glob: example names for patterns with non-ASCII,
a / inside, system directories; AI-strict on a device: a private /dev
with the device absent, a file, not writable.
C. WATCH (seen once, not reproduced)
- t/programs.t, SH1 (10 strings with sh), fails as a user other than root when
PATH has a directory that user may not search (here one under /root):
dash says "Permission denied" for a command found nowhere, ours "not
found" (seen 2026-11-10, branch x27-step2; not from its changes).
- X8, X13, X20.
D. REMINDERS (the user, 2026-11-17: "should we keep these as they are?";
mention them now and then, do not insist)
- JK1 to JK3, the decision page "Executor Idiom Review": its last three
items (JK1: its text is on the page only).
- JK2 (the user, 2026-11-15): the
shell layer counts by hand who still needs each open file (hold,
release, release_all, and $tail: the tables to release once the last
process has started; 42 calls). The page said this cannot be left to
Perl, since the rule of 2026-10-30 (pspawn-design.md, section 5) is to
forget an open file once every process that could use it has started,
which comes before the last Perl reference goes (the code waiting for
the child still refers to the table). The user thinks it might be,
with weak references: to investigate whether the waiting code could
hold the tables weakly, so that the last strong reference goes when
the last process has started, and what commands would then see a pipe
closed at another moment ({ false || cat /etc/hostname; } | wc -l is
the document's example); also X15, the reference cycles left there.
- JK3 (the user, 2026-11-15): whether to rewrite test and [ in
a Perlish way (Proc::ShellSpawn::Programs). Today their eight
functions mirror coreutils 9.4's test.c one for one (posixtest, expr,
and, term, unary_operator, binary_operator, advance, beyond), which
keeps the messages (')' expected, found ']') and oddities (-l STRING)
the same; a rewrite must keep them, checked against the program.
Keyboard Shortcuts
Global
s
Focus search bar
?
Bring up this help dialog
GitHub
gp
Go to pull requests
gi
Go to GitHub issues (only if GitHub is preferred repository)