Shell subset for Perl: decided rules and parse data structure

Status: round 1 decisions plus corrections from chat, as of 2026-10-29; the parse tree (section 8) was reworked and V14–V20 added while writing the parser; Go items C1–C9 (C5 and C7 behind options, C6 narrowed) and question Q1 of the checklist, and the compound commands and read (options, decided in chat on 2026-10-29) "Shell Subset Corpus Round" are applied. Item IDs (P1, E3, …) refer to the checklist page "Perl Shell Subset", where each decision and its discussion are kept. Rules marked with a V-number, such as (V3), are provisional: I chose them while writing this spec. Section 10 lists them for the next checklist round.

1. Purpose

Perl's system(), exec, qx and piped open() hand a command string to /bin/sh -c whenever it contains a shell metacharacter. The goal is a sub-dialect of the Bourne shell that Perl can recognize and parse itself:

The rules aim at the same result as sh -c, whether /bin/sh is dash or bash. Where dash and bash differ, the construct is rejected unless this document says otherwise (P2, D3). Accepted differences from one of the shells:

2. Characters and quoting

"Unquoted" below means: not inside '…' or "…", and not right after a backslash.

3. Lists and pipelines

4. Commands

A command is one of:

Rejected (P3, P4, P5):

Simple commands

cd

Compound commands (options if, while, for, case)

Each construct is accepted only with its own option, off by default. All may be followed by redirections, which apply to the whole construct (while read l; do …; done <file). Each part (condition, body) must contain at least one command, except a case item's body. Reserved words are recognized only unquoted and in command position, as in sh.

5. Expansions

Parameter expansion, arithmetic expansion and command substitution are allowed both unquoted and inside double quotes (Q1). Tilde expansion and patterns apply only to unquoted text.

Parameters (E1)

For future reference, the forms rejected now, with what the discussion found about them:

Arithmetic (E1)

Command substitution (E2)

Tilde (E4)

Word splitting and patterns

Here-documents (D1)

<<, <<- and <<< are rejected.

6. Redirections

7. Rejection

The parser rejects the whole string at the first construct outside these rules. It then returns:

{ error => 'unquoted ( outside a subshell', pos => 17 }   # pos: character offset in the original string

The caller then runs the string with /bin/sh -c (Q2).

8. Parse tree (Perl data structure)

Implemented in ShellSubset.pm: ShellSubset::parse($string, %options) returns a List, or the hash of section 7. Strings inside arrays are operators or literal text; references are nodes. A hash key appears only when it has content.

List     = [ AndOr, ... ]
AndOr    = [ Pipeline, '&&' | '||', Pipeline, ..., '&' ]   # the final '&' only for a background chain
Pipeline = [ '!', Command, ... ]                           # '!' only when negated
Command  = { argv  => [ Word, ... ],                       # argv->[0]: the command name, a plain string
             env   => [ NAME, Word, NAME, Word, ... ],     # settings before the name, in order
             redir => [ Redir, ... ],                      # in the order written
             exec  => 1 }                                  # for 'exec cmd ...'; argv->[0] is then cmd
         | { sub   => List, redir => [ Redir, ... ] }      # ( list )
         | { group => List, redir => [ Redir, ... ] }      # { list; }
         | { cd    => Word }                               # cd alone: { cd => undef }
         | { set   => [ NAME, Word, NAME, Word, ... ] }    # option assign: only settings
         | { if    => [ Cond, Then, Cond, Then, ..., Else ] }   # option if; Else only if present; all Lists
         | { while => Cond, do => List }                   # option while; also { until => Cond, do => List }
         | { for   => NAME, in => [ Word, ... ], do => List }   # option for
         | { case  => Word, in => [ [ Pattern, ... ], List, ... ] }   # option case; patterns are Words
                                                            # (compound commands may also have redir => [...];
                                                            #  break, continue and read are ordinary argv commands)
Redir    = [ fd, op, Word ]                                # op: < > >> <> >|   fd: 0-9, more with D4
         | [ fd, op, 'digit' or '-' ]                      # op: <& >&
Word     = 'text'                                          # no expansion, no pattern
         | [ Part, ... ]                                   # parts are concatenated after expansion
Part     = 'text'                                          # literal, whether it was quoted or not
         | [ '$', NAME ]  | [ '$((', 'expr' ]  | [ '$(', List ]    # unquoted: split into words; NAME may be ? or !
         | [ '"$', NAME ] | [ '"$((', 'expr' ] | [ '"$(', List ]   # inside "…": not split
         | [ '${', NAME, OP, Word ] | [ '${#', NAME ]        # option param_ops; OP is :- - :+ +
                                                            # ('"${', '"${#' inside "…")
         | [ '~', 'user' ]                                 # 'user' is '' for a plain ~
         | [ '*', 'pattern text' ]                         # see below
         | [ '{', Word, Word, ... ]                        # brace option: {a,b,...}
         | [ '{..', 'from', 'to' ]                         # brace option: {from..to}

Example:

LANG=C sort -k2 <in.txt 2>/dev/null | uniq -c >"$OUT" && echo done
[ [ [ { env => [ LANG => 'C' ], argv => [ 'sort', '-k2' ],
        redir => [ [ 0, '<', 'in.txt' ], [ 2, '>', '/dev/null' ] ] },
      { argv => [ 'uniq', '-c' ], redir => [ [ 1, '>', [ [ '"$', 'OUT' ] ] ] ] } ],
    '&&',
    [ { argv => [ 'echo', 'done' ] } ] ] ]

Note that >"$OUT" is allowed: the expansion in the target is quoted (section 6).

9. Caller's options

| Option | Default | Item | |---|---|---| | Reject ^ (option nocaret) | off: ^ is ordinary | C1, C3 | | Reject $PWD (option nopwd) | off: $PWD allowed | C4 | | Unquoted expansion in a redirection target: fail when empty, blank-containing or a pattern (option redir_strict, executor) | off: value used as it is | Q1 | | Commands with only settings (option assign) | off: rejected | C5 | | ${x:-w}, ${x-w}, ${x:+w}, ${x+w}, ${#x} (option param_ops) | off: rejected | C7 | | if … fi (option if) | off: rejected | chat 2026-10-29 | | while, until, with break and continue (option while) | off: rejected | chat 2026-10-29 | | for NAME in …, with break and continue (option for) | off: rejected | chat 2026-10-29 | | case … esac (option case) | off: rejected | chat 2026-10-29 | | read [-r] NAME… (option read) | off: rejected | chat 2026-10-29 | | Checker for strings sh would read differently in the locale | none | C4 | | Accept an empty, blank or comment-only string | off: rejected | D5 | | Several digits as descriptor number | off: rejected | D4 | | Expand ~ after NAME= in arguments | off (dash) | E4 | | Brace expansion parsed into { and {.. parts | off: rejected | V13 | | What to do with pattern words | decided by the caller | E3 |

10. Provisional decisions

Rules I chose while writing this spec; they were not on the checklist. Each is in force until decided otherwise.

| ID | Rule | Section | |---|---|---| | V1 | ^ is rejected inside a bracket expression […] (bash: negation, dash: literal). | 2 | | V2 | (Replaced by corpus-round item C1: a $ that starts no expansion is literal.) | 2 | | V3 | The command name and the word after exec must be literal: no expansion, no pattern. (The cd argument: corpus-round item C8.) | 4 | | V4 | exit n: n must be a literal number from 0 to 255, without leading zeros. | 4 | | V5 | Settings before : or exit are rejected (POSIX shells keep them after the command). | 4 | | V6 | A redirection error on : or exit stops the shell with status 2, as in dash. | 4 | | V7 | exec without a command, with an option, or followed by a builtin name is rejected. (Where exec may stand is decided: B3, choice A in chat.) | 4 | | V8 | PATH, SHELL, TERM, HOSTNAME are rejected only when not in %ENV; the other shell-set names always. | 5 | | V9 | Arithmetic: bare variable values must be decimal integers; $NAME, ${NAME}, $?, $! are substituted as text first (corpus-round items C9, C6); no base#digits; no other $, no `; ? and : must pair up; an empty expression is rejected. | 5 | | V10 | ~ is rejected when HOME is not in %ENV. | 5 | | V11 | Redirection targets: no unquoted pattern character. (Unquoted expansions: corpus-round question Q1.) | 6 | | V12 | An unquoted [ with a ] later in the same word must form a bracket expression with no quote, backslash, expansion, /, ,, {, } or ^ inside; otherwise the string is rejected. A [ with no later ] is a pattern character that matches itself. | 8 | | V13 | With the brace option: only {a,b,…} (at least one unquoted comma) and {x..y} (integers or single letters) are parsed; nested braces, a ~ at the start of an alternative, and , inside a bracket expression are rejected. | 4, 8 | | V14 | Backslash-newline inside an unquoted word or inside backquotes is rejected; between words and inside "…" it is removed. | 2 | | V15 | cd with settings, redirections, an option, or more than one argument is rejected. (cd '' is accepted since the corpus round.) | 4 | | V16 | ! negates only when followed by a blank; a word ! elsewhere in command-name position is rejected. | 3 | | V17 | Keywords are recognized only when the command name is written unquoted ("if" x runs the program if, as sh does); builtin names are rejected also when quoted ("set" is still the builtin in sh). | 4 | | V18 | After ~, only a user name may come before / or the end of the word (or : in settings); ~"root", ~$x are rejected (both shells keep them literal). | 5 | | V19 | ! exec cmd is rejected (both shells lose the !). | 4 | | V20 | $( ) with an empty list is accepted (empty string); empty ( ) and { } are rejected. | 5 |