/ 
App-Test-Generator-0.46
/ App::Test::Generator::Mutator

NAME

App::Test::Generator::Mutator - Generate and apply mutation tests

VERSION

Version 0.46

SYNOPSIS

use App::Test::Generator::Mutator;

my $mutator = App::Test::Generator::Mutator->new(
    file           => 'lib/My/Module.pm',
    lib_dir        => 'lib',
    mutation_level => 'fast',
);

my $mutants = $mutator->generate_mutants();
printf "Generated %d mutants\n", scalar @{$mutants};

my $workspace = $mutator->prepare_workspace();

for my $m (@{$mutants}) {
    $mutator->apply_mutant($m);
    if($mutator->run_tests()) {
        print "SURVIVED: ${\$m->description}\n";
    } else {
        print "KILLED:   ${\$m->description}\n";
    }
}

DESCRIPTION

App::Test::Generator::Mutator is a mutation engine that programmatically alters Perl source files to evaluate the effectiveness of a project's test suite. It analyses modules, generates systematic code mutations (such as conditional inversions, logical operator changes, and numeric boundary flips), and applies them within an isolated workspace so tests can be executed safely against each modified variant.

By tracking which mutants are killed (cause tests to fail) versus those that survive (tests still pass), the module enables calculation of a mutation score, providing a quantitative measure of how well the test suite detects unintended behavioural changes.

new

Construct a new Mutator for a given source file.

my $mutator = App::Test::Generator::Mutator->new(
    file           => 'lib/My/Module.pm',
    lib_dir        => 'lib',
    mutation_level => 'full',
);

Arguments

  • file

    Path to the Perl source file to mutate. Required. Must exist on disk.

  • lib_dir

    Root library directory. Optional - defaults to lib.

  • mutation_level

    Controls the breadth of mutation. full applies all mutations; fast deduplicates and removes redundant mutants first. Optional - defaults to full.

Returns

A blessed hashref. Croaks if file is missing or does not exist.

API specification

input

{
    file           => { type => SCALAR },
    lib_dir        => { type => SCALAR, optional => 1 },
    mutation_level => { type => SCALAR, optional => 1 },
}

output

{
    type => OBJECT,
    isa  => 'App::Test::Generator::Mutator',
}

EXAMPLE

my $m = App::Test::Generator::Mutator->new(
    file           => 'lib/Acme/Widget.pm',
    mutation_level => 'fast',
);

MESSAGES

file required

file was not supplied.

file not found: PATH

file was supplied but does not exist on disk.

FORMAL SPECIFICATION

Pre: file is defined ∧ -f file

Post: ref(result) eq 'App::Test::Generator::Mutator' ∧ result->{file} eq file ∧ result->{mutation_level} ∈ {full, fast}

generate_mutants

Parse the target file and generate all mutants by running each registered mutation strategy against the PPI document.

my $mutants = $mutator->generate_mutants();   # scalar context → arrayref
for my $m (@{$mutants}) { ... }

my @mutants = $mutator->generate_mutants();   # list context → flat list (backward-compat)

Arguments

None beyond $self.

Returns

An arrayref of App::Test::Generator::Mutant objects. In fast mode, redundant and duplicate mutants are removed before returning. Lines within ## MUTANT_SKIP_BEGIN / ## MUTANT_SKIP_END annotation blocks are excluded from the candidate list entirely. After this method returns, $self->{skip_lines} contains a hashref mapping excluded line numbers to 1.

API specification

input

{
    self => { type => OBJECT, isa => 'App::Test::Generator::Mutator' },
}

output

{
    type     => ARRAYREF,
    elements => { type => OBJECT, isa => 'App::Test::Generator::Mutant' },
}

EXAMPLE

my $mutants = $mutator->generate_mutants();
printf "%d mutants generated\n", scalar @{$mutants};

for my $m (@{$mutants}) {
    printf "line %d: %s\n", $m->line, $m->description;
}

MESSAGES

Unable to parse FILE

PPI could not parse the source file (syntax error or unreadable file). FILE is the path passed to new.

FILE: MUTANT_SKIP_BEGIN at line N with no prior MUTANT_SKIP_END

A ## MUTANT_SKIP_BEGIN marker was found while already inside a skip block.

FILE: MUTANT_SKIP_END at line N with no matching MUTANT_SKIP_BEGIN

A ## MUTANT_SKIP_END marker was found with no preceding ## MUTANT_SKIP_BEGIN.

FILE: MUTANT_SKIP_BEGIN at line N has no matching MUTANT_SKIP_END

The source file ended while still inside a skip block.

FORMAL SPECIFICATION

Pre: prepare_workspace need not have been called before this method.

Post: ref(result) eq 'ARRAY' ∧ ∀ m ∈ result: ref(m) eq 'App::Test::Generator::Mutant' ∧ ∀ m ∈ result: ¬ skip_lines{m-line}>

PSEUDOCODE

parse file with PPI
scan lines for MUTANT_SKIP_BEGIN / MUTANT_SKIP_END pairs → skip_lines
for each registered mutation strategy:
    skip if strategy does not apply_to(doc)
    for each mutant from strategy->mutate(doc):
        include unless mutant.line ∈ skip_lines
if mutation_level == 'fast':
    deduplicate and remove redundant mutants
return arrayref (or flat list in list context)

prepare_workspace

Prepare an isolated temporary workspace for a single mutation test run.

The entire lib_dir tree is copied into the workspace so that all module dependencies resolve correctly when the test suite runs against the mutant. Only after this copy is complete is the single target file overwritten by apply_mutant.

my $workspace = $mutator->prepare_workspace();
$mutator->apply_mutant($mutant);
local $ENV{PERL5LIB} = "$workspace/lib";
my $survived = (system('prove', 't') == 0);

Arguments

None beyond $self.

Returns

A string containing the absolute path to the temporary directory created. The directory is automatically removed when the object goes out of scope via File::Temp's CLEANUP => 1 behaviour.

Side effects

Creates a temporary directory. Recursively copies lib_dir into it. Sets $self->{_workspace}, $self->{_relative}, and $self->{_lib_basename}. Does not modify $self->{lib_dir}.

Notes

Call prepare_workspace once per file, then apply_mutant once per mutant within that file. Do not store the returned path beyond the lifetime of the enclosing scope.

API specification

input

{
    self => { type => OBJECT, isa => 'App::Test::Generator::Mutator' },
}

output

{
    type => SCALAR,
}

EXAMPLE

my $workspace = $mutator->prepare_workspace();
# workspace is an absolute temp dir path
# original lib_dir value is unchanged
printf "original lib_dir still: %s\n", $mutator->{lib_dir};

MESSAGES

dircopy failed: $!

The lib_dir tree could not be copied into the temporary workspace directory, usually a permissions error.

FORMAL SPECIFICATION

Pre: -d self->{lib_dir} ∧ self->{file} begins with self->{lib_dir}

Post: -d result ∧ self->{_workspace} eq result ∧ self->{lib_dir} unchanged

apply_mutant

Apply a single mutant's transform to the target file in the workspace.

$mutator->apply_mutant($mutant);

Arguments

Returns

Nothing. Modifies the workspace copy of the target file in place.

Side effects

Overwrites the target file in the workspace with the mutated version.

API specification

input

{
    self   => { type => OBJECT, isa => 'App::Test::Generator::Mutator' },
    mutant => { type => OBJECT, isa => 'App::Test::Generator::Mutant'  },
}

output

{ type => UNDEF }

EXAMPLE

$mutator->prepare_workspace();
for my $m (@{$mutants}) {
    $mutator->apply_mutant($m);
    # workspace file is now mutated; run tests against it
}

MESSAGES

Workspace not prepared -- call prepare_workspace first

apply_mutant was called before prepare_workspace.

Relative path not set -- call prepare_workspace first

Internal: the relative-path field was not set by prepare_workspace.

Failed to parse TARGET

PPI could not parse the workspace copy of the target file.

FORMAL SPECIFICATION

Pre: self->{_workspace} is defined ∧ self->{_relative} is defined ∧ ref(mutant->{transform}) eq 'CODE'

Post: workspace copy of target file contains the mutated content

run_tests

Run the test suite against the current workspace and return whether all tests passed.

my $survived = $mutator->run_tests();

Arguments

None beyond $self.

Returns

1 if all tests passed (mutant survived), 0 if any test failed (mutant killed).

Side effects

Executes an external process running the test suite.

Notes

Uses prove found on PATH. Sets PERL5LIB to include the workspace lib directory before running.

API specification

input

{
    self => { type => OBJECT, isa => 'App::Test::Generator::Mutator' },
}

output

{ type => SCALAR }

EXAMPLE

my $survived = $mutator->run_tests();
if($survived) {
    print "mutant survived\n";
} else {
    print "mutant killed\n";
}

FORMAL SPECIFICATION

Post: result ∈ {0, 1} ∧ result == 1 ⟺ all tests in t/ passed against current lib/

COMMON PITFALLS

Calling apply_mutant before prepare_workspace

apply_mutant requires prepare_workspace to have been called first. The workspace holds the isolated copy of lib/ that receives mutations.

Passing an absolute path as lib_dir

lib_dir must be a relative path (e.g. lib). An absolute path causes apply_mutant to construct a doubled directory under the workspace and then fail to find the target file.

Checking $mutator->{workspace} after the refactor

Internal workspace state is stored in _workspace, _relative, and _lib_basename (note the underscore prefix). The original lib_dir value is never overwritten. Old code that reads $mutator->{workspace} or $mutator->{relative} (without underscores) will not find these keys.

Forgetting that run_tests drives prove from $^X

run_tests resolves prove from the same Perl binary used to run the script. If you shell out to prove directly in your own integration code, make sure you are using the matching prove.

generate_mutants in list vs scalar context

generate_mutants returns a flat list in list context and an arrayref in scalar context. Assign to an arrayref (my $m = $mutator->generate_mutants()) to guarantee you always get a reference regardless of calling context.

LIMITATIONS

  • Single strategy registry

    The four built-in mutation strategies are hardcoded in new. There is no plugin mechanism for registering additional strategies without subclassing. A future version should accept a strategies arrayref argument.

  • No parallelism

    run_tests is synchronous. For large test suites or large mutant sets, wall-clock time scales linearly. The in-place mutation strategy also serialises all mutants behind a single file lock.

  • PPI re-parse per apply_mutant call

    apply_mutant re-parses the workspace copy of the target file for every mutant. For very large single-file modules the PPI parse time may dominate.

  • apply_mutant does not restore on abnormal exit

    If the process is killed between the write and restore in bin/test-generator-mutate, the project file is left mutated. A git restore lib/... recovers it.

SEE ALSO

bin/test-generator-mutate
Devel::Mutator

AUTHOR

Nigel Horne, <njh at nigelhorne.com>

LICENCE AND COPYRIGHT

Copyright 2026 Nigel Horne.

Usage is subject to the terms of GPL2. If you use it, please let me know.

1 POD Error

The following errors were encountered while parsing the POD:

Around line 160:

Non-ASCII character seen before =encoding in '∧'. Assuming UTF-8