NAME
generate_index.pl - Test coverage dashboard generator
DESCRIPTION
generate_index.pl generates an HTML test coverage dashboard for publication on GitHub Pages, combining four sources of test quality data into a single report:
Statement and branch coverage from Devel::Cover, showing which lines and branches were exercised by the test suite.
LCSAJ path coverage (TER3) from the LCSAJ runtime debugger, showing which control-flow paths were executed. Displayed as blue (covered) or red (uncovered) dots on per-file mutation pages.
Mutation testing results from
bin/app-test-generator-mutate, showing which injected faults the test suite detected (killed) and which it missed (survived).CPAN Testers failure analysis, showing which Perl versions and operating systems are failing, with automatic root cause detection including Perl version cliffs, locale sensitivity, and dependency version cliffs.
In addition to the dashboard, the script drives the mutation-guided test generation pipeline described in "MUTATION-GUIDED TEST GENERATION" in App::Test::Generator. Surviving mutants are used to automatically generate new test stubs and fuzz schemas that target the exact boundary conditions the test suite missed:
--generate_mutant_tests=DIRproduces TODO stub files inDIR/for all surviving mutants, grouped by source file and deduplicated by line.--generate_test=mutantadditionally attempts to produce runnable YAML schemas inDIR/conf/for NUM_BOUNDARY survivors using App::Test::Generator::SchemaExtractor.--generate_fuzzaugments existing schemas inDIR/conf/with boundary values from surviving mutants, writing timestamped copies that are picked up automatically byt/fuzz.t.
The script is designed to be shared across projects. Copy it into the scripts/ directory of each project that uses it:
cp ../App-Test-Generator/scripts/generate_index.pl scripts/It is invoked automatically by scripts/generate_test_dashboard on each CI push via .github/workflows/dashboard.yml.
SYNOPSIS
https://$github_user.github.io/$github_repo/coverage/INPUTS
cover_html/cover.json - Devel::Cover JSON report (statement/branch/condition)
mutation.json - Mutation testing results from app-test-generator-mutate
cover_html/lcsaj_hits.json - LCSAJ path hit data from the LCSAJ runtime debugger
cover_html/mutation_html/lib/ - Per-file LCSAJ path definitions (.lcsaj.json)
coverage_history/*.json - Historical coverage snapshots for the trend chartOUTPUTS
cover_html/index.html - Main dashboard (coverage table, trend chart,
CPAN Testers failures, mutation report)
cover_html/mutation_html/ - Per-file mutation heatmap pagesOPTIONS
--generate_mutant_tests=DIR
Generate a timestamped test stub file in DIR (typically 't/') for
surviving mutants. The file is named mutant_YYYYMMDD_HHMMSS.t and
contains:
- TODO test stubs for High/Medium difficulty survivors, with
boundary value suggestions, environment variable hints, and
the enclosing subroutine name for navigation context
- Comment-only hints for Low difficulty survivors
Multiple mutations on the same source line are deduplicated into
one stub - one good test kills all variants on that line.
File is skipped entirely if there are no survivors to report.
If not given, no test stubs are generated.
--generate_test=CLASS
When combined with --generate_mutant_tests=DIR, attempts to produce
runnable test artefacts for surviving mutants rather than TODO stubs.
Currently supported classes:
mutant For NUM_BOUNDARY survivors, calls
App::Test::Generator::SchemaExtractor to extract the schema
for the enclosing subroutine and augments it with the
boundary value from the mutant (plus one value either side).
The resulting YAML schema is written to DIR/conf/ and is
picked up automatically by t/fuzz.t on the next test run.
Falls back to a TODO stub if SchemaExtractor fails, the
enclosing sub cannot be determined, or the extracted schema
confidence is too low (very_low or none).
This option is designed to accept additional classes in future, for
example corpus-driven or property-based test generation.
If not given, only TODO stubs are produced.
--generate_fuzz
Scans t/conf/ for existing YAML schema files and augments copies
of them with boundary values extracted from surviving NUM_BOUNDARY
mutants whose enclosing subroutine matches the schema's function
field. The original schema is never modified. Augmented copies are
written to t/conf/mutant_fuzz_YYYYMMDD_HHMMSS_FUNCTION.yml and
are picked up automatically by t/fuzz.t on the next test run.
Schemas whose filename already starts with mutant_fuzz_ are skipped
to avoid augmenting previously augmented schemas. Schemas with no
matching survivors are skipped (with a note if --verbose is active).
New boundary values are merged into whichever edge key already
exists in the schema (edge_case_array or edge_cases), with
deduplication against existing values.
This flag is independent of --generate_test and can be used alone.DASHBOARD SECTIONS
Coverage Table - Per-file statement/branch/condition/subroutine
percentages with delta vs previous snapshot,
sortable columns, and sparkline trend per file
Coverage Trend - Chart of total coverage over recent commits with
linear regression line, zoom and pan support
RT Issues - Count of open RT tickets for the distribution
CPAN Testers - Failure table for the current release, with
Perl version cliff detection, locale analysis,
dependency version cliff detection, and root
cause confidence scoring
Mutation Report - Per-file mutation score (killed/survived/total),
cyclomatic complexity, and TER3 (LCSAJ path
coverage) with raw fraction
Per-file Pages - Line-by-line mutation heatmap with TER1/TER2/TER3
metrics, LCSAJ path markers, and expandable
mutant details with suggested testsDEPENDENCIES
Cwd, Data::Dumper, File::Basename, File::Glob, File::Path,
File::Slurp, File::Spec, File::stat, Getopt::Long, HTML::Entities,
HTTP::Tiny, IPC::Run3, JSON::MaybeXS, List::Util, POSIX,
Readonly, Time::HiRes, URI::Escape, WWW::RT::CPAN, versionAUTHOR
Nigel Horne <njh@nigelhorne.com>