/ 
Perl5-TestEachCommit-0.07
/ Perl5::TestEachCommit

NAME

Perl5::TestEachCommit - Test each commit in a pull request to Perl core

SYNOPSIS

use Perl5::TestEachCommit;

$self = Perl5::TestEachCommit->new();

$self->prepare_repository();
$self->display_plan();
$self->get_commits();
$self->display_commits();
$self->examine_all_commits();
$self->get_results();
$self->display_results();

DESCRIPTION

This library is intended for use by people working to maintain the Perl core distribution.

Commits to blead, the main development branch in the Perl repository, are most often done by pull requests. Most such p.r.s consist of a single commit, but commits of forty or eighty are not unknown. A continuous integration system (CI) ensures that each p.r. is configured, built and tested on submission and on subsequent modifications. That CI system, however, only executes that cycle on the *final* commit in each p.r. It cannot detect any failure in a *non-final* commit. This library provides a way to test each commit in the p.r. to the same extent that the CI system tests the final commit.

Why is this important? Suppose that we have a pull request that consists of 5 commits. In commit 3 the developer makes an error which causes make to fail. The developer notices that and corrects the error in commit 4. Commit 5 configures, builds and tests satisfactorily, so the CI system gives the p.r. as a whole a PASS. The committer uses that PASS as the basis for approving a merge of the branch into blead.

Commit  Configure   Build       Test
------------------------------------
1abcd       X         X           X
2efab       X         X           X
3cdef       X         0           -
4dcba       X         X           X
5fedc       X         X           X

If, for any reason (*e.g.,* bisection), some other developer in the future needs to say git checkout 3cdef, they will discover that at that commit the build was actually broken.

Additional Functionality

As of version 0.06, this distribution now provides the functionality to analyze breakage at the miniperl level. If it is known (or suspected) that tests will break during make minitest_prep or make minitest, then runtime can be considerably reduced from the more typical case where breakage occurs during make test_prep or make test_harness. However, the two cases cannot be mixed.

METHODS

new()

  • Purpose

    Perl5::TestEachCommit constructor. Ensures that supplied arguments are plausible, e.g., directories needed can be located.

  • Arguments

    my $self = Perl5::TestEachCommit->new( { %opts } );

    Single hash reference. That hash must include the following key-value pairs:

      * workdir

      String holding path to a directory which is a git checkout of the Perl core distribution. If you have previously set an environmental variable SECONDARY_CHECKOUT_DIR holding the path to such a directory, that will be used; otherwise, path must be specified.

      * start

      String holding SHA of the first commit in the series on which you wish reporting.

      * end

      String holding SHA of the last commit in the series on which you wish reporting.

    In addition, that hash may include the following key-value pairs:

      * branch

      git branch which must exist and be available for git checkout in the directory specified by workdir. Defaults to blead.

      * configure_command

      String holding arguments to be passed to ./Configure. Defaults to sh ./Configure -des -Dusedevel. Add < 1<Egt>/dev/null> to that string if you don't need voluminous output to STDOUT.

      * make_test_prep_command

      String holding arguments to be passed to make test_prep. Defaults to make test_prep. Add < 1<Egt>/dev/null> to that string if you don't need voluminous output to STDOUT.

      * make_test_harness_command

      String holding arguments to be passed to make test_harness. Defaults to make test_harness. Add < 1<Egt>/dev/null> to that string if you don't need voluminous output to STDOUT.

      * skip_test_harness

      True/false value. Defaults to false. If true, when proceeding through a series of commits in a branch or pull request, the make test_harness stage will be skipped on the assumption that any significant failures are going to appear in the first two stages.

      * make_minitest_prep_command

      String holding arguments to be passed to make minitest_prep. Defaults to make minitest_prep. Add < 1<Egt>/dev/null> to that string if you don't need voluminous output to STDOUT. Note: If this key-value pair is passed to new(), then any value passed to make_test_prep_command or make_test_harness_command is ignored.

      * make_minitest_command

      String holding arguments to be passed to make minitest. Defaults to make minitest. Add < 1<Egt>/dev/null> to that string if you don't need voluminous output to STDOUT. Note: If this key-value pair is passed to new(), then any value passed to make_test_prep_command or make_test_harness_command is ignored.

      * verbose

      True/false value. Defaults to false. If true, prints to STDOUT a summary of switches in use and commits being tested.

  • Return Value

    Perl5::TestEachCommit object (blessed hash reference).

prepare_repository()

  • Purpose

    Prepare the workdir directory for git operations, e.g., terminates any bisection in process, cleans the directory, fetches from origing, checks out blead, then checks out any non-blead branch indicated in the branch argument to new().

  • Arguments

    None.

    my $rv = $self->prepare_repostory();

  • Return Value

    Returns true value upon success.

display_plan()

  • Purpose

    Display most important configuration choices.

  • Arguments

    $self->display_plan();

  • Return Value

    Implicitly returns true value upon success.

  • Comment

    The output will look like this:

    branch:                    blead
configure_command:         sh ./Configure -des -Dusedevel 1>/dev/null
make_test_prep_command:    make test_prep 1>/dev/null
make_test_harness_command: make_test_harness 1>/dev/null

    Or like this, if you are doing miniperl-level testing on a debugging build:

    branch:                        blead
configure_command:             sh ./Configure -des -Dusedevel -DDEBUGGING 1>/dev/null
make_minitest_prep_command:    make minitest_prep 1>/dev/null
make_minitest_command:         make minitest 1>/dev/null

get_commits()

  • Purpose

    Get a list of SHAs of all commits being tested.

  • Arguments

    my $lines = $self->get_commits();

  • Return Value

    Reference to an array holding list of all commits being tested.

  • Comment

    An exception will be thrown if the array holding the list of all commits being tested is empty. This can occur, for example, if you mistakenly interchange the values for the --start and --end commits.

display_commits()

  • Purpose

    Display a list of SHAs of all commits being tested.

  • Arguments

    $self->display_commits();

  • Return Value

    Implicitly returns true value upon success.

  • Comment

    The output will look like this:

    c9cd2e0cf4ad570adf68114c001a827190cb2ee9
79b32d926ef5961b4946ebe761a7058cb235f797
0dfa8ac113680e6acdef0751168ab231b9bf842c

examine_all_commits()

  • Purpose

    Iterate over all commits in the selected range, configuring, building and -- assuming we have not elected to skip_test_harness -- testing each commit.

  • Arguments

    $self->examine_all_commits();

  • Return Value

    For possible future chaining, returns the Perl5::TestEachCommit object, which now includes the results of the examination of each commit in the selected range.

get_results()

  • Purpose

    Get a list of the SHA and score for each commit.

  • Arguments

    my $results_ref = $self->get_results();

  • Return Value

    Reference to an array holding a hashref for each commit. Each such hashref has two elements: commit and score. (See examine_one_commit.)

display_results()

  • Purpose

    Pretty-print to STDOUT the results obtained via get_results().

  • Arguments

    $self->display_results();

  • Return Value

    Implicitly returns a true value upon success.

  • Comment

    The output will look like this:

                     commit                    score
------------------------------------------------
c9cd2e0cf4ad570adf68114c001a827190cb2ee9 |   2
79b32d926ef5961b4946ebe761a7058cb235f797 |   1
0dfa8ac113680e6acdef0751168ab231b9bf842c |   2

examine_one_commit()

  • Purpose

    Configure, build and test one commit in the selected range.

  • Arguments

    my $score_ref = $self->examine_one_commit($this_SHA);

  • Return Value

    Returns the Perl5::TestEachCommit object, how holding a list of results.

    • commit: the commit's SHA.

    • score: A numeral between 0 and 3 indicating how many stages the commit completed successfully:

      0 Unable to configure.
      1 Completed configuration only.
      2 Completed configuration and build only.
      3 Completed all of configuration, build and testing.

  • Comment

    Called internally within examine_all_commits().

cleanup_repository()

  • Purpose

    Clean up the repository in the directory designated by workdir.

  • Arguments

    $self->cleanup_respository();

  • Return Value

    Implicitly returns a true value upon success.

  • Comment

    Performs a git clean and git checkout blead but does not do any fetching from origin or updating of blead.

BUGS

None reported so far.

SUPPORT

Contact the author.

AUTHOR

James E Keenan
CPAN ID: JKEENAN
jkeenan@cpan.org
https://thenceforward.net/perl/modules/Perl5-TestEachCommit

COPYRIGHT

Copyright 2025 James E Keenan

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

The full text of the license can be found in the LICENSE file included with this module.

SEE ALSO

perl(1).