/ 
Perl5-TestEachCommit-0.02
/ Perl5::TestEachCommit

NAME

Perl5::TestEachCommit - Test each commit in a pull request to Perl core

SYNOPSIS

use Perl5::TestEachCommit;

$self = Perl5::TestEachCommit->new();

$self->prepare_repository();
$self->display_plan();
$self->get_commits();
$self->display_commits();
$self->examine_all_commits();
$self->get_results();
$self->display_results();

DESCRIPTION

This library is intended for use by people working to maintain the Perl core distribution.

Commits to blead, the main development branch in the Perl repository, are most often done by pull requests. Most such p.r.s consist of a single commit, but commits of forty or eighty are not unknown. A continuous integration system (CI) ensures that each p.r. is configured, built and tested on submission and on subsequent modifications. That CI system, however, only executes that cycle on the *final* commit in each p.r. It cannot detect any failure in a *non-final* commit. This library provides a way to test each commit in the p.r. to the same extent that the CI system tests the final commit.

Why is this important? Suppose that we have a pull request that consists of 5 commits. In commit 3 the developer makes an error which causes make to fail. The developer notices that and corrects the error in commit 4. Commit 5 configures, builds and tests satisfactorily, so the CI system gives the p.r. as a whole a PASS. The committer uses that PASS as the basis for approving a merge of the branch into blead.

Commit  Configure   Build       Test
------------------------------------
1abcd       X         X           X
2efab       X         X           X
3cdef       X         0           -
4dcba       X         X           X
5fedc       X         X           X

If, for any reason (*e.g.,* bisection), some other developer in the future needs to say git checkout 3cdef, they will discover that at that commit the build was actually broken.

METHODS

new()

  • Purpose

    Perl5::TestEachCommit constructor. Ensures that supplied arguments are plausible, e.g., directories needed can be located.

  • Arguments

    my $self = Perl5::TestEachCommit->new( { %opts } );

    Single hash reference. That hash must include the following key-value pairs:

      * workdir

      String holding path to a directory which is a git checkout of the Perl core distribution. If you have previously set an environmental variable SECONDARY_CHECKOUT_DIR holding the path to such a directory, that will be used; otherwise, path must be specified.

      * start

      String holding SHA of the first commit in the series on which you wish reporting.

      * end

      String holding SHA of the last commit in the series on which you wish reporting.

    In addition, that hash may include the following key-value pairs:

      * branch

      git branch which must exist and be available for git checkout in the directory specified by workdir. Defaults to blead.

      * configure_command

      String holding arguments to be passed to ./Configure. Defaults to sh ./Configure -des -Dusedevel. Add < 1<Egt>/dev/null> to that string if you don't need voluminous output to STDOUT.

      * make_test_prep_command

      String holding arguments to be passed to make test_prep. Defaults to make test_prep. Add < 1<Egt>/dev/null> to that string if you don't need voluminous output to STDOUT.

      * make_test_harness_command

      String holding arguments to be passed to make test_harness. Defaults to make test_harness. Add < 1<Egt>/dev/null> to that string if you don't need voluminous output to STDOUT.

      * skip_test_harness

      True/false value. Defaults to false. If true, when proceeding through a series of commits in a branch or pull request, the make test_harness stage will be skipped on the assumption that any significant failures are going to appear in the first two stages.

      * verbose

      True/false value. Defaults to false. If true, prints to STDOUT a summary of switches in use and commits being tested.

  • Return Value

    Perl5::TestEachCommit object (blessed hash reference).

prepare_repository()

  • Purpose

    Prepare the workdir directory for git operations, e.g., terminates any bisection in process, cleans the directory, fetches from origing, checks out blead, then checks out any non-blead branch indicated in the branch argument to new().

  • Arguments

    None.

    my $rv = $self->prepare_repostory();

  • Return Value

    Returns true value upon success.

display_plan()

  • Purpose

    Display most important configuration choices.

  • Arguments

    $self->display_plan();

  • Return Value

    Implicitly returns true value upon success.

  • Comment

    The output will look like this:

    branch:                    blead
configure_command:         sh ./Configure -des -Dusedevel 1>/dev/null
make_test_prep_command:    make test_prep 1>/dev/null
make_test_harness_command: make_test_harness 1>/dev/null

get_commits()

  • Purpose

    Get a list of SHAs of all commits being tested.

  • Arguments

    my $lines = $self->get_commits();

  • Return Value

    Reference to an array holding list of all commits being tested.

display_commits()

  • Purpose

    Display a list of SHAs of all commits being tested.

  • Arguments

    $self->display_commits();

  • Return Value

    Implicitly returns true value upon success.

  • Comment

    The output will look like this:

    c9cd2e0cf4ad570adf68114c001a827190cb2ee9
79b32d926ef5961b4946ebe761a7058cb235f797
0dfa8ac113680e6acdef0751168ab231b9bf842c

examine_all_commits()

  • Purpose

    Iterate over all commits in the selected range, configuring, building and -- assuming we have not elected to skip_test_harness -- testing each commit.

  • Arguments

    $self->examine_all_commits();

  • Return Value

    For possible future chaining, returns the Perl5::TestEachCommit object, which now includes the results of the examination of each commit in the selected range.

get_results()

  • Purpose

    Get a list of the SHA and score for each commit.

  • Arguments

    my $results_ref = $self->get_results();

  • Return Value

    Reference to an array holding a hashref for each commit. Each such hashref has two elements: commit and score. (See examine_one_commit.)

display_results()

  • Purpose

    Pretty-print to STDOUT the results obtained via get_results().

  • Arguments

    $self->display_results();

  • Return Value

    Implicitly returns a true value upon success.

  • Comment

    The output will look like this:

                     commit                    score
------------------------------------------------
c9cd2e0cf4ad570adf68114c001a827190cb2ee9 |   2
79b32d926ef5961b4946ebe761a7058cb235f797 |   1
0dfa8ac113680e6acdef0751168ab231b9bf842c |   2

examine_one_commit()

  • Purpose

    Configure, build and test one commit in the selected range.

  • Arguments

    my $score_ref = $self->examine_one_commit($this_SHA);

  • Return Value

    Returns the Perl5::TestEachCommit object, how holding a list of results.

    • commit: the commit's SHA.

    • score: A numeral between 0 and 3 indicating how many stages the commit completed successfully:

      0 Unable to configure.
      1 Completed configuration only.
      2 Completed configuration and build only.
      3 Completed all of configuration, build and testing.

  • Comment

    Called internally within examine_all_commits().

BUGS

None reported so far.

SUPPORT

Contact the author.

AUTHOR

James E Keenan
CPAN ID: JKEENAN
jkeenan@cpan.org
https://thenceforward.net/perl/modules/Perl5-TestEachCommit

COPYRIGHT

Copyright 2025 James E Keenan

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

The full text of the license can be found in the LICENSE file included with this module.

SEE ALSO

perl(1).