NAME

Devel::Git::MultiBisect::AllCommits - Gather test output over an entire range of git commits

SYNOPSIS

use Devel::Git::MultiBisect::AllCommits;

$self = Devel::Git::MultiBisect::AllCommits->new(\%parameters);

$commit_range = $self->get_commits_range();

$full_targets = $self->set_targets(\@target_args);

$outputs = $self->run_test_files_on_one_commit($commit_range->[0]);

$all_outputs = $self->run_test_files_on_all_commits();

$rv = $self->get_digests_by_file_and_commit();

$transitions = $self->examine_transitions();

DESCRIPTION

Given a Perl library or application kept in git for version control, it is often useful to be able to compare the output collected from running one or several test files over a range of git commits. If that range is sufficiently large, a test may fail in more than one way over that range.

If that is the case, then simply asking, "When did this file start to fail?" is insufficient. We may want to capture the test output for each commit, or, more usefully, may want to capture the test output only at those commits where the output changed.

Devel::Git::MultiBisect provides methods to achieve that objective. More specifically:

When you want to capture the test output for each commit in a specified range, you can use this package, Devel::Git::MultiBisect::AllCommits.
When the number of commits in the specified range is large and you only need the test output at those commits where the output materially changed, you can use another package found in this library, Devel::Git::MultiBisect::Transitions.

METHODS

This package inherits methods from Devel::Git::MultiBisect. Only methods unique to Devel::Git::MultiBisect::AllCommits are documented here. See the documentation for Devel::Git::MultiBisect for all other methods, including:

new()
get_commits_range()
set_targets()
run_test_files_on_one_commit()

`run_test_files_on_all_commits()`

Purpose

Capture the output from a run of the selected test files at each specific git checkout in the selected commit range.
Arguments
```
$all_outputs = $self->run_test_files_on_all_commits();
```
None; all data needed is already present in the object.

Return Value

Array reference, each of whose elements is an array reference, each of whose elements is a hash reference with the same four keys as in the return value from run_test_files_on_one_commit():

commit
commit_short
file
file_stub
md5_hex

Example:

[
  # Array where each element corresponds to a single git checkout

  [
    # Array where each element corresponds to one of the selected test
    # files (here, 2 test files were targetd)

    {
      # Hash where each element correponds to the result of running a
      # single test file at a single commit point

      commit => "2a2e54af709f17cc6186b42840549c46478b6467",
      commit_short => "2a2e54a",
      file => "/tmp/AHC_YkwUYg/2a2e54a.t_44_func_hashes_mult_unsorted_t.output.txt",
      file_stub => "t_44_func_hashes_mult_unsorted_t",
      md5_hex => "31b7c93474e15a16d702da31989ab565",
    },
    {
      commit => "2a2e54af709f17cc6186b42840549c46478b6467",
      commit_short => "2a2e54a",
      file => "/tmp/AHC_YkwUYg/2a2e54a.t_45_func_hashes_alt_dual_sorted_t.output.txt",
      file_stub => "t_45_func_hashes_alt_dual_sorted_t",
      md5_hex => "6ee767b9d2838e4bbe83be0749b841c1",
    },
  ],
  [
    {
      commit => "a624024294a56964eca53ec4617a58a138e91568",
      commit_short => "a624024",
      file => "/tmp/AHC_YkwUYg/a624024.t_44_func_hashes_mult_unsorted_t.output.txt",
      file_stub => "t_44_func_hashes_mult_unsorted_t",
      md5_hex => "31b7c93474e15a16d702da31989ab565",
    },
    {
      commit => "a624024294a56964eca53ec4617a58a138e91568",
      commit_short => "a624024",
      file => "/tmp/AHC_YkwUYg/a624024.t_45_func_hashes_alt_dual_sorted_t.output.txt",
      file_stub => "t_45_func_hashes_alt_dual_sorted_t",
      md5_hex => "6ee767b9d2838e4bbe83be0749b841c1",
    },
  ],
# ...
  [
    {
      commit => "d304a207329e6bd7e62354df4f561d9a7ce1c8c2",
      commit_short => "d304a20",
      file => "/tmp/AHC_YkwUYg/d304a20.t_44_func_hashes_mult_unsorted_t.output.txt",
      file_stub => "t_44_func_hashes_mult_unsorted_t",
      md5_hex => "31b7c93474e15a16d702da31989ab565",
    },
    {
      commit => "d304a207329e6bd7e62354df4f561d9a7ce1c8c2",
      commit_short => "d304a20",
      file => "/tmp/AHC_YkwUYg/d304a20.t_45_func_hashes_alt_dual_sorted_t.output.txt",
      file_stub => "t_45_func_hashes_alt_dual_sorted_t",
      md5_hex => "6ee767b9d2838e4bbe83be0749b841c1",
    },
  ],
]

Comment

Note: If the number of commits in the commits range is large, this method will take a long time to run. That time will be even longer if the configuration and build times for each commit are large. For example, to run one test over 160 commits from the Perl 5 core distribution might take 15 hours. YMMV. If either of these conditions holds, you are probably better off using Devel::Git::MultiBisect::Transitions.

The implementation of this method is very much subject to change.

`get_digests_by_file_and_commit()`

Purpose

Present the same outcomes as run_test_files_on_all_commits(), but formatted by target file, then commit.
Arguments
```
$rv = $self->get_digests_by_file_and_commit();
```
None; all data needed is already present in the object.

Return Value

Reference to a hash keyed on the basename of the target file, modified to substitute underscores for forward slashes and dots. The value of each element in the hash is a reference to an array which, in turn, holds a list of hash references, one per git commit. Each such hash has the following keys:

commit
file
md5_hex

Example:

{
  t_44_func_hashes_mult_unsorted_t   => [
      {
        commit  => "2a2e54af709f17cc6186b42840549c46478b6467",
        file    => "/tmp/Xhilc8ZSgS/2a2e54a.t_44_func_hashes_mult_unsorted_t.output.txt",
        md5_hex => "31b7c93474e15a16d702da31989ab565",
      },
      {
        commit  => "a624024294a56964eca53ec4617a58a138e91568",
        file    => "/tmp/Xhilc8ZSgS/a624024.t_44_func_hashes_mult_unsorted_t.output.txt",
        md5_hex => "31b7c93474e15a16d702da31989ab565",
      },
      # ...
      {
        commit  => "d304a207329e6bd7e62354df4f561d9a7ce1c8c2",
        file    => "/tmp/Xhilc8ZSgS/d304a20.t_44_func_hashes_mult_unsorted_t.output.txt",
        md5_hex => "31b7c93474e15a16d702da31989ab565",
      },
  ],
  t_45_func_hashes_alt_dual_sorted_t => [
      {
        commit  => "2a2e54af709f17cc6186b42840549c46478b6467",
        file    => "/tmp/Xhilc8ZSgS/2a2e54a.t_45_func_hashes_alt_dual_sorted_t.output.txt",
        md5_hex => "6ee767b9d2838e4bbe83be0749b841c1",
      },
      {
        commit  => "a624024294a56964eca53ec4617a58a138e91568",
        file    => "/tmp/Xhilc8ZSgS/a624024.t_45_func_hashes_alt_dual_sorted_t.output.txt",
        md5_hex => "6ee767b9d2838e4bbe83be0749b841c1",
      },
      # ...
      {
        commit  => "d304a207329e6bd7e62354df4f561d9a7ce1c8c2",
        file    => "/tmp/Xhilc8ZSgS/d304a20.t_45_func_hashes_alt_dual_sorted_t.output.txt",
        md5_hex => "6ee767b9d2838e4bbe83be0749b841c1",
      },
  ],
}

Comment

This method currently presumes that you have called run_test_files_on_all_commits(). It will die otherwise.

`examine_transitions()`

Purpose

Determine whether a run of the same targeted test file run at two consecutive commits produced the same or different output (as measured by string equality or inequality of each commit's md5_hex value.

Arguments

$hashref = $self->get_digests_by_file_and_commit();

$transitions = $self->examine_transitions($hashref);

Hash reference returned by get_digests_by_file_and_commit();

Return Value

Reference to a hash keyed on the basename of the target file, modified to substitute underscores for forward slashes and dots. The value of each element in the hash is a reference to an array which, in turn, holds a list of hash references, one per each pair of consecutive git commits. Each such hash has the following keys:

older
newer
compare

The value for each of the older and newer elements is a reference to a hash with two elements:

md5_hex
idx

... where md5_hex is the digest of the test output file and idx is the position (count starting at 0) of that element in the list of commits in the commit range.

Example:

{
  t_44_func_hashes_mult_unsorted_t   => [
      {
        compare => "same",
        newer   => { md5_hex => "31b7c93474e15a16d702da31989ab565", idx => 1 },
        older   => { md5_hex => "31b7c93474e15a16d702da31989ab565", idx => 0 },
      },
      {
        compare => "same",
        newer   => { md5_hex => "31b7c93474e15a16d702da31989ab565", idx => 2 },
        older   => { md5_hex => "31b7c93474e15a16d702da31989ab565", idx => 1 },
      },
      # ...
      {
        compare => "same",
        newer   => { md5_hex => "31b7c93474e15a16d702da31989ab565", idx => 9 },
        older   => { md5_hex => "31b7c93474e15a16d702da31989ab565", idx => 8 },
      },
  ],
  t_45_func_hashes_alt_dual_sorted_t => [
      {
        compare => "same",
        newer   => { md5_hex => "6ee767b9d2838e4bbe83be0749b841c1", idx => 1 },
        older   => { md5_hex => "6ee767b9d2838e4bbe83be0749b841c1", idx => 0 },
      },
      {
        compare => "same",
        newer   => { md5_hex => "6ee767b9d2838e4bbe83be0749b841c1", idx => 2 },
        older   => { md5_hex => "6ee767b9d2838e4bbe83be0749b841c1", idx => 1 },
      },
      {
        compare => "same",
        newer   => { md5_hex => "6ee767b9d2838e4bbe83be0749b841c1", idx => 3 },
        older   => { md5_hex => "6ee767b9d2838e4bbe83be0749b841c1", idx => 2 },
      },
      # ...
      {
        compare => "same",
        newer   => { md5_hex => "6ee767b9d2838e4bbe83be0749b841c1", idx => 9 },
        older   => { md5_hex => "6ee767b9d2838e4bbe83be0749b841c1", idx => 8 },
      },
  ],
}

Comment

This method currently may be called only after calling run_test_files_on_all_commits() and will die otherwise.

Since in this method we are concerned with the transition in the test output between a pair of commits, the second-level arrays returned by this method will have one fewer element than the second-level arrays returned by get_digests_by_file_and_commit().

To install Devel::Git::MultiBisect, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Devel::Git::MultiBisect

CPAN shell

perl -MCPAN -e shell
install Devel::Git::MultiBisect

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

SYNOPSIS

DESCRIPTION

METHODS

run_test_files_on_all_commits()

get_digests_by_file_and_commit()

examine_transitions()

Module Install Instructions

`run_test_files_on_all_commits()`

`get_digests_by_file_and_commit()`

`examine_transitions()`