/ 
Devel-Git-MultiBisect-0.06
River stage zero No dependents
/ Devel::Git::MultiBisect::Transitions

NAME

Devel::Git::MultiBisect::Transitions - Gather test output where it changes over a range of git commits

SYNOPSIS

use Devel::Git::MultiBisect::Transitions;

$self = Devel::Git::MultiBisect::Transitions->new(\%parameters);

$commit_range = $self->get_commits_range();

$full_targets = $self->set_targets(\@target_args);

$self->multisect_all_targets();

$multisected_outputs = $self->get_multisected_outputs();

$transitions = $self->inspect_transitions();
}

DESCRIPTION

Given a Perl library or application kept in git for version control, it is often useful to be able to compare the output collected from running one or several test files over a range of git commits. If that range is sufficiently large, a test may fail in more than one way over that range.

If that is the case, then simply asking, "When did this file start to fail?" is insufficient. We may want to capture the test output for each commit, or, more usefully, may want to capture the test output only at those commits where the output changed.

Devel::Git::MultiBisect provides methods to achieve that objective. More specifically:

  • When the number of commits in the specified range is large and you only need the test output at those commits where the output materially changed, you can use this package, Devel::Git::MultiBisect::Transitions.

  • When you want to capture the test output for each commit in a specified range, you can use another package in this library, Devel::Git::MultiBisect::AllCommits.

METHODS

This package inherits methods from Devel::Git::MultiBisect. Only methods unique to Devel::Git::MultiBisect::AllCommits are documented here. See the documentation for Devel::Git::MultiBisect for all other methods, including:

new()
get_commits_range()
set_targets()

multisect_all_targets()

  • Purpose

    For selected files within an application's test suite, determine the points within a specified range of git commits where the output of a run of each test materially changes. Store the test output at those transition points for human inspection.

  • Arguments

    $self->multisect_all_targets();

    None; all data needed is already present in the object.

  • Return Value

    Returns true value upon success.

  • Comment

    As multisect_all_targets() runs it does two kinds of things:

    • It stores results data within the object which you can subsequently access through method calls.

    • It captures each test output and writes it to a file on disk for later human inspection.

get_multisected_outputs()

  • Purpose

    Get results of multisect_all_targets() (other than test output files created) reported on a per target/per commit basis.

  • Arguments

    my $multisected_outputs = $self->get_multisected_outputs();

    None; all data needed is already present in the object.

  • Return Value

    Reference to a hash with one element for each targeted test file.

    Each element's key is a "stub" version of the target's relative path below the git checkout directory in which forward slashes and dot characters have been replaced with underscores. So,

    t/44_func_hashes_mult_unsorted.t

    ... becomes:

    t_44_func_hashes_mult_unsorted_t

    Each element's value is a reference to an array with one element for each commit in the commit range.

    • If a particular commit was not visited in the course of multisect_all_targets(), then the array element is undefined. (The point of multisection, of course, is to not have to visit every commit in the commit range in order to figure out the commits at which test output changed.)

    • If a particular commit was visited in the course of multisect_all_targets(), then the array element is a hash reference whose elements have the following keys:

      commit
commit_short
file
md5_hex

    Example:

    {
  t_001_load_t => [
      {
        commit => "d2bd2c75a2fd9afd3ac65a808eea2886d0e41d01",
        commit_short => "d2bd2c7",
        file => "/tmp/LHEG4uXfj1/d2bd2c7.t_001_load_t.output.txt",
        md5_hex => "318ce8b2ccb3e92a6e516e18d1481066",
      },
      undef,
      {
        commit => "f2bc0ec377776b42928a29cebe04954975a30eb2",
        commit_short => "f2bc0ec",
        file => "/tmp/LHEG4uXfj1/f2bc0ec.t_001_load_t.output.txt",
        md5_hex => "318ce8b2ccb3e92a6e516e18d1481066",
      },
      # ...
      },
      {
        commit => "199494ee204dd78ed69490f9e54115b0e83e7d39",
        commit_short => "199494e",
        file => "/tmp/LHEG4uXfj1/199494e.t_001_load_t.output.txt",
        md5_hex => "d7125615b2e5dbb4750ff107bbc1bad3",
      },
    ],
  t_002_add_t  => [
      {
        commit => "d2bd2c75a2fd9afd3ac65a808eea2886d0e41d01",
        commit_short => "d2bd2c7",
        file => "/tmp/LHEG4uXfj1/d2bd2c7.t_002_add_t.output.txt",
        md5_hex => "0823e5d7628802e5a489661090109c56",
      },
      undef,
      {
        commit => "f2bc0ec377776b42928a29cebe04954975a30eb2",
        commit_short => "f2bc0ec",
        file => "/tmp/LHEG4uXfj1/f2bc0ec.t_002_add_t.output.txt",
        md5_hex => "0823e5d7628802e5a489661090109c56",
      },
      # ...
      {
        commit => "199494ee204dd78ed69490f9e54115b0e83e7d39",
        commit_short => "199494e",
        file => "/tmp/LHEG4uXfj1/199494e.t_002_add_t.output.txt",
        md5_hex => "7716009f1af9a562a3edad9e2af7dedc",
      },
    ],
}

inspect_transitions()

  • Purpose

    Get a data structure which reports on the most meaningful results of multisect_all_targets(), namely, the first commit, the last commit and all transitional commits.

  • Arguments

    my $transitions = $self->inspect_transitions();

    None; all data needed is already present in the object.

  • Return Value

    Reference to a hash with one element per target. Each element's key is a "stub" version of the target's relative path below the git checkout directory. (See example in documentation for get_multisected_outputs above.)

    Each element's value is another hash reference. The elements of that hash will have the following keys:

    • oldest

      Value is reference to hash keyed on idx, md5_hex and file, whose values are, respectively, the index position of the very first commit in the commit range, the digest of that commit's test output and the path to the file holding that output.

    • newest

      Value is reference to hash keyed on idx, md5_hex and file, whose values are, respectively, the index position of the very last commit in the commit range, the digest of that commit's test output and the path to the file holding that output.

    • transitions

      Value is reference to an array with one element for each transitional commit. Each such element is a reference to a hash with keys older and newer. In this context older refers to the last commit in a sub-sequence with a particular digest; newer refers to the next immediate commit which is the first commit in a new sub-sequence with a new digest.

      The values of older and newer are, in turn, references to hashes with keys idx, md5_hex and file. Their values are, respectively, the index position of the particular commit in the commit range, the digest of that commit's test output and the path to the file holding that output.

    Example:

    {
  t_001_load_t => {
      newest => {
        file => "/tmp/IvD3Zwn3FJ/199494e.t_001_load_t.output.txt",
        idx => 13,
        md5_hex => "d7125615b2e5dbb4750ff107bbc1bad3",
      },
      oldest => {
        file => "/tmp/IvD3Zwn3FJ/d2bd2c7.t_001_load_t.output.txt",
        idx => 0,
        md5_hex => "318ce8b2ccb3e92a6e516e18d1481066",
      },
      transitions => [
        {
          newer => {
                     file => "/tmp/IvD3Zwn3FJ/1debd8a.t_001_load_t.output.txt",
                     idx => 5,
                     md5_hex => "e5a839ea2e34b8976000c78c258299b0",
                   },
          older => {
                     file => "/tmp/IvD3Zwn3FJ/707da97.t_001_load_t.output.txt",
                     idx => 4,
                     md5_hex => "318ce8b2ccb3e92a6e516e18d1481066",
                   },
        },
        {
          newer => {
                     file => "/tmp/IvD3Zwn3FJ/6653d84.t_001_load_t.output.txt",
                     idx => 8,
                     md5_hex => "f4920ddfdd9f1e6fc21ebfab09b5fcfe",
                   },
          older => {
                     file => "/tmp/IvD3Zwn3FJ/b35b4d7.t_001_load_t.output.txt",
                     idx => 7,
                     md5_hex => "e5a839ea2e34b8976000c78c258299b0",
                   },
        },
        {
          newer => {
                     file => "/tmp/IvD3Zwn3FJ/aa1ed28.t_001_load_t.output.txt",
                     idx => 12,
                     md5_hex => "d7125615b2e5dbb4750ff107bbc1bad3",
                   },
          older => {
                     file => "/tmp/IvD3Zwn3FJ/65bf77c.t_001_load_t.output.txt",
                     idx => 11,
                     md5_hex => "f4920ddfdd9f1e6fc21ebfab09b5fcfe",
                   },
        },
      ],
  },
  t_002_add_t  => {
      newest => {
        file => "/tmp/IvD3Zwn3FJ/199494e.t_002_add_t.output.txt",
        idx => 13,
        md5_hex => "7716009f1af9a562a3edad9e2af7dedc",
      },
      oldest => {
        file => "/tmp/IvD3Zwn3FJ/d2bd2c7.t_002_add_t.output.txt",
        idx => 0,
        md5_hex => "0823e5d7628802e5a489661090109c56",
      },
      transitions => [
        {
          newer => {
                     file => "/tmp/IvD3Zwn3FJ/646fd8a.t_002_add_t.output.txt",
                     idx => 3,
                     md5_hex => "dbd8c7a70877b3c8d3fd93a7a66d8468",
                   },
          older => {
                     file => "/tmp/IvD3Zwn3FJ/f2bc0ec.t_002_add_t.output.txt",
                     idx => 2,
                     md5_hex => "0823e5d7628802e5a489661090109c56",
                   },
        },
        {
          newer => {
                     file => "/tmp/IvD3Zwn3FJ/b35b4d7.t_002_add_t.output.txt",
                     idx => 7,
                     md5_hex => "50aac31686ac930aad7fdd23df679f28",
                   },
          older => {
                     file => "/tmp/IvD3Zwn3FJ/55ab1f9.t_002_add_t.output.txt",
                     idx => 6,
                     md5_hex => "dbd8c7a70877b3c8d3fd93a7a66d8468",
                   },
        },
        {
          newer => {
                     file => "/tmp/IvD3Zwn3FJ/6653d84.t_002_add_t.output.txt",
                     idx => 8,
                     md5_hex => "256f466d35533555dce93a838ba5ab9d",
                   },
          older => {
                     file => "/tmp/IvD3Zwn3FJ/b35b4d7.t_002_add_t.output.txt",
                     idx => 7,
                     md5_hex => "50aac31686ac930aad7fdd23df679f28",
                   },
        },
        {
          newer => {
                     file => "/tmp/IvD3Zwn3FJ/abc336e.t_002_add_t.output.txt",
                     idx => 9,
                     md5_hex => "037be971470cb5d96a7a7f9764a6f3aa",
                   },
          older => {
                     file => "/tmp/IvD3Zwn3FJ/6653d84.t_002_add_t.output.txt",
                     idx => 8,
                     md5_hex => "256f466d35533555dce93a838ba5ab9d",
                   },
        },
        {
          newer => {
                     file => "/tmp/IvD3Zwn3FJ/65bf77c.t_002_add_t.output.txt",
                     idx => 11,
                     md5_hex => "7716009f1af9a562a3edad9e2af7dedc",
                   },
          older => {
                     file => "/tmp/IvD3Zwn3FJ/bbe25f4.t_002_add_t.output.txt",
                     idx => 10,
                     md5_hex => "037be971470cb5d96a7a7f9764a6f3aa",
                   },
        },
      ],
  },
}

  • Comment

    The return value of inspect_transitions() should be useful to the developer trying to determine the various points in a long series of commits where a target's test output changed in meaningful ways. Hence, it is really the whole point of Devel::Git::MultiBisect::Transitions.