/ 
Dancer2-Plugin-MarkdownFilesToHTML-0.017
River stage zero No dependents
⭐ Starred 0 GitHub stars
/ t/data/markdown_files/dzil_tutorial/tutorial09.md

Testing Tutorial Part I: [@Starter] Bundle Tests

Right up there with use strict, use warnings and "eat your vegetables" is the admonition to "test your code!" We'll spare nagging you with all the excellent reasons why you should write tests for your code. But if you've been avoiding testing, this is a good opportunity to start doing it.

The testing tutorial is split into three parts. The first is a basic overview of testing and provides details about [@Starter] bundle test plugins. The second is more hands-on and demonstrates how to add basic tests and run them with Dist::Zilla. The third introduces useful test plugins.

If you aren't familiar with tests, don't worry. This tutorial is designed for beginners. Just remember that a test file is nothing but a perl script that runs functions to determine if your module works the way you intend. Each function in the script returns a simple pass or fail value. After these tutorials, you'll hopefully have the confidence to write simple tests for your code and/or get motivated to learn more about testing from other very fine learning resources.

If you're already comfortable writing tests, you'll learn about the dzil test command, how Dist::Zilla integrates tests with a distribution, and maybe some new testing tools and strategies you're not using presently.

Tests Supplied with the [@Starter] Bundle

The [@Starter] bundle includes plugins that generate two different types of tests for us: standard tests and author tests.

Standard Tests

Files containing standard tests are placed in the distribution's t ("t" for "tests") directory. During module development, developers frequently run standard tests to ensure new features work as expected and old code still works. Standard tests should also be executed immediately before shipping the distribution to end users.

End user's should run standard tests before installing modules to ensure that the module still works properly despite having different hardware and software than the developer. As much as possible, these tests are automated and require no end user intervention. However, end users can skip them if they want to.

Author Tests

Author tests are not run by end users but by developers, often in conjunction with standard tests. The [@Starter] bundle plugins adds author tests to the xt/author ("xt" for "extra tests") directory. Like standard tests, author tests should be run just before releasing your code to the world.

Running Tests Manually with the [@Starter] Bundle

Developers can manually initiate tests with:

dzil test

Normally, dzil test runs only standard tests. The [@Starter] bundle's [RunExtraTests] plugin modifies dzil test to run author tests as well. We'll cover other kinds of tests and how to run them in the next tutorial.

Behind the scenes, the test subcommand triggers the construction of a temporary build of your distribution. Tests are run on this build using the Makefile.PL build script generated by the build. If all tests pass, the build files are removed. Otherwise, a copy of them are left behind in a hidden .build directory for your inspection, though, in practice, you will rarely need to.

Examining Our Test Output

If you executed the previous tutorial tasks faithfully, dzil test will reward you with a clean test and an All's well message at the very end:


[DZ] building distribution under .build/VHTxZpnZvW for installation
[DZ] beginning to build App-sayhi
[DZ] guessing dist's main_module is lib/App/sayhi.pm
[DZ] writing App-sayhi in .build/VHTxZpnZvW
[ReadmeAnyFromPod] overriding README.md in root
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for App::sayhi
Writing MYMETA.yml and MYMETA.json
cp lib/App/sayhi.pm blib/lib/App/sayhi.pm
cp bin/sayhi blib/script/sayhi
"/usr/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/sayhi
PERL_DL_NONLAZY=1 PERL_USE_UNSAFE_INC=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00-report-prereqs.t .. #
# Versions for all modules listed in MYMETA.json (including optional ones):
#
# === Configure Requires ===
#
#     Module              Want    Have
#     ------------------- ---- -------
#     ExtUtils::MakeMaker  any 7.10_02
#
# === Build Requires ===
#
#     Module              Want    Have
#     ------------------- ---- -------
#     ExtUtils::MakeMaker  any 7.10_02
#
# === Test Requires ===
#
#     Module              Want     Have
#     ------------------- ---- --------
#     ExtUtils::MakeMaker  any  7.10_02
#     File::Spec           any  3.63_01
#     Test::More           any 1.302140
#
# === Test Recommends ===
#
#     Module         Want     Have
#     ---------- -------- --------
#     CPAN::Meta 2.120900 2.150005
#
# === Develop Requires ===
#
#     Module     Want     Have
#     ---------- ---- --------
#     File::Spec  any  3.63_01
#     IO::Handle  any     1.36
#     IPC::Open3  any     1.20
#     Test::More  any 1.302140
#     Test::Pod  1.41     1.52
#
t/00-report-prereqs.t .. ok
All tests successful.
Files=1, Tests=1,  1 wallclock secs ( 0.01 usr  0.00 sys +  0.10 cusr  0.01 csys =  0.12 CPU)
Result: PASS
xt/author/00-compile.t .. ok
xt/author/pod-syntax.t .. ok
All tests successful.
Files=2, Tests=5,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.16 cusr  0.01 csys =  0.20 CPU)
Result: PASS
[DZ] all's well; removing .build/VHTxZpnZvW

If you see errors, review this tutorial's previous instructions to make sure your code is correct and that the Greetings module is installed. If that looks OK, check that you ran dzil test from the top level of the the work area (source tree) and not from inside the distribution (build tree). Having a misconfigured Perlbrew installation or some other complication regarding how modules are installed on your machine might also result in errors. Whatever the cause, try to resolve the issues before proceeding.

The output generated by dzil test can be confusing and takes practice deciphering. It's a mix of information from a chain of tools that work together to run the tests located in the t and xt/author directories in the disribution:

  1. Our old friend Dist::Zilla which...
  2. builds the distribution end executes its Makefile.PL build script which...
  3. loads the tests into the Test::Harness module which...
  4. executes the module tests which may generate their own output.

You'll want to get good at finding the output from Test::Harness which tells you which tests have passed and which have failed. In the output above, look for the 'ok' messages and the three lines near the end, beginning with the All tests successful. If you don't see that success message, you'll want to review all error messages that will jump out at you as you gain more experience parsing test output.

The Prerequisites Standard Test Output

Most of the test output–the stuff in the middle in a tabular layout-is a report generated by [Test::ReportPrereqs], listing the software requirements for installing, testing, configuring and running your module. Together, these are known as prerequisites. The report can be helpful to CPAN testers, developers, and end users who know what they are looking at. Again, the most important part is the small bit from Test::Harness:

t/00-report-prereqs.t .. ok

If you look at sayhi-0.001/t/00-report-prereqs.t you'll see a lot of hairy, scary code but don't let that worry you. Most tests are much, much simpler than this. Accompanying the 00-report-prereqs.t file in the t directory is 00-report-prereqs.dd which is not a test but a helper file to the test and you don't have to concern yourself with how that works either.

The [@Starter] Bundle Author Test Output

The [@Starter] bundle generates two author test files, both found in the xt/author distribution directory. Test::Harness output reports the tests in these file passed:


xt/author/00-compile.t .. ok
xt/author/pod-syntax.t .. ok
Files=2, Tests=5,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.15 cusr  0.02 csys =  0.20 CPU)
Result: PASS

The first of these test files is generated by the [Test::Compile] plugin and the second by the [PodSyntaxTests].

You may recall a while back we mentioned that bundles can modify the behavior of plugins. The [Test::Compile] plugin is an example of that with the [@Starter] bundle passing an xt_mode = 1 parameter to the plugin, telling Dist::Zilla to treat it like an author test instad of a standard test.

Let's look at what these tests do to make your job as a developer easier.

The Compile Test

The tests in 00-compile.t ensure that our perl code won't throw any errors when compiled by the perl interpreter. In other words, it acts as a syntax checker and will report code that fails to compile.

The POD Syntax Test

The tests performed by pod-syntax.t will inspect your pod documenation and report any syntax issues with it. We'll see an example of a failed pod syntax test in the next tutorial.

Automated Testing with the [TestRelease] Plugin

The [TestRelease] plugin, supplied by [@Starter], runs all the standard, author, and release tests when you issue a dzil release command to try to stop you from introducing crappy code into the world. If a test doesn't pass, it will alert you and stop the release process. More on the release subcommand soon.

This completes our look at the tests and test plugins the [@Starter] bundle provides. Now it's time to don a pair of coveralls and mess with some tests.