NAME
Pinto::Manual::Tutorial - A narrative introduction to Pinto
VERSION
version 0.048
INTRODUCTION
This tutorial walks you through some of the typical use cases for a Pinto repository. Along the way, it demonstrates most of the pinto commands. You are encouraged to try the commands as you read along. If you would prefer to get a more condensed summary of features and commands, please read the Pinto::Manual::QuickStart
PREREQUISITES
To perform the operations described in this tutorial, you'll need to install App::Pinto (which provides the pinto command line application) and Pinto (which provides the back-end libraries for managing local repositories). Both of these ship separately.
You can also perform (almost) all of these operations remotely using Pinto::Remote. But setting up a remote repository is outside the scope of this tutorial. Please see Pinto::Server and pintod for more information on that.
BASIC OPERATIONS
Creating a repository
The first step in using Pinto is to create a repository, using the init command like this:
$> pinto -r ~/repos initThis will create a new repository in the ~/repos directory. If that directory does not exist, it will be created for you. If it already does exist, then it must be empty.
The -r (or --root) option specifies where the repository is. This argument is required for every pinto command. But if you get tired of typing it, you can set the PINTO_REPOSITORY_ROOT environment variable to point to your repository instead.
Inspecting the repository
Now that you have a repository, let's look inside it. To see the contents of a repository, use the list command:
$> pinto -r ~/repos listYou will use the list command quite often. But at this point, the listing will be empty because there is nothing in the repository. So let's go ahead and add something...
Adding dependencies
Suppose we are working on an application called My-App that contains a package called My::App. The application also depends on the URI package. Using the pull command, you can bring the URI package into your repository:
$> pinto -r ~/repos pull URINow, you should have URI in your local repository. So lets look and see what we really got. Once again, you use the list command to see inside the repository:
$> pinto -r ~/repos listThis time, the listing will look something like this:
rf URI 1.60 GAAS/URI-1.60.tar.gz
rf URI::Escape 3.31 GAAS/URI-1.60.tar.gz
rf URI::Heuristic 4.20 GAAS/URI-1.60.tar.gz
...You can see that the URI package has been added to the repository, as well as all the other packages that were in that distribution. You can repeat this process for each package that your application depends on. In fact, you can pull all your prerequisites into the repository at once...
$> pinto -r ~/repos pull URI LWP Catalyst Plack ...Or if you're using Dist::Zilla to build your application, you can ask it to provide the list of prerequisites...
# From the My-App dist directory...
$> dzil listdeps | pinto -r ~/repos pullIf any of those prerequisites have prerequisites of their own, then they will be recursively pulled as well, until the repository contains all the distributions required to build My-App from scratch.
Adding your own distributions
Now suppose that you've finished work on My-App and your ready to release the first version. Using your preferred build tool (ExtUtils::MakeMaker, Module::Build, Module::Install etc.) you package a release as My-App-1.0.tar.gz. Now put the distribution into the repository with the add command:
$> pinto -r ~/repos add path/to/My-App-1.0.tar.gzWhen you list the repository contents now, it will include the My::App package and show you as the author of the distribution:
rl My::App 1.0 JEFF/My-App-1.0.tar.gz
rf URI 1.60 GAAS/URI-1.60.tar.gz
rf URI::Escape 3.31 GAAS/URI-1.60.tar.gz
rf URI::Heuristic 4.20 GAAS/URI-1.60.tar.gz
...Installing packages
Now the repository contains both your application and all of its prerequisites, so you can install it using the install command (this requires cpanm 1.5013 or newer):
$> pinto -r ~/repos install My::AppWhen My::App is installed, it will only use the prerequisites that are in your repository. Even if a newer version of URI is released to the CPAN in the future, My::App will always be built with the same versions of the same prerequisites that you developed and tested against. This ensures your application builds will be stable and predictable.
Upgrading a dependency
Suppose that several weeks have passed since you first released My-App and now URI version 1.62 is available on the CPAN. It has some bug critical fixes that you'd like to get. Again, we can bring that into the repository using the pull command. But since your repository already contains a version of URI, you must indicate that you want a *newer* one by specifying the minimum version that you want:
$> pinto -r ~/repos pull URI~1.62If you look at the listing again, this time you'll see the newer version of URI (and possibly other packages as well):
rl My::App 1.0 JEFF/My-App-1.0.tar.gz
rf URI 1.62 GAAS/URI-1.62.tar.gz
rf URI::Escape 3.38 GAAS/URI-1.62.tar.gz
rf URI::Heuristic 4.20 GAAS/URI-1.62.tar.gz
...If the new version of URI requires any new prerequisites, those will be in the repository too. Now when you install My::App, you'll get version 1.62 of URI.
WORKING WITH STACKS
So far in this tutorial, we've treated the repository as a singular resource. For example, when we upgraded URI in the last section, it impacted every person and every application that might have been using the repository. But this kind of broad impact is undesirable. You would prefer to make those kinds of changes in isolation and test them before forcing everyone else to upgrade. This is what stacks are designed for.
What is a stack
All CPAN-like repositories have an index which maps the latest version of each package to the archive that contains it. Usually, there is only one such index per repository. But with Pinto, there can be many indexes. Each of these indexes is called a "stack". This allows you to create different stacks of dependencies within a single repository. So you could have a development stack and a production stack. Whenever you add a distribution or upgrade a prerequisite, it only affects one stack.
The default stack
Before getting into the gory details, you first need to know about the default stack. For most operations, the name of the stack is an optional parameter. So if you do not specify a stack explicitly, then the operation is applied to whichever stack is marked as the default.
In any repository, there is exactly one default stack. When you create a repository, it comes with a stack called init and this stack is marked as the default. You can also change the default stack, but we won't go into that here. See the edit command to learn more about that.
Just remember that init is the name of the built-in stack that comes with a new Pinto repository.
Creating a stack
Suppose your repository contains version 1.60 of URI, but version 1.62 has been released to the CPAN, just like in the earlier section. You want to try upgrading, but this time you're going to do it on a separate stack.
Thus far, everything you've added or pulled into the repository has gone onto the init stack. You could create an entirely new stack, but the init stack already has the prerequisites for My-App, so we're just going to make a clone using the copy command:
$> pinto -r ~/repos copy init uri_upgradeThis creates a new stack called uri_upgrade (stack names must be alphanumeric and are forced to lowercase). If you want to see the contents of that stack, just use the list command with the --stack option:
$> pinto -r ~/repos list --stack uri_upgradeThe listing should be identical to the init stack:
rl My::App 1.0 JEFF/My-App-1.0.tar.gz
rf URI 1.60 GAAS/URI-1.60.tar.gz
...Upgrading a stack
Now that you've got a separate stack, you can try upgrading URI. Just as before, you'll use the pull command, but this time, you'll tell Pinto that you want the package to be pulled onto the uri_upgrade stack:
$> pinto -r ~/repos pull --stack uri_upgrade URI~1.62Now look at the repository contents again. You'll use the list command, but this time you'll use the special stack name "%" to indicate that we want to look at all the stacks. In addition, you'll use the --packages option to limit the listing to just the packages that you are interested in:
$> pinto -r ~/repos/ list --stack % --packages URIYou'll see that the init stack still contains version 1.60 of URI, but the uri_upgarde stack now contains version 1.62:
init: rf URI 1.60 GAAS/URI-1.60.tar.gz
uri_upgrade: rf URI 1.62 GAAS/URI-1.62.tar.gzInstalling from a stack
With URI upgraded on the uri_upgrade stack, you can now try building and testing our application. All you have to do is run the install command and point to to the right stack:
$> pinto -r ~/repos install --stack uri_upgrade My::AppThis will build My::App using only the prerequisites that are on the uri_upgrade stack. If the tests pass, then you can confidently upgrade URI on the init stack as well.
USING PINS
In the last section, we used a stack to experiment with upgrading a dependency. Fortunately, all the tests passed. But what if the tests didn't pass? If the problem lies within My-App and you can quickly correct it, you might just modify your code, release version 2.0 of My-App, and then proceed to upgrade URI on the init stack.
But if the issue is a bug in URI or it will take a long time to fix My-App, then you have a real problem. You don't want someone else to upgrade URI, nor do you want it to be upgraded inadvertently to satisfy some other prerequisite that My-App may have. Until the bug is fixed (in either URI or My-App) you need to prevent URI from being upgraded. This is what pins are for.
Pinning a package
When you pin a package, that version of the package is forced to stay in a stack. Any attempt to upgrade it (either directly or via another prerequisite) will fail. To pin a package, use the pin command like this:
$> pinto -r ~/repos pin URIIf you look at the listing for the init stack again, you'll see something like this:
...
rl My::App 1.0 JEFF/My-App-1.0.tar.gz
rf+ URI 1.60 GAAS/URI-1.60.tar.gz
rf+ URI::Escape 3.31 GAAS/URI-1.60.tar.gz
...The "+" near the beginning of the line indicates the package has been pinned. Notice every package in the URI-1.60.tar.gz distribution has been pinned, so it is impossible to partially upgrade a distribution (this situation could happen when a package moves into a different distribution).
Unpinning a packages
After a while, suppose you fix the problem in My-App or a new version of URI is released that fixes the bug. When that happens, you can unpin URI from the stack using the unpin command:
$> pinto -r ~/repos unpin URIAt this point you're free to upgrade URI to the latest version whenever you're ready. Just as with pinning, when you unpin a package, it unpins every other package it that distribution as well.
USING PINS AND STACKS TOGETHER
Pins and stacks are used together to help manage change during the development cycle. For example, you could create a stack called prod that contains your known-good dependencies. Likewise, you could create a stack called dev that contains experimental dependencies for your next release. Initially, the dev stack is just a copy of the prod stack.
As development proceeds, you may upgrade or add several packages on the dev stack. If an upgraded package breaks your application, then you'll place a pin in that package on the prod stack to signal that it shouldn't be upgraded.
As you reach the end of the development cycle, you can merge all the changes on the dev stack back to the prod stack using the merge command like this:
$> pinto -r ~/repos merge dev prodIf the prod stack contains pins that conflict with the packages in the dev stack, then the merge is aborted. If the tests are all passing when you build from the dev stack, then you can probably go ahead and unpin the packages on the prod stack and merge again. But if not, then you continue development until the tests pass and then merge.
Alternatively, you could downgrade the appropriate packages on the dev stack to match prod, but this is only allowed if something on the stack doesn't require the newer version. And of course, you should verify that your tests will still pass with the older version too.
Pins and Patches
Sometimes you may find that a new version of a CPAN distribution has a bug but the author is unable or unwilling to fix it (at least not before your next release is due). In that situation, you may elect to make a local patch of the CPAN distribution.
So suppose that you forked the code for URI and made a local version of the distribution called URI-1.60_PATCHED.tar.gz. You can add it to your repository using the add command:
$> pinto -r ~/repos add path/to/URI-1.60_PATCHED.tar.gzIn this situation, it is wise to pin the package as well, since you do not want it to be upgraded until you are sure that the new release includes your patch or the author has fixed the bug by other means.
$> pinto -r ~/repos pin URIWhen the author of URI releases version 1.62 with your patch, you'll want to try it before deciding to unpin from your locally patched version. Just as before, this can be done by cloning the stack with the copy command. Let's call it the "trial" stack this time:
$> pinto -r ~/repos copy init trialBut before you can upgrade URI on the trial stack, you'll have to unpin it there:
$> pinto -r ~/repos unpin --stack trial URINow you can proceed to upgrade URI on the stack and try building My::App like this:
$> pinto -r ~/repos pull --stack trial URI~1.62
$> pinto -r ~/repos install --stack trial My::AppIf all goes well, remove the pin from the init stack and pull your latest version of URI back to it.
$> pinto -r ~/repos unpin URI
$> pinto -r ~/repos pull URI~1.62Rolling Back Changes
In a perfect world, your unit tests immediately tell you when an upgraded dependency has broken your application. But the world is not perfect, so you may not learn that your application is broken until long after you upgraded the stack and released the app. In those situations, you can use the revert command to restore the stack a prior state.
Suppose that we've discovered that version 1.62 of URI has broken MyApp. If the problem lies within our code and we can fix it quickly, then we could just put a new release into the repository. But if the problem lies within URI, or one of its dependencies, or we just don't know where the problem is, then we can roll the stack back to the previous state
$> pinto -r ~/repos revertIf we look at the listing now, we see that the stack has gone back to the locally patched version of URI that we added just before upgrading to version 1.62 of URI.
...
rl My::App 1.0 JEFF/My-App-1.0.tar.gz
rf URI 1.60 JEFF/URI-1.60_PATCHED.tar.gz
rf URI::Escape 3.31 JEFF/URI-1.60_PATCHED.tar.gz
...Each pinto command that changes the state of a stack (e.g. add, pull, merge, new, copy, revert) constitutes one revision on that stack. You can then restore the stack to the state at any prior revision. Use the log command to review the changes that have occurred on a stack and identify the revision that you want to revert to.
$> pinto -r ~/repos log -dThat should display something like this:
---------------------------------------------------------------------------
init@4 | jeff | Thu Sep 20 11:12:18 GMT 2012
URI-1.60 is broken, so replacing it with our patched version
D GAAS/URI-1.60.tar.gz/URI~1.60
A JEFF/URI-1.60_PATCHED.tar.gz/URI~1.60
D GAAS/URI-1.60.tar.gz/URI::Escape~1.60
A JEFF/URI-1.60_PATCHED.tar.gz/URI::Escape~1.60
...The header for each log record shows the stack name, revision number, the user who performed the revision, and the message that accompanied it. Below the header, you can see exactly which packages were deleted (D) and added (A) to the stack.
WORKING WITH LEGACY CODE
In the My-App example, there was only one dependency. But any non-trivial legacy application will already have many dependencies. And by the time you create a Pinto repository for them, the CPAN may already have newer versions of those dependencies. Or in some cases, the version you've been using may only be available on the BackPAN.
For large legacy projects, I recommend using Dist::Surveyor to discover which versions of which dependencies your application has been using thus far. Dist::Surveyor will examine your run-time environment and produce a list of the precise distributions that you'll need. You can then bring all those distributions into your repository with the pull command:
$> pinto -r ~/repos pull --norecurse < LIST_OF_DISTSNotice the --norecurse option here. This time, you don't want Pinto to recursively pull prerequisites. Instead, you just want it to fetch the specific distributions that are in your list. If the list is complete, the repository will end up with everything that is needed to build your environment exactly the way it is. If the list is not complete, then you'll need to hunt down the missing distributions and add them to the repository manually (perhaps you stashed them in your VCS), or settle for upgrading to the current versions on the CPAN.
PINTO AND THE DEVELOPMENT CYCLE
In the My-App example, we started out with an application that already depended on URI. But in practice, we usually don't know what our application will depend on in advance. Often times, we shop around and try out several dependencies before settling on the ones we really want.
By the time you reach a decision, you may have installed several different things in your development environment. You usually know what *your* application depends on, but you can easily loose track of all the prerequisites for those dependencies. If this process takes several days or weeks, there is a risk that the latest packages in the CPAN may drift from your development environment. Then you'll have to re-discover your dependencies and add them to the repository, and that is a hassle.
Fortunately, you can use the install command to help build your development environment and populate the repository at the same time. Suppose you keep your own development environment in ~/perl5 (perhaps using local::lib or perlbrew. When you want to try a new web framework, you can ask Pinto to install it into your local development environment, like this:
$> pinto -r ~/repos install Dancer -l ~/perl5 --pullThe -l option is passed to cpanm and directs it to install packages into your ~/perl5 directory instead of site_perl. The --pull option also tells Pinto to pull any prerequisites that it needs (including Dancer itself) if they aren't already in the repository.
In this manner, you can organically grow your development environment. And when you finally decide which packages you really want, they will already be in the repository.
CONCLUSION
In this tutorial, you've seen the basic pinto commands for pulling dependencies into the repository, and adding your own distributions to the repository. You've also seen how to use stacks and pins to manage your dependencies in the face of some common development obstacles.
Each command has several options that were not discussed in this tutorial, and there are some commands that were not mentioned here at all. So you are encouraged to explore the manual pages for each command and learn more.
AUTHOR
Jeffrey Ryan Thalhammer <jeff@imaginative-software.com>
COPYRIGHT AND LICENSE
This software is copyright (c) 2011 by Imaginative Software Systems.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.