NAME
Carmel - CPAN Artifact Repository Manager
SYNOPSIS
# Run with a directory with cpanfile
carmel install
# list all the modules to be loaded
carmel list
# list all the modules in a tree
carmel tree
# show a location where a module is installed
carmel show Plack
# update Plack to the latest
carmel update Plack
# update all the modules in the snapshot
carmel update
# pin modules tp specific versions
carmel update DBI@1.633 Plack@1.0000
# show diffs for cpanfile.snapshot in a nice way
carmel diff
# Runs your perl script with modules from artifacts
carmel exec perl ...
# Requires all your modules in cpanfile in one shot
carmel exec perl -e 'use Carmel::Preload;'
# Roll out the currently selected modules into ./local
carmel rollout
# package modules tarballs and index into ./vendor/cache
carmel package
# use Carmel packages inside a script (without carmel exec)
perl -e 'use Carmel::Setup; ...'
# prints export PATH=... etc for shell scripting
carmel export
# find a module in a repository
carmel find DBI
# find a module matching the version query
carmel find Plack ">= 1.0000, < 1.1000"
DESCRIPTION
Carmel is yet another CPAN module manager.
Unlike traditional CPAN module installer, Carmel keeps the build of your dependencies in a central repository, then select the library paths to include upon runtime in development.
Carmel also allows you to rollout all the files in a traditional perl INC directory structure, which is useful to use in a production environment, such as containers.
WORKFLOW
Development
During the development, run carmel install
when you setup a new environment, and any time you make changes to cpanfile
. This will update your build artifacts, and saves the changes to cpanfile.snapshot
. Commit the snapshot file in version control system so that you can reproduce the exact same versions across machines.
carmel exec
makes it easy to run your application using the versions in cpanfile
and cpanfile.snapshot
dynamically.
# On your development environment
> cat cpanfile
requires 'Plack', '0.9980';
requires 'Starman', '0.2000';
> carmel install
> echo /.carmel >> .gitignore
> git add cpanfile cpanfile.snapshot .gitignore
> git commit -m "add Plack and Starman"
# On a new setup, or another developer's machine
> git pull
> carmel install
> carmel exec starman -p 8080 myapp.psgi
# Add a new dependency
> echo "requires 'Try::Tiny';" >> cpanfile
> carmel install
> git commit -am 'Add Try::Tiny'
# Update Plack to the latest
> carmel update Plack
Production Deployments
Carmel allows you to manage all the dependencies the same way across development environments and production environments. However, there might be cases where you want to avoid running your application with carmel exec
in production, to avoid the overhead with large number of include paths, or to avoid installing Carmel in the production hosts. Carmel provides two easy ways to avoid depending on Carmel on the deploy target environments.
carmel rollout
carmel rollout
rolls out the build artifacts into a regular perl5 library path in local
. Once the rollout is complete, you can include the path just like a regular local::lib directory.
# Production environment: Roll out to ./local
> carmel rollout
> perl -Ilocal/lib/perl5 local/bin/starman -p 8080 myapp.psgi
You can run carmel rollout
in a CI system to create the local
directory next to your application code for a linux package (e.g. deb package), or Docker containers.
carmel package
carmel package
(similar to carton bundle
) creates a directory with tarballs and CPAN-style package index files, which you can pass to cpanm on a target machine. This way, you only need cpanm
, which is available as a self-contained single executable, to bootstrap the installation on a host with a stock perl.
# Vendor all the packages to vendor/cache
> carmel package
> git add vendor/cache
> git commit -m 'Vendor all the tarballs'
# Remote environment (CI etc.)
> git clone https://.../myapp.git && cd myapp
# Install the modules to ./local (like carmel rollout)
> cpanm -L ./local --from $PWD/vendor/cache -nq --installdeps .
HOW IT WORKS
Carmel will keep the build directory (artifacts) after a cpanm installation in a repository, which defaults to $HOME/.carmel/{version}-{archname}/builds
, and your directory structure would look like:
$HOME/.carmel/5.20.1-darwin-2level/builds
Plack-1.0033/
blib/
arch/
lib/
URI-1.64/
blib/
arch/
lib/
URI-1.63/
blib/
arch/
lib/
Carmel scans this directory and creates the mapping of which version of any package belongs to which build directory.
Given the list of modules and requirements from cpanfile
, carmel install
computes which versions satisfy the requirements best, and if there isn't, installs the modules from CPAN to put it to the artifact repository. The computed mappings are preserved as a snapshot in cpanfile.snapshot
.
Once the snapshot is created, each following carmel
command runs uses both cpanfile
and cpanfile.snapshot
to determine the best versions to satisfy the requirements. When you update cpanfile
to bump a version or add a new module, carmel
will install the new dependencies and update the snapshot accordingly.
carmel exec
command, like install
command, lists the build directories and .pm
files you need from the repository, and then prepend the mappings of these files in the @INC
hook. This is a handy way to run a perl program using the dependencies pinned by Carmel, without changing any include path.
carmel update
command allows you to selectively update a dependency while preserving other dependencies in the snapshot. carmel update Plack
for example pulls the latest version of Plack from CPAN (and its dependencies, if it needs a newer version than pinned in the snapshot), and updates the snapshot properly. Running carmel update
without any arguments would update all the modules in cpanfile
, including its dependencies.
On a production environment, you might want to use the carmel rollout
command, which saves all the files included in the cpanfile
, pinned with cpanfile.snapshot
, to the local
directory. This directory can be included like a regular perl's library path, with PERL5LIB=/path/to/local/lib/perl5
, or with use lib
, and you don't need to use carmel
command in production this way.
SNAPSHOT SUPPORT
As of v0.1.29, Carmel supports saving and loading snapshot file in cpanfile.snapshot
, in a compatible format with Carton. Versions saved in the snapshot file will be preserved across multiple runs of Carmel across machines, so that versions frozen in one environment can be committed to a source code repository, and can be reproduced in another box, so long as the perl version and architecture is the same.
CAVEATS / KNOWN ISSUES
If you run multiple instances of
carmel
, or hit Ctrl-C to interrupt the cpanm install session, Carmel might get into a state where some modules have been installed properly, while some modules in the dependency chain are missing. Make sure you don't run multiple instances ofcarmel
at the same time, and let it finish the installation to get the full builds properly.There're certain dependencies that get missed during the initial
carmel install
, and you'll see the error message "Can't find an artifact for Foo".Please report it to the issue tracker if you can reliably reproduce this type of errors.
Usually you run
carmel install
again and the error will be gone.In some situation, you might encounter conflicts in version resolutions, between what's pinned in the snapshot and a new version that's needed when you introduce a new module.
For example, suppose you have:
# cpanfile requires 'Foo'; requires 'Bar'; # which requires Foo >= 1.001
Without the snapshot, Carmel has no trouble resolving the correct versions for this combination. But if you have:
# cpanfile.snapshot Foo-1.000
The first time you run
carmel install
, Carmel will try to install Foo-1.000, because that's the version pinned in the snapshot, while trying to pull the module Bar, which would conflict with that version of Foo.This can happen 50% of the time, because if cpanm (called internally by Carmel) installs Bar first, then the resolution is done correctly and the version in the snapshot would be skipped, and the snapshot will be updated accordingly. This is due to perl's hash randomization after Perl 5.18.
To avoid this, you're recommended to run
carmel install
before making any changes to cpanfile. That will put the build caches to satisfy what's in cpanfile and the snapshot. After that, adding a new dependency will likely reuse what's in the build cache, while adding a new dependency can update the transient dependency (for Foo) without having conflicts.If you encounter conflicts like this, you can work around it by:
Run
carmel update Foo
to pull the latest version of Foo from CPAN, ignoring what's in the snapshot.Update
cpanfile
to explicitly update the version requirement forFoo
.
COMPARISONS WITH SIMILAR TOOLS
Carton
Carmel shares the same goal with Carton, where you can manage your dependencies by declaring them in cpanfile
, and pinning them in cpanfile.snapshot
. Most of the commands work the same way, so Carmel can most effectively be a drop-in replacement for Carton, if you're currently using it.
Here's a few key differences between Carmel and Carton:
Carton does not manage what's currently being installed in
local
directory. It just runscpanm
command with-L local
, with a hope that nothing has changed the directory except Carton, and whatever is in the directory won't conflict with the snapshot file. This can easily conflict whencpanfile.snapshot
is updated by multiple developers or when you continuously update the dependencies across multiple machines.Carmel manages all the dependencies for your project in the Carmel repository under
$HOME/.carmel
, and nothing is installed under your project directory on development. Thelocal
directory is only created when you request it viacarmel rollout
command, and it's safe to run multiple times. Runningcarmel install
after pulling the changes to the snapshot file will always install the correct dependencies from the snapshot file, as compared to Carton, which doesn't honor the snapshot on a regular install command, if whatever version inlocal
already satisfies the version incpanfile
.Carton has no easy way to undo a change once you update a version of a module in
local
, because which version is actually selected is only preserved as a file inside the directory, that's not managed by Carton. To undo a change you have to remove the entirelocal
directory to start over.Carmel preserves this information to the
cpanfile.snapshot
file, and every invocation of Carmel resolves the dependencies declared incpanfile
and pinned incpanfile.snapshot
dynamically, to create a stable dependency tree, without relying on anything in a directory under your project other than the snapshot file. Undoing the change incpanfile.snapshot
file immediately reverts the change.
cpm
App::cpm is an excellent standalone CPAN installer.
Like Carton, cpm installs the dependencies declared in
cpanfile
tolocal
. Carmel installs them into a build cache, and doesn't uselocal
directory for state management. You can runcarmel rollout
to copy the dependencies tolocal
directory.cpm installs the modules in parallel, which makes the installation very fast. Like Carmel, cpm also manages its build artifacts cache, so a module that has previously been installed would be very fast to install, since there's no build process.
Unlike Carton and Carmel, cpm doesn't have the ability to manage
cpanfile.snapshot
file on its own. It can read the snapshot however, so it's possible to use Carmel in a development environment, and then usecpm install
instead ofcarmel install
andcarmel rollout
, if all you need is to build out a perl5 library path out ofcpanfile
andcpanfile.snapshot
in the source code repository.
COMMUNITY
- https://github.com/miyagawa/Carmel
-
Code repository, Wiki and Issue Tracker
AUTHOR
Tatsuhiko Miyagawa <miyagawa@bulknews.net>
COPYRIGHT
Copyright 2015- Tatsuhiko Miyagawa
LICENSE
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.