/ 
DBIx-QuickDB-0.000035
River stage zero No dependents
/ DBIx::QuickDB::Pool

NAME

DBIx::QuickDB::Pool - Define a pool of databases to clone on demand.

DESCRIPTION

This library lets you define a pool of databases to clone on demand. This lets you incrementally build databases starting with a driver. Each database you build can either be based of a previous one, or started from scratch.

This tool lets you buid multiple nested clean database trees. You can then clone instances off these clean ones as needed.

This library can spin up clean copies of databases at any states you define, and it can do so FAST by cloning data directories instead of building everything back up from nothing.

This library will build databases on demand, no cost for databases you define but never use.

This library will rebuild databases when schema files or installed database version change.

This library can be used by concurrent processes that may try to build/consume the same databases from the same cache dir.

DECLARATIVE SYNOPSIS

YOUR POOL LIBRARY

package My::Pool;
use strict;
use warnings;

use DBIx::QuickDB::Pool cache_dir => "$ENV{HOME}/databases";

# This will define a clean database called 'PostgreSQL' that we can always
# clone, or build on top of.
driver 'PostgreSQL';

build schema => (
    # This one will be built on top of a clone of the 'PostgreSQL' clean
    # db.
    from => 'PostgreSQL',

    # You must provide a way to invalidate the current, usually this would
    # return a sha1 of the schema files or similar. If you do not want it
    # to detect changes just have the subroutine return a string such as
    # "never changes".
    checksum => sub { ...; return "something" },

    # Any database you define, apart from drivers, needs a build callback
    build => sub {
        my $class = shift; # The current package
        my ($db) = @_;

        # $db will already be started for you, and will be stopped as soon
        # as this sub returns.

        @ Load the schema
        $db->load_sql(myapp => "path/to/myapp_schema.sql");
    },
);

build scenario_foo => (
    from => 'schema',
    checksum => sub { ... },
    build => sub {
        my $class = shift;
        my ($db) = @_;

        ... Load data for scenario 'foo' ...
    },
);

build scenario_bar => (
    from => 'schema',
    checksum => sub { ... },
    build => sub {
        my $class = shift;
        my ($db) = @_;

        ... Load data for scenario 'bar' ...
    },
);

build scenario_foo_and_bar => (
    from => 'foo',
    checksum => sub { ... },
    build => sub {
        my $class = shift;
        my ($db) = @_;

        ... Load data for scenario 'bar' ...
    },
);

1;

POOL CONSUMERS

# This will import db() from the My::Pool library.
use Importer 'My::Pool' => 'db';

# This will return a new clone of foo, changing it will not effect the
# original 'foo' built by the library.
my $clone_of_foo = db('foo');

# Same for bar
my $clone_of_bar = db('bar');

# This gets a NEW clone of 'foo'. Changes in this db will NOT effect
# $clone_of_foo.
my $other_clone_of_foo = db('foo');

... Change Schema ...

# This will rebuild schema, then rebuild foo using the new schema.
# (If update_checksums has not been turned off)
my $fresh_foo = db('foo');

EXPORTS

$pool = QDB_POOL()

This will return the instance of DBIx::QuickDB::Pool associated with your package.

driver $DRIVER
driver $DRIVER => (%SPEC)

Define a new driver you can use as a basis for your database states.

$DRIVER can be shorthand 'PostgreSQL' or it can be fully qualified if you prefix it with a '+' '+DBIx::QuickDB::Driver::PostgreSQL'.

The name will be the last part of the driver package name Example: 'PostgreSQL' unless you override it.

The following specifications are allowed:

name => $NAME

You can use this to override the default database name.

checksum => sub { ... }
checksum => "method_name"

You can use this to override the default checksum calculator drivers use. Normally the default will return a sha1 of the version information of the locally installed database tools.

This can be a subref, or the name of a method defined on your package.

build => sub { ... }
build => "method_name"

This is not necessary on a driver database, however if you want to do anything to the clean database before anything builds off of it you can. If you only have 1 schema to load you can pop it in here instead of creating a schema specific database. Keep in mind you will have to manage checksum calculation for both, and this will have to be rebuilt for both schema changes and db tool version changes.

This can be a subref, or the name of a method defined on your package.

driver_args => \%ARGS

You can pass in a hashref of arguments to pass into the driver when initializing the database:

driver_args => { verbose => 0, autostart => 0, autostop => 1 },
clone_args => \%ARGS

Same as driver_args, except these are used when cloning a database. This will be inherited by databases that are built off of this one.

build $NAME => (from => $PARENT_OR_DRIVER, build => \&BUILDER, checksum => \&CHECKSUM)
build $NAME => (from => $PARENT_OR_DRIVER, build => $BUILDER_METHOD_NAME, checksum => $CHECKSUM_METHOD_NAME)
build $NAME => (%SPEC)

Define a new database state with the given $NAME.

The following specifications are available:

from => $PARENT_OR_DRIVER

This specifies the parent database or driver to build off of.

build => sub { ... }
build => $BUILDER_METHOD_NAME

Any database that is not a base driver needs to do build some kind of state to be useful. An example is loading schema, or loading fixture data. This is where you do that.

sub {
    my $class = shift;
    my ($db) = @_;

    # $db will already be started for you
    $db->load_sql(myapp => "path/to/myapp_schema.sql");

    # $db will be stopped for you automatically.
}
checksum => sub { ... }
checksum => $CHECKSUM_METHOD_NAME

This must return a string. If the data this database is built from will never change you can return a constant string. If the data can change you should probably either return a version string, or a sha1 of the data.

This is used to check if a database needs to be rebuilt due to external changes, such as a schema.sql file being modified.

clone_args => \%ARGS

You can pass in a hashref of arguments to pass into the driver when cloning the database:

clone_args => { verbose => 0, autostart => 0, autostop => 1 },

This will be inherited by databases that are built off of this one. This will also override any that may have been inherited from a parent.

$db = db($NAME)

Fetch a fresh clone of the specified database. This will be an isolated copy that you can play with. Neither the original nor any other copy will be effected by anything you do. When you are done simply disgard the copy.

If the database, or ant of its parents have not been built yet, they will be built before you get your fresh copy. The first time this is called may be slow, but future calls will use cached data making them very fast.

@EXPORT_OK

db() is added to your packages @EXPORT_OK variable on import. This allows other modules to import the method in order to get clones of the databases you defined.

use Importer 'My::Pool' => 'db';

my $db = db('foo');

OO SYNOPSIS

use DBIx::Class::Pool();

my $pool = DBIx::Class::Pool->new(
    cache_dir => "$ENV{HOME}/databases",
);

# This will define a clean database called 'PostgreSQL' that we can always
# clone, or build on top of.
$pool->add_driver('PostgreSQL');

$pool->add_db(
    'schema',

    # This one will be built on top of a clone of the 'PostgreSQL' clean
    # db.
    from => 'PostgreSQL',

    # You must provide a way to invalidate the current, usually this would
    # return a sha1 of the schema files or similar. If you do not want it
    # to detect changes just have the subroutine return a string such as
    # "never changes".
    checksum => sub { ...; return "something" },

    # Any database you define, apart from drivers, needs a build callback
    build => sub {
        my $class = shift; # The current package
        my ($db) = @_;

        # $db will already be started for you, and will be stopped as soon
        # as this sub returns.

        @ Load the schema
        $db->load_sql(myapp => "path/to/myapp_schema.sql");
    },
);

$pool->add_db(
    'scenario_foo',
    from => 'schema',
    checksum => sub { ... },
    build => sub {
        my $class = shift;
        my ($db) = @_;

        ... Load data for scenario 'foo' ...
    },
);

$pool->add_db(
    'scenario_bar',
    from => 'schema',
    checksum => sub { ... },
    build => sub {
        my $class = shift;
        my ($db) = @_;

        ... Load data for scenario 'bar' ...
    },
);

$pool->add_db(
    'scenario_foo_and_bar',
    from => 'foo',
    checksum => sub { ... },
    build => sub {
        my $class = shift;
        my ($db) = @_;

        ... Load data for scenario 'bar' ...
    },
);

And to then use the databases:

# This will return a new clone of foo, changing it will not effect the
# original 'foo' built by the library.
my $clone_of_foo = $pool->fetch_db('foo');

# Same for bar
my $clone_of_bar = $pool->fetch_db('bar');

# This gets a NEW clone of 'foo'. Changes in this db will NOT effect
# $clone_of_foo.
my $other_clone_of_foo = $pool->fetch_db('foo');

... Change Schema ...

# This will rebuild schema, then rebuild foo using the new schema.
# (If update_checksums has not been turned off)
my $fresh_foo = $pool->db('foo');

ATTRIBUTES

cache_dir => "path/to/cache"

Required.

Can only be specified at import or construction.

No accessors.

instance_dir => "path/to/instances"

Normally db's are spun up in the system temp dir. This allows you to provide an alternate temporary database location.

library => $PACKAGE
$pkg = $pool->library

Set automatically from caller during construction unless specified.

Can be read, but not modified.

update_checksums => $BOOL
$bool = $pool->update_checksums()
$pool->set_update_checksums($bool)

Defaults to true.

Can be set during construction, or altered at any time.

When true checksums will be recalculated every time a database is requested, if any checksum has changed since the last time they were built then all db downstream of the changed checksum will be rebuilt to account for the changes.

Most of the time you want this to be on so that databases are rebuilt if schema changes or a new version of the drivers are installed. However if you are not worried about changes, or checksum calculation is expensive for your pool you can turn this off.

NOTE: even when this is turned on, no exisitng/active databases will be rebuilt. To get changes you need to close connections tot he db, stop it, and request it again via db($NAME) or $pool->fetch_db($NAME) to get an updated build.

purge_old => $BOOL
$bool = $pool->purge_old()
$pool->set_purge_old($bool)

Defaults to false.

Can be set during construction or changed at any time.

When true old builds will be deleted from cache whenever they expire.

NOTE: THIS IS NOT RECOMMENDED when multiple processes share a cache dir, such as during concurrent unit testing.

verbose => $POSITIVE_INTEGER
$POSITIVE_INTEGER = $pool->verbose()
$pool->set_verbose($POSITIVE_INTEGER)

Defaults to 0.

Can be set during construction or changed at any time.

When set to 1 or greater diagnostics messages about what the pool is doing will be printed. In addition database command output will be displayed unless you have overriden the verbose parameter in the driver_args or clone_args settings.

When set to 2 or greater the diagnostic messages will be sent to STDERR instead of STDOUT.

When set to 3 or greater you will also see the output of the copy commands that clone the database data directories.

METHODS

INTERFACE

$pool->add_driver($DRIVER)
$pool->add_driver($DRIVER, %SPEC)

Define a new driver you can use as a basis for your database states.

$DRIVER can be shorthand 'PostgreSQL' or it can be fully qualified if you prefix it with a '+' '+DBIx::QuickDB::Driver::PostgreSQL'.

The name will be the last part of the driver package name Example: 'PostgreSQL' unless you override it.

The following specifications are allowed:

name => $NAME

You can use this to override the default database name.

checksum => sub { ... }
checksum => "method_name"

You can use this to override the default checksum calculator drivers use. Normally the default will return a sha1 of the version information of the locally installed database tools.

This can be a subref, or the name of a method defined on your package.

build => sub { ... }
build => "method_name"

This is not necessary on a driver database, however if you want to do anything to the clean database before anything builds off of it you can. If you only have 1 schema to load you can pop it in here instead of creating a schema specific database. Keep in mind you will have to manage checksum calculation for both, and this will have to be rebuilt for both schema changes and db tool version changes.

This can be a subref, or the name of a method defined on your package.

driver_args => \%ARGS

You can pass in a hashref of arguments to pass into the driver when initializing the database:

driver_args => { verbose => 0, autostart => 0, autostop => 1 },
clone_args => \%ARGS

Same as driver_args, except these are used when cloning a database. This will be inherited by databases that are built off of this one.

$pool->add_db(from => $PARENT_OR_DRIVER, build => \&BUILDER)
$pool->add_db(from => $PARENT_OR_DRIVER, build => $BUILDER_METHOD_NAME)
$pool->add_db(%SPEC)

Define a new database state with the given $NAME.

The following specifications are available:

from => $PARENT_OR_DRIVER

This specifies the parent database or driver to build off of.

build => sub { ... }
build => $BUILDER_METHOD_NAME

Any database that is not a base driver needs to do build some kind of state to be useful. An example is loading schema, or loading fixture data. This is where you do that.

sub {
    my $class = shift;
    my ($db) = @_;

    # $db will already be started for you
    $db->load_sql(myapp => "path/to/myapp_schema.sql");

    # $db will be stopped for you automatically.
}
checksum => sub { ... }
checksum => $CHECKSUM_METHOD_NAME

This must return a string. If the data this database is built from will never change you can return a constant string. If the data can change you should probably either return a version string, or a sha1 of the data.

This is used to check if a database needs to be rebuilt due to external changes, such as a schema.sql file being modified.

clone_args => \%ARGS

You can pass in a hashref of arguments to pass into the driver when cloning the database:

clone_args => { verbose => 0, autostart => 0, autostop => 1 },

This will be inherited by databases that are built off of this one. This will also override any that may have been inherited from a parent.

$db = $pool->fetch_db($NAME)

Fetch a fresh clone of the specified database. This will be an isolated copy that you can play with. Neither the original nor any other copy will be effected by anything you do. When you are done simply disgard the copy.

If the database, or ant of its parents have not been built yet, they will be built before you get your fresh copy. The first time this is called may be slow, but future calls will use cached data making them very fast.

$pool->clear_old_cache($age_in_seconds);

This will check all database directories in the cashe dir to see when they were last cloned, if the last clone was at or before the specified age then the dir will be deleted.

INTERNAL

Listed for completeness, but you should not use these, except maybe in a subclass.

DB BUILDING

$pool->build_db()
$pool->build_via_clone()
$pool->build_via_driver()
$pool->vivify_db()
$pool->reclaim()

CHECKSUM/CACHE VALIDATION

$pool->cache_check()
$pool->driver_checksum()

IPC

$pool->lock()
$pool->unlock()

DIAGNOSTICS

$pool->throw()
$pool->alert()
$pool->diag()

MISC

$pool->export()

SOURCE

The source code repository for DBIx-QuickDB can be found at https://github.com/exodist/DBIx-QuickDB/.

MAINTAINERS

Chad Granum <exodist@cpan.org>

AUTHORS

Chad Granum <exodist@cpan.org>

COPYRIGHT

Copyright 2020 Chad Granum <exodist7@gmail.com>.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://dev.perl.org/licenses/