Name

App::Sqitch::Engine - Sqitch Deployment Engine

Synopsis

my $engine = App::Sqitch::Engine->new( sqitch => $sqitch );

Description

App::Sqitch::Engine provides the base class for all Sqitch storage engines. Most likely this will not be of much interest to you unless you are hacking on the engine code.

Interface

Class Methods

key

my $name = App::Sqitch::Engine->key;

The key name of the engine. Should be the last part of the package name.

name

my $name = App::Sqitch::Engine->name;

The name of the engine. Returns the same value as key by default, but should probably be overridden to return a display name for the engine.

default_registry

my $reg = App::Sqitch::Engine->default_registry;

Returns the name of the default registry for the engine. Most engines just inherit the default value, sqitch, but some must do more munging, such as specifying a file name, to determine the default registry name.

default_client

my $cli = App::Sqitch::Engine->default_client;

Returns the name of the default client for the engine. Must be implemented by each engine.

driver

my $driver = App::Sqitch::Engine->driver;

The name and version of the database driver to use with the engine, returned as a string suitable for passing to use. Used internally by use_driver() to use the driver and, if it dies, to display an appropriate error message. Must be overridden by subclasses.

use_driver

App::Sqitch::Engine->use_driver;

Uses the driver and version returned by driver. Returns an error on failure and returns true on success.

config_vars

my %vars = App::Sqitch::Engine->config_vars;

Returns a hash of names and types to use for configuration variables for the engine. These can be set under the engine.$engine_name section in any configuration file.

The keys in the returned hash are the names of the variables. The values are the data types. Valid data types include:

any
int
num
bool
bool-or-int

Values ending in + (a plus sign) may be specified multiple times. Example:

(
    client => 'any',
    host   => 'any',
    port   => 'int',
    set    => 'any+',
)

In this example, the port variable will be stored and retrieved as an integer. The set variable may be of any type and may be included multiple times. All the other variables may be of any type.

By default, App::Sqitch::Engine returns:

(
    target   => 'any',
    registry => 'any',
    client   => 'any',
)

Subclasses for supported engines will return more.

registry_release

Returns the version of the registry understood by this release of Sqitch. The needs_upgrade() method compares this value to that returned by registry_version() to determine whether the target's registry needs upgrading.

default_lock_timeout

Returns 60, the default value for the lock_timeout attribute.

Constructors

load

my $cmd = App::Sqitch::Engine->load(%params);

A factory method for instantiating Sqitch engines. It loads the subclass for the specified engine and calls new, passing the Sqitch object. Supported parameters are:

sqitch

The App::Sqitch object driving the whole thing.

new

my $engine = App::Sqitch::Engine->new(%params);

Instantiates and returns a App::Sqitch::Engine object.

Instance Accessors

sqitch

The current Sqitch object.

target

An App::Sqitch::Target object identifying the database target, usually derived from the name of target specified on the command-line, or the default.

uri

A URI::db object representing the target database. Defaults to a URI constructed from the App::Sqitch db_* attributes.

destination

A string identifying the target database. Usually the same as the target, unless it's a URI with the password included, in which case it returns the value of uri with the password removed.

registry

The name of the registry schema or database.

start_at

The point in the plan from which to start deploying changes.

log_only

Boolean indicating whether or not to log changes without running deploy or revert scripts. This is useful for an existing database schema that needs to be converted to Sqitch. False by default.

with_verify

Boolean indicating whether or not to run the verification script after each deploy script. False by default.

variables

A hash of engine client variables to be set. May be set and retrieved as a list.

Instance Methods

username

my $username = $engine->username;

The username to use to connect to the database, for engines that require authentication. The username is looked up in the following places, returning the first to have a value:

lock_timeout

Number of seconds to lock_destination() to wait to acquire a lock before timing out. Defaults to 60.

  1. The $SQITCH_USERNAME environment variable.

  2. The username from the target URI.

  3. An engine-specific default password, which may be derived from an environment variable, engine configuration file, the system user, or none at all.

See sqitch-authentication for details and best practices for Sqitch engine authentication.

password

my $password = $engine->password;

The password to use to connect to the database, for engines that require authentication. The password is looked up in the following places, returning the first to have a value:

  1. The $SQITCH_PASSWORD environment variable.

  2. The password from the target URI.

  3. An engine-specific default password, which may be derived from an environment variable, engine configuration file, or none at all.

See sqitch-authentication for details and best practices for Sqitch engine authentication.

registry_destination

my $registry_destination = $engine->registry_destination;

Returns the name of the registry database. In other words, the database in which Sqitch's own data is stored. It will usually be the same as target(), but some engines, such as SQLite, may use a separate database. Used internally to name the target when the registration tables are created.

variables

set_variables

clear_variables

my %vars = $engine->variables;
$engine->set_variables(foo => 'bar', baz => 'hi there');
$engine->clear_variables;

Get, set, and clear engine variables. Variables are defined as key/value pairs to be passed to the engine client in calls to deploy and revert, if the client supports variables. For example, the PostgreSQL and Vertica engines pass all the variables to their psql and vsql clients via the --set option, while the MySQL engine engine sets them via the SET command and the Oracle engine engine sets them via the SQL*Plus DEFINE command.

deploy

$engine->deploy($to_change);
$engine->deploy($to_change, $mode);
$engine->deploy($to_change, $mode);

Deploys changes to the target database, starting with the current deployment state, and continuing to $to_change. $to_change must be a valid change specification as passable to the index_of() method of App::Sqitch::Plan. If $to_change is not specified, all changes will be applied.

The second argument specifies the reversion mode in the case of deployment failure. The allowed values are:

all

In the event of failure, revert all deployed changes, back to the point at which deployment started. This is the default.

tag

In the event of failure, revert all deployed changes to the last successfully-applied tag. If no tags were applied during this deployment, all changes will be reverted to the pint at which deployment began.

change

In the event of failure, no changes will be reverted. This is on the assumption that a change failure is total, and the change may be applied again.

Note that, in the event of failure, if a reversion fails, the target database may be left in a corrupted state. Write your revert scripts carefully!

revert

$engine->revert;
$engine->revert($tag);
$engine->revert($tag);

Reverts the App::Sqitch::Plan::Tag from the database, including all of its associated changes.

Note that this method does not respect the $cmd.strict configuration variables, which therefore must be checked by the caller.

verify

$engine->verify;
$engine->verify( $from );
$engine->verify( $from, $to );
$engine->verify( undef, $to );

Verifies the database against the plan. Pass in change identifiers, as described in sqitchchanges, to limit the changes to verify. For each change, information will be emitted if:

  • It does not appear in the plan.

  • It has not been deployed to the database.

  • It has been deployed out-of-order relative to the plan.

  • Its verify script fails.

Changes without verify scripts will emit a warning, but not constitute a failure. If there are any failures, an exception will be thrown once all verifications have completed.

check

$engine->check;
$engine->check( $from );
$engine->check( $from, $to );
$engine->check( undef, $to );

Compares the state of the working directory and the database by comparing the SHA1 hashes of the deploy scripts. Fails and reports divergence for all changes with non-matching hashes, indicating that the project deploy scripts differ from the scripts that were used to deploy to the database.

Pass in change identifiers, as described in sqitchchanges, to limit the changes to check. For each change, information will be emitted if the SHA1 digest of the current deploy script does not match its SHA1 digest at the time of deployment.

check_deploy_dependencies

$engine->check_deploy_dependencies;
$engine->check_deploy_dependencies($to_index);

Validates that all dependencies will be met for all changes to be deployed, starting with the currently-deployed change up to the specified index, or to the last change in the plan if no index is passed. If any of the changes to be deployed would conflict with previously-deployed changes or are missing any required changes, an exception will be thrown. Used internally by deploy() to ensure that dependencies will be satisfied before deploying any changes.

check_revert_dependencies

$engine->check_revert_dependencies(@changes);

Validates that the list of changes to be reverted, which should be passed in the order in which they will be reverted, are not depended upon by other changes. If any are depended upon by other changes, an exception will be thrown listing the changes that cannot be reverted and what changes depend on them. Used internally by revert() to ensure no dependencies will be violated before revering any changes.

deploy_change

$engine->deploy_change($change);
$engine->deploy_change($change);

Used internally by deploy() to deploy an individual change.

revert_change

$engine->revert_change($change);
$engine->revert_change($change);

Used internally by revert() (and, by deploy() when a deploy fails) to revert an individual change.

verify_change

$engine->verify_change($change);

Used internally by deploy_change() to verify a just-deployed change if with_verify is true.

is_deployed

say "Tag deployed"  if $engine->is_deployed($tag);
say "Change deployed" if $engine->is_deployed($change);

Convenience method that dispatches to is_deployed_tag() or is_deployed_change() as appropriate to its argument.

earliest_change

my $change = $engine->earliest_change;
my $change = $engine->earliest_change($offset);

Returns the App::Sqitch::Plan::Change object representing the earliest applied change. With the optional $offset argument, the returned change will be the offset number of changes following the earliest change.

latest_change

my $change = $engine->latest_change;
my $change = $engine->latest_change($offset);

Returns the App::Sqitch::Plan::Change object representing the latest applied change. With the optional $offset argument, the returned change will be the offset number of changes before the latest change.

change_for_key

my $change = if $engine->change_for_key($key);

Searches the deployed changes for a change corresponding to the specified key, which should be in a format as described in sqitchchanges. Throws an exception if the key matches more than one changes. Returns undef if it matches no changes.

change_id_for_key

my $change_id = if $engine->change_id_for_key($key);

Searches the deployed changes for a change corresponding to the specified key, which should be in a format as described in sqitchchanges, and returns the change's ID. Throws an exception if the key matches more than one change. Returns undef if it matches no changes.

change_for_key

my $change = if $engine->change_for_key($key);

Searches the list of deployed changes for a change corresponding to the specified key, which should be in a format as described in sqitchchanges. Throws an exception if the key matches multiple changes.

change_id_for_depend

say 'Dependency satisfied' if $engine->change_id_for_depend($depend);

Returns the change ID for a dependency, if the dependency resolves to a change currently deployed to the database. Returns undef if the dependency resolves to no currently-deployed change.

find_change

my $change = $engine->find_change(%params);

Finds and returns a deployed change, or undef if the change has not been deployed. The supported parameters are:

change_id

The change ID.

change

A change name.

tag

A tag name.

project

A project name. Defaults to the current project.

offset

The number of changes offset from the change found by the other parameters should actually be returned. May be positive or negative.

The order of precedence for the search is:

  1. Search by change ID, if passed.

  2. Search by change name as of tag, if both are passed.

  3. Search by change name or tag.

The offset, if passed, will be applied relative to whatever change is found by the above algorithm.

find_change_id

my $change_id = $engine->find_change_id(%params);

Like find_change(), taking the same parameters, but returning an ID instead of a change.

run_deploy

$engine->run_deploy($deploy_file);

Runs a deploy script. The implementation is just an alias for run_file(); subclasses may override as appropriate.

run_revert

$engine->run_revert($revert_file);

Runs a revert script. The implementation is just an alias for run_file(); subclasses may override as appropriate.

run_verify

$engine->run_verify($verify_file);

Runs a verify script. The implementation is just an alias for run_file(); subclasses may override as appropriate.

run_upgrade

$engine->run_upgrade($upgrade_file);

Runs an upgrade script. The implementation is just an alias for run_file(); subclasses may override as appropriate.

needs_upgrade

if ($engine->needs_upgrade) {
    $engine->upgrade_registry;
}

Determines if the target's registry needs upgrading and returns true if it does.

upgrade_registry

$engine->upgrade_registry;

Upgrades the target's registry, if it needs upgrading. Used by the upgrade command.

lock_destination

$engine->lock_destination;

This method is called before deploying or reverting changes. It attempts to acquire a lock in the destination database to ensure that no other instances of Sqitch can act on the database at the same time. If it fails to acquire the lock, it emits a message to that effect, then tries again and waits. If it acquires the lock, it continues its work. If the attempt times out after lock_timeout seconds, it exits with an error.

The default implementation is effectively a no-op; consult the documentation for specific engines to determine whether they have implemented support for destination locking (by overriding try_lock() and wait_lock()).

initialized

$engine->initialize unless $engine->initialized;

Returns true if the database has been initialized for Sqitch, and false if it has not.

initialize

$engine->initialize;

Initializes the target database for Sqitch by installing the Sqitch registry schema and/or tables. Should be overridden by subclasses. This implementation throws an exception

Abstract Instance Methods

These methods must be overridden in subclasses.

try_lock

$engine->try_lock;

This method is called by lock_destination, and this default implementation simply returns true. May be overridden in subclasses to acquire a database lock that would prevent any other instance of Sqitch from making changes at the same time. If it cannot acquire the lock, it should immediately return false without waiting.

wait_lock

$engine->wait_lock;

This method is called by lock_destination when try_lock returns false. It must be implemented if try_lock is overridden; otherwise it throws an error when try_lock returns false. It should attempt to acquire the same lock as try_lock, but wait for it and time out after lock_timeout seconds.

begin_work

$engine->begin_work($change);

This method is called just before a change is deployed or reverted. It should create a lock to prevent any other processes from making changes to the database, to be freed in finish_work or rollback_work. Unlike lock_destination, this method generally starts a transaction for the duration of the deployment or reversion of a single change.

finish_work

$engine->finish_work($change);

This method is called after a change has been deployed or reverted. It should unlock the lock created by begin_work.

rollback_work

$engine->rollback_work($change);

This method is called after a change has been deployed or reverted and the logging of that change has failed. It should rollback changes started by begin_work.

_initialized

Returns true if the database has been initialized for Sqitch, and false if it has not.

_initialize

Initializes the target database for Sqitch by installing the Sqitch registry schema and/or tables. Should be overridden by subclasses. This implementation throws an exception.

register_project

$engine->register_project;

Registers the current project plan in the registry database. The implementation should insert the project name and URI if they have not already been inserted. If a project already exists with the same name but different URI, or a different name and the same URI, an exception should be thrown.

is_deployed_tag

say 'Tag deployed' if $engine->is_deployed_tag($tag);

Should return true if the tag has been applied to the database, and false if it has not.

is_deployed_change

say 'Change deployed' if $engine->is_deployed_change($change);

Should return true if the change has been deployed to the database, and false if it has not.

are_deployed_changes

say "Change $_ is deployed" for $engine->are_deployed_change(@changes);

Should return the IDs of any of the changes passed in that are currently deployed. Used by deploy to ensure that no changes already deployed are re-deployed.

change_id_for

say $engine->change_id_for(
    change  => $change_name,
    tag     => $tag_name,
    project => $project,
);

Searches the database for the change with the specified name, tag, project, or ID. Returns undef if it matches no changes. If it matches more than one change, it returns the earliest deployed change if the first parameter is passed; otherwise it throws an exception The parameters are as follows:

change

The name of a change. Required unless tag or change_id is passed.

change_id

The ID of a change. Required unless tag or change is passed. Useful to determine whether an ID in a plan has been deployed to the database.

tag

The name of a tag. Required unless change is passed.

project

The name of the project to search. Defaults to the current project.

first

Return the earliest deployed change ID if the search matches more than one change. If false or not passed and more than one change is found, an exception will be thrown.

If both change and tag are passed, find_change_id will search for the last instance of the named change deployed before the tag.

changes_requiring_change

my @requiring = $engine->changes_requiring_change($change);

Returns a list of hash references representing currently deployed changes that require the passed change. When this method returns one or more hash references, the change should not be reverted. Each hash reference should contain the following keys:

change_id

The requiring change ID.

change

The requiring change name.

project

The project the requiring change is from.

asof_tag

Name of the first tag to be applied after the requiring change was deployed, if any.

log_deploy_change

$engine->log_deploy_change($change);

Should write the records to the registry necessary to indicate that the change has been deployed.

log_fail_change

$engine->log_fail_change($change);

Should write to the database event history a record reflecting that deployment of the change failed.

log_revert_change

$engine->log_revert_change($change);

Should write to and/or remove from the registry the records necessary to indicate that the change has been reverted.

log_new_tags

$engine->log_new_tags($change);

Given a change, if it has any tags that are not currently logged in the database, they should be logged. This is assuming, of course, that the change itself has previously been logged.

earliest_change_id

my $change_id = $engine->earliest_change_id($offset);

Returns the ID of the earliest applied change from the current project. With the optional $offset argument, the ID of the change the offset number of changes following the earliest change will be returned.

latest_change_id

my $change_id = $engine->latest_change_id;
my $change_id = $engine->latest_change_id($offset);

Returns the ID of the latest applied change from the current project. With the optional $offset argument, the ID of the change the offset number of changes before the latest change will be returned.

deployed_changes

my @change_hashes = $engine->deployed_changes;

Returns a list of hash references, each representing a change from the current project in the order in which they were deployed. The keys in each hash reference must be:

id

The change ID.

name

The change name.

project

The name of the project with which the change is associated.

note

The note attached to the change.

planner_name

The name of the user who planned the change.

planner_email

The email address of the user who planned the change.

timestamp

An App::Sqitch::DateTime object representing the time the change was planned.

tags

An array reference of the tag names associated with the change.

deployed_changes_since

my @change_hashes = $engine->deployed_changes_since($change);

Returns a list of hash references, each representing a change from the current project deployed after the specified change. The keys in the hash references should be the same as for those returned by deployed_changes().

name_for_change_id

my $change_name = $engine->name_for_change_id($change_id);

Returns the tag-qualified name of the change identified by the ID. If a tag was applied to a change after that change, the name will be returned with the tag qualification, e.g., app_user@beta. Otherwise, it will include the symbolic tag @HEAD. e.g., widgets@HEAD. This value should be suitable for uniquely identifying the change, and passing to the get or index_of methods of App::Sqitch::Plan.

registered_projects

my @projects = $engine->registered_projects;

Returns a list of the names of Sqitch projects registered in the database.

current_state

my $state = $engine->current_state;
my $state = $engine->current_state($project);

Returns a hash reference representing the current project deployment state of the database, or undef if the database has no changes deployed. If a project name is passed, the state will be returned for that project. Otherwise, the state will be returned for the local project.

The hash contains information about the last successfully deployed change, as well as any associated tags. The keys to the hash should include:

project

The name of the project for which the state is reported.

change_id

The current change ID.

script_hash

The deploy script SHA-1 hash.

change

The current change name.

note

A brief description of the change.

tags

An array reference of the names of associated tags.

committed_at

An App::Sqitch::DateTime object representing the date and time at which the change was deployed.

committer_name

Name of the user who deployed the change.

committer_email

Email address of the user who deployed the change.

planned_at

An App::Sqitch::DateTime object representing the date and time at which the change was added to the plan.

planner_name

Name of the user who added the change to the plan.

planner_email

Email address of the user who added the change to the plan.

current_changes

my $iter = $engine->current_changes;
my $iter = $engine->current_changes($project);
while (my $change = $iter->()) {
    say '* ', $change->{change};
}

Returns a code reference that iterates over a list of the currently deployed changes in reverse chronological order. If a project name is not passed, the current project will be assumed. Each change is represented by a hash reference containing the following keys:

change_id

The current change ID.

script_hash

The deploy script SHA-1 hash.

change

The current change name.

committed_at

An App::Sqitch::DateTime object representing the date and time at which the change was deployed.

committer_name

Name of the user who deployed the change.

committer_email

Email address of the user who deployed the change.

planned_at

An App::Sqitch::DateTime object representing the date and time at which the change was added to the plan.

planner_name

Name of the user who added the change to the plan.

planner_email

Email address of the user who added the change to the plan.

current_tags

my $iter = $engine->current_tags;
my $iter = $engine->current_tags($project);
while (my $tag = $iter->()) {
    say '* ', $tag->{tag};
}

Returns a code reference that iterates over a list of the currently deployed tags in reverse chronological order. If a project name is not passed, the current project will be assumed. Each tag is represented by a hash reference containing the following keys:

tag_id

The tag ID.

tag

The name of the tag.

committed_at

An App::Sqitch::DateTime object representing the date and time at which the tag was applied.

committer_name

Name of the user who applied the tag.

committer_email

Email address of the user who applied the tag.

planned_at

An App::Sqitch::DateTime object representing the date and time at which the tag was added to the plan.

planner_name

Name of the user who added the tag to the plan.

planner_email

Email address of the user who added the tag to the plan.

search_events

my $iter = $engine->search_events( %params );
while (my $change = $iter->()) {
    say '* $change->{event}ed $change->{change}";
}

Searches the deployment event log and returns an iterator code reference with the results. If no parameters are provided, a list of all events will be returned from the iterator reverse chronological order. The supported parameters are:

event

An array of the type of event to search for. Allowed values are "deploy", "revert", and "fail".

project

Limit the events to those with project names matching the specified regular expression.

change

Limit the events to those with changes matching the specified regular expression.

committer

Limit the events to those logged for the actions of the committers with names matching the specified regular expression.

planner

Limit the events to those with changes who's planner's name matches the specified regular expression.

limit

Limit the number of events to the specified number.

offset

Skip the specified number of events.

direction

Return the results in the specified order, which must be a value matching /^(:?a|de)sc/i for "ascending" or "descending".

Each event is represented by a hash reference containing the following keys:

event

The type of event, which is one of:

deploy
revert
fail
project

The name of the project with which the change is associated.

change_id

The change ID.

change

The name of the change.

note

A brief description of the change.

tags

An array reference of the names of associated tags.

requires

An array reference of the names of any changes required by the change.

conflicts

An array reference of the names of any changes that conflict with the change.

committed_at

An App::Sqitch::DateTime object representing the date and time at which the event was logged.

committer_name

Name of the user who deployed the change.

committer_email

Email address of the user who deployed the change.

planned_at

An App::Sqitch::DateTime object representing the date and time at which the change was added to the plan.

planner_name

Name of the user who added the change to the plan.

planner_email

Email address of the user who added the change to the plan.

run_file

$engine->run_file($file);

Should execute the commands in the specified file. This will generally be an SQL file to run through the engine's native client.

run_handle

$engine->run_handle($file_handle);

Should execute the commands in the specified file handle. The file handle's contents should be piped to the engine's native client.

load_change

my $change = $engine->load_change($change_id);

Given a deployed change ID, loads an returns a hash reference representing the change in the database. The keys should be the same as those in the hash references returned by deployed_changes(). Returns undef if the change has not been deployed.

change_offset_from_id

my $change = $engine->change_offset_from_id( $change_id, $offset );

Given a change ID and an offset, returns a hash reference of the data for a deployed change (with the same keys as defined for deployed_changes()) in the current project that was deployed $offset steps before the change identified by $change_id. If $offset is 0 or undef, the change represented by $change_id should be returned (just like load_change()). Otherwise, the change returned should be $offset steps from that change ID, where $offset may be positive (later step) or negative (earlier step). Returns undef if the change was not found or if the offset is more than the number of changes before or after the change, as appropriate.

change_id_offset_from_id

my $id = $engine->change_id_offset_from_id( $change_id, $offset );

Like change_offset_from_id() but returns the change ID rather than the change object.

planned_deployed_common_ancestor_id

my $change_id = $engine->planned_deployed_common_ancestor_id;

Compares the SHA1 hashes of the deploy scripts to their values at the time of deployment to the database and returns the latest change ID prior to any changes for which the values diverge. Used for the --modified option to the revert and rebase commands.

registry_version

Should return the current version of the target's registry.

See Also

sqitch

The Sqitch command-line client.

Author

David E. Wheeler <david@justatheory.com>

License

Copyright (c) 2012-2023 iovation Inc., David E. Wheeler

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.