NAME

Net::CascadeCopy - efficiently replicate files across many servers

SYNOPSIS

use Net::CascadeCopy;

# create a new CascadeCopy object
my $ccp = Net::CascadeCopy->new( { ssh          => "/path/to/ssh",
                                   ssh_flags    => "-x -A",
                                   max_failures => 3,
                                   max_forks    => 2,
                               } );

# set the command and arguments to use to transfer file(s)
$ccp->set_command( "scp", "-p" );

# set path on the local server
$ccp->set_source_path( "/path/on/local/server" );
# set path on all remote servers
$ccp->set_source_path( "/path/on/remote/servers" );

# add lists of servers in multiple datacenters
$ccp->add_group( "datacenter1", \@dc1_servers );
$ccp->add_group( "datacenter2", \@dc2_servers );

# transfer all files
$ccp->transfer();

DESCRIPTION

This module efficiently replicates a file or directory across a large number of servers in multiple datacenters via rsync or scp.

The usual approach to copying a file to many servers is to copy it to one or more central file servers and then copy it from there to every other server in the group. The speed at which the file can be copied is a function of the local cpu/disk utilization and the network throughput on the file server(s).

This module takes a different approach. Once the file has been copied to a remote server, that server will be used as a source point for copying to additional servers in the same group. Hence the speed of the transfer can increase exponentially rather than linearly.

A new process will be forked for every transfer, so as soon as each transfer completes, new transfers will immediately be started.

CONSTRUCTOR

new( )

Returns a reference to a new use Net::CascadeCopy object.

Here are the currently supported options at creation time:

This is an inside-out perl class. For more info, see "Perl Best Practices" by Damian Conway

CONSTRUCTOR OPTIONS

ssh => "/path/to/ssh"

Name or path of ssh script ot use to log in to each remote server to begin a transfer to another remote server.

ssh_flags => "-option1 -option2"

Command line options to be passed to ssh script.

max_failures => 3

The Maximum number of transfer failures to allow before giving up on a target host.

max_forks => 2

The maximum number of simultaneous transfers that should be running per source server.

INTERFACE

$self->add_group( $groupname, \@servers )

Add a group of servers. Ideally all servers will be located in the same datacenter. This may be called multiple times with different group names to create multiple groups.

$self->set_command( $command, $args )

Set the command and arguments that will be used to transfer files. For example, "rsync" and "-ravuz" could be used for rsync, or "scp" and "-p" could be used for scp.

$self->set_source_path( $path )

Specify the path on the local server where the source files reside.

$self->set_target_path( $path )

Specify the target path on the remote servers where the files should be copied.

$self->transfer( )

Transfer all files. Will not return until all files are transferred.

BUGS AND LIMITATIONS

Note that this is still an alpha release.

There are no known bugs in this module. Please report problems to VVu@geekfarm.org

Patches are welcome.

AUTHOR

Alex White <vvu@geekfarm.org>

LICENCE AND COPYRIGHT

Copyright (c) 2007, Alex White <vvu@geekfarm.org>. All rights reserved.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

- Neither the name of the geekfarm.org nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.