Upgrading from Disbatch 3 to Disbatch 4

Preliminary steps

• Rename the tasks and queues collections to tasks and queues if they have different names

• Set each queue's threads to how many maximum concurrent threads should be ran for that queue across all DENs. The queue field maxthreads, which applied per DEN, is no longer used.

• Run one of the following on each database, as the constructor field has been renamed to plugin:

    // for back-compat with Disbatch 3
    db.queues.distinct("constructor").forEach(function(c){
        db.queues.update({constructor: c}, {$set: {plugin: c}}, {multi: 1})
    })
  
    // or for no back-compat with Disbatch 3
    db.queues.update({}, {$rename: {constructor: "plugin"}})
  

• If using MongoDB authentication, make sure the plugin role has the proper permissions for any collections the plugin modifies.

Configure

See Configuring

Consult /etc/disbatch/disbatch.ini for reference of current settings.

Modify your plugins:

• To support only Disbatch 4:

  • Remove these lines:

        use Synacor::Disbatch::Task;
        use Synacor::Disbatch::Engine;
        our @ISA=qw(Synacor::Disbatch::Task);
    

  • Modify new():

    The plugin was formerly instantiated as new($queue, $parameters), and is now instantiated as new(workerthread => $workerthread, task => $doc).

    See Example new() below.

  • Remove any $Synacor::Disbatch::Engine::EventBus call (namely, report_task_done).

    Any other EventBus usage will no longer work.

  • Finally, the task must return this when finished:

        {status => $status, stdout => $stdout, stderr => $stderr};
    

• To support both Disbatch 3 and Disbatch 4:

  • Replace these lines:

        use Synacor::Disbatch::Task;
        use Synacor::Disbatch::Engine;
        our @ISA=qw(Synacor::Disbatch::Task);
    

  • with the following:

        warn "Synacor::Disbatch::Task not found\n"
            unless eval 'use base "Synacor::Disbatch::Task"; 1';
        warn "Synacor::Disbatch::Engine not found\n"
            unless eval 'use Synacor::Disbatch::Engine; 1';
    

  • or if you don't care for the warnings if they aren't installed:

        eval 'use base "Synacor::Disbatch::Task"';
        eval 'use Synacor::Disbatch::Engine';
    

  • Modify new():

    The plugin was formerly instantiated as new($queue, $parameters), and is now instantiated as new(workerthread => $workerthread, task => $doc).

    See Example new() below.

  • Append to any $Synacor::Disbatch::Engine::EventBus call (namely, report_task_done):

        if defined $Synacor::Disbatch::Engine::EventBus;
    

    Any other EventBus usage will no longer work.

  • Finally, the task must return this when finished:

        {status => $status, stdout => $stdout, stderr => $stderr};
    

Example new()

Disbatch 3 was called via new($queue, $parameters), with $queue containing {id => $queue_id} and $parameters containing the task's parameters.

Disbatch 4 is called via new(workerthread => $workerthread, task => $doc), with $workerthread being a Disbatch object using the plugin MongoDB user and role, and $doc being the task's document from MongoDB.

The below is from lib/Disbatch/Plugin/Demo.pm.

sub new {
    my $class = shift;

    # deprecated Disbatch 3 format
    if (ref $_[0]) {
        my ($queue, $parameters) = @_;
        warn Dumper $parameters;
        my %self = map { $_ => $parameters->{$_} } keys %$parameters;	# modifying $parameters breaks something in Disbatch 3.
        $self{queue_id} = $queue->{id};
        return bless \%self, $class;
    }

    my $self = { @_ };
    $self->{task}{params} //= $self->{task}{parameters} if defined $self->{task}{parameters};	# for deprecated Disbatch 3 format
    warn Dumper $self->{task}{params};

    # back-compat, so as to not change Disbatch 3 plugins so much
    # stick all params in $self
    for my $param (keys %{$self->{task}{params}}) {
        next if $param eq 'workerthread' or $param eq 'task';
        $self->{$param} = $self->{task}{params}{$param};
    }
    $self->{queue_id} = $self->{task}{queue};
    $self->{id} = $self->{task}{_id};

    bless $self, $class;
}