NAME

Catmandu::Plugin::SideCar - Automatically update a parallel Catmandu::Store with metadata

SYNOPSIS

# Using the command line

$ cat catmandu.yml
---
store:
    files:
     package: File::Simple
     options:
         root: /data/test123
         bags:
             index:
                 plugins:
                     - SideCar
                 sidecar:
                         package: ElasticSearch
                         options:
                             client: '1_0::Direct'
                             index_name: catmandu

   ...

# Add files to the FileStore in bag 1234
$ catmandu stream /tmp/test.txt to files --bag 1234 --id test.txt

# Add metadata to the FileStore for bag 1234
$ cat metadata.yml
---
_id: 1234
colors:
   - red
   - green
   - blue
name: test
...
$ catmandu import YAML to files < metadata.yml

# Export the metadata again from the FileStore
$ catmandu export files to YAML
---
_id: 1234
colors:
   - red
   - green
   - blue
name: test
...

# Or in your Perl program
my $store = Catmandu->store('File::Simple',
           root => 'data/test123'
           bags => {
               index => {
                   plugins => [qw(SideCar)],
                   sidecar => {
                       package => "ElasticSearch",
                       options => {
                           client => '1_0::Direct',
                           index_name => 'catmandu',
                       }
                   }
              }
           });

my $index = $store->index;

$index->add({ _id => '1234' , colors => [qw(red green blue)] , name => 'test'});

my $files = $index->files('1234');
$files->upload(IO::File->new('</tmp/test.txt'), 'test.txt');

my $file = $files->get('text.txt');

$files->steam(IO::File->new('>/tmp/test.txt'),$file);

DESCRIPTION

The Catmandu::Plugin::SideCar can be used to combine Catmandu::Store-s , Catmandu::FileStore-s (and Catmandu::Store::Multi , Catmandu::Store::File::Multi) as one access point. Every get,add,delete,drop and commit action in the store will be first executed in the original store and re-executed in the SideCar store.

COMBINING A FILESTORE WITH A STORE

To add metadata to a Catmandu::FileStore a SideCar needs to be added to the index bag of the FileStore:

package: File::Simple
options:
    root: /data/test123
    bags:
        index:
            plugins:
                - SideCar
            sidecar:
                    package: ElasticSearch
                    options:
                        client: '1_0::Direct'
                        index_name: catmandu
            sidecar_bag: data

COMBINING A STORE WITH A FILESTORE

To add files to a Catmandu::Store a SideCar needs to be added to the bag containing the metadata (by default data):

package: ElasticSearch
options:
    client: '1_0::Direct'
    index_name: catmandu
    bags:
        data:
            plugins:
                - SideCar
            sidecar:
                    package: File::Simple
                    options:
                        root: /data/test123
                        uuid: 1
            sidecar_bag: index

Notice that we added for the Catmandu::Store::File::Simple the requires uuid options because the Catmandu::Store::ElasticSearch is using UUIDs as default identifiers.

RESTRICTIONS

Some Catmandu::FileStore-s may set restrictions on the _id-s that can be used in records.

CONFIGURATION

sidecar STRING

sidecar HASH

sidecar Catmandu::Store | Catmandu::FileStore

The pointer to a configured Catmandu::Store or Catmandu::FileStore.

sidecar_bag

The SideCar Catmandu::Bag into which to store the data (default 'bag').

SEE ALSO

Catmandu::Store, Catmandu::Bag, Catmandu::FileStore