NAME

KiokuDB - Object Graph storage engine

SYNOPSIS

use KiokuDB;

# use a DSN
my $d = KiokuDB->connect( $dsn, %args );

# or manually instantiate a backend
my $d = KiokuDB->new(
    backend => KiokuDB::Backend::Files->new(
        dir        => "/tmp/foo",
        serializer => "yaml",
    ),
);


# takes a snapshot of $some_object
my $uuid = $d->store($some_object);

# or with a custom ID:
$d->store( $id => $some_object ); # $id can be any string


# retrieve by ID
my $some_object = $d->lookup($uuid);



# some backends (like DBI) support simple searchs
$d->search({ name => "foo" });


# others use GIN queries (DBI supports both)
$d->search($gin_query);

ALPHA WARNING

KiokuDB is still in its infancy. No fundamental changes are expected, but nevertheless backwards compatibility is not yet guaranteed.

DESCRIPTION

Kioku is a Moose based frontend to various databases, somewhere in between Tangram and Pixie. It builds on Class::MOP's solid foundation.

Its purpose is to provide persistence for "regular" objects with as little effort as possible, without sacrificing control over how persistence is actually done.

Kioku is also non-invasive: it does not use ties, AUTOLOAD, proxy objects, sv_magic or any other type of trickery to get its job done, to avoid unwanted surprises.

Many features important for proper Perl space semantics are supported, including shared data, circular structures, weak references, tied structures, etc.

KiokuDB is meant to solve two related persistence problems:

Transparent persistence

Store arbitrary objects without changing their class definitions or worrying about schema details, and without needing to conform to the limitations of a relational model.

Interoperability

Persisting arbitrary objects in a way that is compatible with existing data/code (for example interoprating with another app using CouchDB with JSPON semantics).

FUNDAMENTAL CONCEPTS

In order to use any persistence framework it is important to understand what it does and how it does it.

Systems like Tangram or DBIx::Class generally require explicit meta data and use a schema, which makes them fairly predictable.

When using transparent systems like KiokuDB or Pixie it is more important to understand what's going on behind the scenes in order to avoid surprises and limitations.

Collapsing

When an object is stored using KiokuDB it's collapsed into an KiokDB::Entry.

An entry is a simplified representation of the object, allowing the data to be saved in formats as simple as JSON.

References to other objects are converted to symbolic references in the entry, so objects can be saved independently of each other.

The entries are given to the KiokuDB::Backend for actual storage.

Collapsing is explained in detail in KiokuDB::Collapser. The way an entry is created varies with the object's class.

Linking

When objects are loaded, entries are retrieved from the backend using their UIDs.

When a UID is already loaded (in the live object set of a KiokuDB instance, see KiokuDB::LiveObjects) the live object is used. This way references to shared objects are shared in memory regardless of the order the objects were stored or loaded.

This process is explained in detail in KiokuDB::Linker.

ATTRIBUTES

KiokuDB uses a number of delegates which do the actual work.

Of these only backend is required, the rest have default definitions.

Additional attributes that are not commonly used are listed in "INTERNAL ATTRIBUTES".

backend

This attribute is required.

This must be an object that does KiokuDB::Backend.

The backend handles storage and retrieval of entries.

typemap

This is an instance KiokuDB::TypeMap.

The typemap contains entries which control how KiokuDB::Collapser and KiokuDB::Linker handle different types of objects.

METHODS

connect $dsn, %args

DWIM wrapper for new.

$dsn represents some sort of backend (much like DBI dsns map to DBDs).

An example DSN is:

my $dir = KiokuDB->connect("bdb:dir=path/to/data/");

The backend moniker name is extracted by splitting on the colon. The rest of the string is passed tp new_from_dsn, which is documented in more detail in KiokuDB::Backend.

Typically DSN arguments are separated by ;, with = separating keys and values. Arguments with no value are assumed to denote boolean truth (e.g. jspon:dir=foo;pretty means c<dir => "foo", pretty => 1>).

Extra arguments are passed both to the backend constructor, and the KiokuDB constructor.

Note that if you need a typemap you still need to pass it in:

KiokuDB->connect( $dsn, typemap => $typemap );

configure $config_file, %args

TODO

new %args

Creates a new directory object.

See "ATTRIBUTES"

new_scope

Creates a new object scope. Handled by live_objects.

The object scope artificially bumps up the reference count of objects to ensure that they live at least as long as the scope does.

This ensures that weak references aren't deleted prematurely, and the object graph doesn't get corrupted without needing to create circular structures and cleaning up leaks manually.

lookup @ids

Fetches the objects for the specified IDs from the live object set or from storage.

store @objects

Recursively collapses @objects and inserts or updates the entries.

This performs a full update of every reachable object from @objects, snapshotting everything.

update @objects

Performs a shallow update of @objects.

It is an error to update an object not in the database.

insert @objects

Inserts objects to the database.

It is an error to insert objects that are already in the database, all elements of @objects must be new.

@objects will be collapsed recursively, but the collapsing stops at known objects, which will not be updated.

delete @objects_or_ids

Deletes the specified objects from the store.

Note that this can cause lookup errors if the object you are deleting is referred to by another object, because that link will be broken.

txn_do $code, %args

Executes $code within the scope of a transaction.

This requires that the backend supports transactions (KiokuDB::Backend::Role::TXN).

Transactions may be nested.

search \%proto

search @args

Searching requires a backend that supports querying.

The \%proto form is currently unspecified but in the future should provide a simple but consistent way of looking objects by attributes.

The second form is backend specific querying, for instance Search::GIN::Query objects passed to KiokuDB::Backend::BDB::GIN or the generic GIN backend wrapper KiokuDB::GIN.

root_set

Returns a Data::Stream::Bulk of all the root objects in the database.

all_objects

Returns a Data::Stream::Bulk of all the objects in the database.

grep $filter

Returns a Data::Stream::Bulk of the objects in root_set filtered by $filter.

scan $callback

Iterates the root set calling $callback for each object.

GLOBALS

$SERIAL_IDS

If set at compile time, the default UUID generation role will use serial IDs, instead of UUIDs.

This is useful for testing, since the same IDs will be issued each run, but is utterly broken in the face of concurrency.

INTERNAL ATTRIBUTES

These attributes are documented for completeness and should typically not be needed.

collapser

KiokuDB::Collapser

The collapser prepares objects for storage, by creating KiokDB::Entry objects to pass to the backend.

linker

KiokuDB::Linker

The linker links entries into functioning instances, loading necessary dependencies from the backend.

live_objects

KiokuDB::LiveObjects

The live object set keeps track of objects and entries for the linker and the resolver.

It also creates scope objects that help ensure objects don't garbage collect too early ("new_scope" in KiokuDB::LiveObjects, KiokuDB::LiveObjects::Scope), and transaction scope objects used by txn_do (KiokuDB::LiveObjects::TXNScope).

typemap_resolver

An instance of KiokuDB::TypeMap::Resolver. Handles actual lookup and compilation of typemap entries, using the user typemap.

SEE ALSO

Prior Art on the CPAN

Pixie

DBM::Deep

OOPS

Tangram

DBIx::Class

Polymorphic retrieval is possible with DBIx::Class::DynamicSubclass

Fey::ORM

MooseX::Storage

VERSION CONTROL

KiokuDB is maintained using Git. Information about the repository is available on http://www.iinteractive.com/kiokudb/

AUTHOR

Yuval Kogman <nothingmuch@woobling.org>

COPYRIGHT

Copyright (c) 2008 Yuval Kogman, Infinity Interactive. All rights
reserved This program is free software; you can redistribute
it and/or modify it under the same terms as Perl itself.