NAME

Data::MagicTie - This module implements a proxy like Perl TIE interface over local and remote Berkeley DB files containing BLOBs

SYNOPSIS

  	use Data::MagicTie;
	use Fcntl;

	my $hash = tie %a,'Data::MagicTie','test';
	my $hash = tie %a,'Data::MagicTie','test',( Q => 7, mode => O_RDONLY); #query 7 dbs in one
	my $hash = tie %a,'Data::MagicTie','test',( Q => 1, noft => 1); #normal hash
	my $hash = tie %a,'Data::MagicTie','test',( style => "BerkeleyDB"); #sleepycat-ish :-)
	my $hash = tie %a,'Data::MagicTie','test',( lr => 1, dbms_host => 'me.jrc.it'); #cool way

	$a{mykey} = 'myvalue'; #store
	my $b = $a{mykey}; #fetch
	#iterator
	while (($k,$v) = each %a) {
		my $c = $v;
	};
	#clear
	%a=();

	#basic delegation model - first match %a then %b
	my $hash1 = tie %b,'Data::MagicTie','test1';
	$hash1->set_parent($hash);
	print $b{mykey}; # looks up in %a :)
	untie %b;

	untie %a;

DESCRIPTION

This module acts as a proxy for the actual implementations of local and remote counterparts Data::MagicTie::DBMS(3) Data::MagicTie::DB_File(3) Data::MagicTie::BerkeleyDB(3) modules. It allows to an application script to transparently TIE hashes and arrays to either local or remote Berkeley DB files, containing key/value pairs. The values can be either strings or in-memory data structures (BLOBs) - see Storable(3); each tie database can then be splitted up on several files for eccifency and reduce the size of the actual database files. More, for query purposes only, tie operations can be "chained" to transparently access different databases; such a chain feature does not need any additional field in the database, but it is just using in-memory Perl OO methods to delegate read operations (FETCH, EXISTS, FIRSTKEY, NETXKEY). I.e. a look up for a key or value in a database ends up in a read operation in the current database or in one of its "delegates".

Each atomic operation using the Perl operators actually trigger either local or remote database lookups and freeze/thaw operations on values. Perl constructs such as each, keys and values can be used to iterate over the content of a tied database; when the file is splitted over several single files the module iterates over the whole set of files. Even when a parent (delegate) is set for a database these operators allow to scan the whole set of storages (note: this feature might be not efficent over large databases).

By using such a Perl TIE model is becoming easy to write simple "cache" systems for example using the Apache Web server and mod_perl. This could really important for RDF storages and cumbersome and cpu-consuming queries. (see RDFStore::Model and RDFStore::FindIndex)

CONSTRUCTORS

The following methods construct/tie Data::MagicTie databases and objects:

$db_hash = tie %b, 'Data::MagicTie', $filename, %whateveryoulikeit;

Tie the hash %b to a MagicTie database called $filename. The %whateveryoulikeit hash contains a set of configuration options about how and where store actual data. Possible options are the following:

lr

This is an integer flag 1/0 if a database is going to be stored in the local filesystem or on a remote DBMS(3) server - see Data::MagicTie::DBMS(3). Default is 0, local storage

Q

An integer about how many files to split around the database. Default to 1 (normal perltie behaviour). Please note that set a number too high here might exceed you operating system MAX filedescriptors threshold (see man dbmsd(8) and DBMS(3) if installed on your system)

mode

Some valid predefied constant specifing if the database must be created, opened read/write/readonly. For Data::MagicTie::DB_File(3) and Data::MagicTie::DBMS(3) possible values are O_CREAT, O_RDWR or O_RDONLY. For Data::MagicTie::BerkeleyDB(3) it culd either be DB_CREATE or DB_RDONLY - see DB_File(3) and BerkeleyDB(3)

style

A string identifing if the database is going to be Data::MagicTie::DB_File(3), Data::MagicTie::BerkeleyDB(3) or Data::MagicTie::DBMS(3). Possible values are 'DB_File', 'BerkeleyDB' or 'DBMS'. Default is 'DB_File'.

noft

This is in integer flag 1/0 to tell to the Data::MagicTie module whether or not use Storable(3) freeze/thaw operations on values. This could be renamed blobs option. See BUGS section below.

dbms_host

This option is only valid for Data::MagicTie::DBMS(3) style and tells to the system which is the IP address or machine name of the DBMS(3) server. Default is 'localhost'. See man dbmsd(8)

dbms_port

This option is only valid for Data::MagicTie::DBMS(3) style and tells to the system which is the TCP/IP port to connect to for the DBMS protocol. Default is '1234'. See man dbmsd(8)

$db_array = tie @b, 'Data::MagicTie', $filename, %whateveryoulikeit;

Tie the array @b to a MagicTie database called $filename. The %whateveryoulikeit hash is the same as above.

METHODS

Most of the method are common to the standard perltie(3) interface (sync, TIEHASH, TIEARRAY, FETCH, STORE, EXISTS, FIRSTKEY, NEXTKEY, CLEAR, DELETE, DESTROY)

In addition Data::MagicTie provides additional method that allow to magane a simple delegation or pass-through model for database for read methods such as FETCH, EXISTS, FIRSTKEY, NEXTKEY. These are the following:

get_Options()

Return an hash reference containing all the major options plus the directory and filename of the database. See CONSTRUCTORS

set_parent($ref)

Set the parent delegate to which forward read requests. $ref must be a valid Data::MagicTie blessed Perl object, othewise the delegate is not set. After this method call any FETCH, EXISTS, FIRSTKEY or NEXTKEY invocation (normally automagically called by Perl for you :-) starts up a chain of requests to parents till the result has been found or undef.

get_parent()

Return a valid Data::MagicTie blessed Perl object pointing to the parent of a tied database

reset_parent()

Remove the parent of the database and the operations are back to normal.

EXAMPLES

delegates

use Data::MagicTie;

my $hash = tie %a,'Data::MagicTie','test'; my $hash1 = tie %b,'Data::MagicTie','test1'; my $hash2 = tie %c,'Data::MagicTie','test2';

for (1..10) { $a{"A".$_} = "valueA".$_; $b{"B".$_} = "valueB".$_; $c{"C".$_} = "valueC".$_; };

#basic delegation model - first match %a then %a1 then %2 $hash->set_parent($hash1); $hash1->set_parent($hash2); print $a{B3}; # looks up in %b print $a{C9}; # looks up in %c

#I think this one is much cooler :-> my $hash3 = tie %d,'Data::MagicTie','test3',( lr => 1 ); my $hash4 = tie %e,'Data::MagicTie','test4',( style => "BerkeleyDB" );

for (1..10) { $d{"D".$_} = "valueD".$_; $e{"E".$_} = "valueE".$_; };

#...and then use local or remote databases transparently $hash2->set_parent($hash3); $hash3->set_parent($hash4); print $a{D1}; # really the Perl way of doing ;-) print $a{E1},"\n";

#iterator while (($k,$v) = each %a) { print $k,"=",$v,"\n"; };

undef $hash; untie %a; undef $hash1; untie %b; undef $hash2; untie %c; undef $hash3; untie %d; undef $hash4; untie %e;

BUGS

- The current implementation of TIE supports only the TIEHASH and TIEARRAY interfaces.
- Data::MagicTie::DBMS does not support TIEARRAY yet.
- Data::MagicTie ARRAY support is not complete and probably broken
- a well-known problem using BLOBs is the following:
	
	tie %a,"Data::MagicTie","test"; #by default is using BLOBs
	$a{key1} = sub { print "test"; }; # works
	$a{key2} = { a => [ 1,2,3], b => { tt => [6,7],zz => "test1234" } }; # it works too
	$a{key3}->{this}->{is}->{not} = sub { "working"; }; #does not always work

The problem seems to be realated to the fact Perl is "automagically" extending/defining
hashes (or other in-memory structures). As soon as you start to reference a value it
gets created "spontaneously" :-( 
E.g.
	$a = {};
	$a->{a1} = { a2 => [] };

	$b->{a1}->{a2} = []; # this is the same of the two lines above

In the Data::MagicTie realm this problem affects the Storable freeze/thaw method results.
Any idea how to fix this?

SEE ALSO

perltie(3) Storable(3) DBMS(3) Data::MagicTie::DBMS(3) Data::MagicTie::DB_File(3) Data::MagicTie::BerkeleyDB(3)

AUTHOR

Alberto Reggiori <alberto.reggiori@jrc.it> You can send your postcards and bugfixes to

Contact Details

TP270 - Reliable Information Technologies
Institute for Systems, Informatics and Safety
Joint Research Center of the European Community
Ispra VA
21020 Italy

Fax +39 332 78 9185