NAME

File::LinearRaid - Treat multiple files as one large seamless file for reading and writing.

SYNOPSIS

use File::LinearRaid;

my $fh = File::LinearRaid->new( "+<",
    "data/datafile0" => 100_000,
    "data/datafile1" =>  50_000,
    "data/datafile2" => 125_000
);

## this chunk of data actually crosses a physical file boundary
seek $fh, 90_000, 0;
read $fh, my $buffer, 20_000;

## replace that chunk with X's
seek $fh, 90_000, 0;
print $fh "X" x 20_000;

DESCRIPTION

This module provides a single-filehandle interface to multiple files, in much the same way that a linear RAID provides a single-device interface to multiple physical hard drives.

This module was written to provide random fixed-width record access to a series of files. For example, in the BitTorrent filesharing protocol, several files are shared as a single entity. The final sizes of the individual files are known, but the protocol only sends fixed-width chunks of data. These chunks are not aligned to file boundaries and can span several physical files, but they are only identified by their number and not by the files they span.

This module was created to provide a layer of abstraction around this kind of storage. Instead of calculating possibly many file offsets, and dividing data into smaller pieces, a simple seek and read (or print) on the abstract filehandle will do the right thing, regardless of how the chunk spans the physical files:

seek $fh, ($chunk_id * $chunk_size), 0;
read $fh, my $buffer, $chunk_size;

## or if opened with mode "+<" or similar:

seek $fh, ($chunk_id * $chunk_size), 0;
print $fh $chunk;

At this time the module is still beta quality, but most common file activities should work fine.

USAGE

File::LinearRaid->new( $mode, $path1 => $size1, ... )

Returns a new File::LinearRaid object consisting of the listed paths. Each physical file is opened using the given mode.

Each file needs an associated maximum length. This need not be the current length of the file. If the file is shorter than this length, the LinearRaid filehandle will behave as if the file were null-padded to this length (although it will not modify the file for reading). If the file is longer than this length, the portion of thie file past this length will be ignored (but preserved). When writing to the LinearRaid filehandle for random access, the physical files will be grown (with null characters) as needed.

Currently, read, readline, print, getc and friends are implemented, so you should be able to use most file operations seamlessly. Writing beyond the specified limit of the final physical file is not supported.

CAVEATS

This module is currently not much more than proof-of-concept. Although there is a test suite, things might not be perfect yet.

• Probably doesn't play well with unicode / wide characters.

• Error checking is quite limited.

• Formats are untested, I don't use them and don't know if they'll work.

AUTHOR

File::LinearRaid is written by Mike Rosulek <mike@mikero.com>. Feel free to contact me with comments, questions, patches, or whatever.

COPYRIGHT

Copyright (c) 2004 Mike Rosulek. All rights reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.