/ 
PDL-IO-BareStore-0.01
/ PDL::IO::BareStore

NAME

PDL::IO::BareStore - Simple PDL extension for reading 2 dimensional "Big Data"

SYNOPSIS

use PDL::IO::BareStore;

# simple way to tarnsform CSV files into "database" file
# perl -E 'say join ",", 1..14 for 1..1000' | perl -nF, -e 'chomp;print pack $p, @F}BEGIN{$p = shift' "C*" > ./output.rdb

my $D = new PDL::IO::BareStore(
  file => 'output.rdb',   # file name
  dims => [14, 100],      # 14 record, 100 line each time
                          # Not quite handy, any suggestion?
  type => 'C*',           # How you packed the database
  readCount => 5,         # read file for five times
);                        # only readCount is optional, you should specify those parameters

while ( $D->nextBlock(\my $b) > 0 ) {   # sequentially read the file
  # $b is a piddle containing a subset of the data
  # do something with $b, which is a subset of output.rdb
}

DESCRIPTION

Written for the 2-Dimensional "Big data" set reading. As I got "Out of Memory" for loading a large data into piddle with PDL::IO::FastRaw, and still cannot figure out how to use PDL::IO::FlexRaw; This module provide a simple wrapper for reading large binary data into smaller piddles.

kmx's PDL::IO::DBI is a really good solution, but I don't want to use Relational databases to store my data.

HISTORY

0.01

Just found that the call $$pdl->upd_data() is essential.

0.00_02

Bugfixes and improved performance

0.00_011

Fixed Preq, in order to pass the cpan test

0.00_01

Original version.

SEE ALSO

You May also interested in PDL::IO::FlexRaw PDL::IO::DBI

AUTHOR

Kwok Lok Chung, Baggio, <rootkwok @ cpan.org>

LICENSE

Copyright (C) 2015 by Kwok Lok Chung, Baggio.

This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/.