TITLE

fastpack-split - split fastpack data files into message/bytes size limited output sequences suitable for JPack

SYNOPSIS

fastpack-split  [options] fastpack_files ...
fastpack-split  --html_container path/to/index.html   [-o group] file1.fastpack file2.fastpack

Options include:

html_container

A required option. The path the html file which will act as the 'container' for this data. The containing dir of this file becomes the root dir of the data sctructures.

This can be a directory, in which case it is used directly as the html root

prefix

The default output directory, relative to the html container root. Default is data/jpack

compress

Flag indicating if jpack output is to be compressed. Default is true

message_limit

The maximum number of messages allowed in an output file. Once reached, a new output file is created.

byte_limit

The maximum bytes allowed in an output file. Once reached,a new output file is created. Note the exact size of the file may be larger than this limit, due to unkown compression sizing

read_size

Maximum bytes to read from an in put file at a time. Allows for less memory usage. Default is 8*4096 bytes

output group

If a input has an output groups specified, this is used instead of the prefix. Output files are always numbered sequentiallly based of the existing files in the dir

DESCRIPTION

Takes a list of input files, expected to be in fastpack format. The files are read and each record is accounted. Once a limit of records or file size has be reache a new output file is created. Hence the splitting behaviour. This can also be used to merge multiple fastpack files, as the split happend on output limits not input limits.