/ 
Flat-Profile-0.02
/ Flat::Profile

NAME

Flat::Profile - Streaming-first profiling for CSV/TSV flat files

SYNOPSIS

use Flat::Profile;

my $p = Flat::Profile->new();

my $it = $p->iter_rows(
    path       => "data.csv",
    has_header => 1,
    delimiter  => ",",
    encoding   => "UTF-8",
);

while (my $row = $it->next_row) {
    # $row is an arrayref
}

my $report = $p->profile_file(
    path        => "data.csv",
    has_header  => 1,
    delimiter   => ",",
    null_empty  => 1,
    null_tokens => ["NULL", "NA"],
    example_cap => 10,
    max_errors  => 1000,
);

DESCRIPTION

Flat::Profile is part of the Flat::* series. It provides streaming-first profiling for CSV/TSV inputs for practical ETL and legacy data workflows.

Design goals:

  • Streaming-first (single pass, predictable memory)

  • Practical diagnostics (ragged rows, null policy, examples)

  • Stable report format intended to feed Flat::Schema / Flat::Validate

METHODS

new

my $p = Flat::Profile->new();

Constructor. Takes named arguments (currently reserved for future configuration).

iter_rows

my $it = $p->iter_rows(%args);

Returns an iterator object (Flat::Profile::Iterator).

Required named arguments:

  • path

Common named arguments:

  • has_header (boolean)

  • delimiter ("," or "\t")

  • encoding (default "UTF-8")

profile_file

my $report = $p->profile_file(%args);

Profiles a CSV/TSV file in a streaming pass and returns a hashref report.

Key named arguments include:

  • path (required)

  • has_header

  • delimiter

  • encoding

  • null_empty (default true)

  • null_tokens (arrayref; default empty)

  • example_cap (default 10)

  • max_errors (threshold stop; default 1000)

NULL SEMANTICS

By default, empty string is treated as null:

null_empty => 1   # default

To treat empty string as a value:

null_empty => 0

You can also treat specific exact tokens as null:

null_tokens => ["NULL", "N/A"]

Notes:

  • Token matching is exact (no trimming, case-sensitive) in v1.

  • undef is always treated as null.

RAGGED ROWS

Flat::Profile tracks width mismatches relative to an expected width:

  • If has_header is true, expected width is the header width.

  • Otherwise, expected width is the first data row width.

Row numbers in ragged examples use data-row numbering (header excluded): the first data row is row_number 1.

REPORT FORMAT

profile_file() returns a hashref with stable top-level metadata including:

  • report_version

  • generated_at (UTC timestamp string)

  • perl_version

  • module_version

  • header (arrayref or undef)

  • rows (data rows processed; header excluded)

  • ragged (counts + capped examples)

  • columns (arrayref of per-column stats)

AUTHOR

Sergio de Sousa

Issues: https://github.com/sergio-desousa/Flat-Profile/issues

LICENSE

Perl 5 (Artistic/GPL dual).