NAME
Ekorn::Pipeline::JSON - read and write JSON and JSON Lines, as sources, stages or sinks
SYNOPSIS
use Ekorn::Pipeline::JSON qw(:all);
# Sources: one document, an array giving its elements; or JSON
# Lines, a document a line.
my $next = Ekorn::Pipeline->new(@stages)->iterator(from_json(file => 'tracks.json'));
my $lines = Ekorn::Pipeline->new(@stages)->iterator(from_jsonl(file => 'events.jsonl'));
# Sinks: one array of every value; or a value a line. Each gives how
# many it wrote.
my $n = Ekorn::Pipeline->new(@stages, to_json(file => 'out.json'))->(@values);
my $m = Ekorn::Pipeline->new(@stages, to_jsonl(file => 'out.jsonl'))->(@values);
# Stages: decoding texts, and encoding values.
my @rows = Ekorn::Pipeline->new(from_jsonl())->(@texts);
my @lines = Ekorn::Pipeline->new(to_jsonl(json => {canonical => 1}))->(@rows);
DESCRIPTION
Four functions, one for each direction and format, each making a different part by the options it's given. Given a file or an fh, a from_ function makes a source and a to_ function a sink; given neither, each makes a stage. Each part is made afresh for each run.
The json functions read and write one JSON document; the jsonl functions JSON Lines, also called NDJSON: a document a line. A stage passes on what a file of its format would hold, so to_json and from_json undo each other, as to_jsonl and from_jsonl do.
The encoder is JSON::MaybeXS's: Cpanel::JSON::XS where it's installed, JSON::PP otherwise. Text is UTF-8 bytes, as encode_json and decode_json have it, and files are opened :raw. A file is opened as the run starts, emptied if it's written, and closed once the run has ended, however it ended; a filehandle is left open, for its opener to close.
Nothing is exported by default; ask for each, or for :all, or import them from Ekorn::Pipeline::Tooling's :json. JSON exports a from_json and a to_json of its own by default, so a module using both needs to import one under another name, or call it by its full name.
FUNCTIONS
from_json
from_json(file => $path, %options)
from_json(fh => $fh, %options)
from_json(%options)
Given a file or an fh, a source of the one document read from it: its elements, if it's an array, or itself. An empty input gives none. The input is read whole before the first value is given, and one holding more than one document croaks, pointing to "from_jsonl".
Given neither, a stage decoding each value, one JSON text, the same way: an array gives its elements, and anything else itself.
from_jsonl
from_jsonl(file => $path, %options)
from_jsonl(fh => $fh, %options)
from_jsonl(%options)
Given a file or an fh, a source of the documents read from it, JSON Lines, each as it is: an array is one value. The input is read a block at a time, as values are pulled, so a stage that stops the run early stops the reading too. Documents simply written one after another, on a line or not, are read the same.
Given neither, a stage decoding each value, a text of one or more lines, into each line's document.
to_json
to_json(file => $path, %options)
to_json(fh => $fh, %options)
to_json(%options)
Given a file or an fh, a sink writing every value as one JSON document, an array, written as the values come, and giving how many. An encoder that indents, such as with pretty, has each value indented in the array.
Given neither, a stage gathering every value, and passing on one text, the array, once they've all come.
to_jsonl
to_jsonl(file => $path, %options)
to_jsonl(fh => $fh, %options)
to_jsonl(%options)
Given a file or an fh, a sink writing each value as a line of JSON Lines, and giving how many.
Given neither, a stage passing on each value as its line, without its newline, ready for "to_file" in Ekorn::Pipeline::Sinks. JSON Lines can't be indented, so it croaks given an encoder that indents.
OPTIONS
- file
-
The path of a file to read or write.
- fh
-
An open filehandle to read or write. It should be
:raw, as the text is bytes, unless the encoder'sutf8is off. - json
-
The encoder: a hash reference of options for JSON::MaybeXS's
new, such as{canonical => 1, pretty => 1}, given over the defaultsutf8andallow_nonref; or an object of your own, such as a Cpanel::JSON::XS or JSON::PP you've set up, used as it is. An object reading a source, or decoding with "from_jsonl"'s stage, has its incremental parser reset for each run, so mustn't serve two at once.
DIAGNOSTICS
- Expected options as name => value pairs
-
A function was given an odd number of arguments.
- Expected the options file, fh and json, not %s
-
A function was given an option it doesn't know, named after "not".
- Expected file to be a path
-
filewas undefined. - Expected a file or a filehandle, not both
-
A function was given both
fileandfh. - Expected an open filehandle
-
fhwas something other than an open filehandle. - Expected json to be a hash reference of options, or an encoder object
-
jsonwas something else. - Expected json options the encoder knows: %s
-
json's options included one the encoder doesn't have; the rest of the message is the encoder's. - Expected an encoder that doesn't indent, to write JSON Lines
-
"to_jsonl" was given an encoder that indents, such as with
pretty. - Can't decode JSON from %s: %s
-
What was read wasn't JSON: the first part is the file's path, "the filehandle", or "a value", for a stage, and the second the decoder's reason. It's "it ends inside a document" if the input ended part way through one, and "it holds more than one document; read JSON Lines with from_jsonl" if "from_json" read more than one.
- Can't encode a value as JSON: %s
-
A value couldn't be encoded, such as an object, by default; the second part is the encoder's reason.
- Can't read %s: %s
- Can't write %s: %s
-
A file couldn't be opened, read, written or closed, or a filehandle read or written; the second part says why.
SEE ALSO
Ekorn::Pipeline, Ekorn::Pipeline::Sources and Ekorn::Pipeline::Sinks, for files of lines, and JSON::MaybeXS.
The user guide's "Walk through a module of parts" builds a smaller from_jsonl and to_jsonl, step by step.
AUTHOR
Iain Campbell <cpanic@cpan.org>
LICENSE
Copyright (c) 2026 by Iain Campbell.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.