NAME

Sereal::Decoder - Fast, compact, powerful binary deserialization

SYNOPSIS

use Sereal::Decoder
  qw(decode_sereal sereal_decode_with_object scalar_looks_like_sereal);

my $decoder = Sereal::Decoder->new({...options...});

my $structure;
$decoder->decode($blob, $structure); # deserializes into $structure

# or if you don't have references to the top level structure, this works, too:
$structure = $decoder->decode($blob);

# alternatively functional interface: (See Sereal::Performance)
sereal_decode_with_object($decoder, $blob, $structure);
$structure = sereal_decode_with_object($decoder, $blob);

# much slower functional interface with no persistent objects:
decode_sereal($blob, {... options ...}, $structure);
$structure = decode_sereal($blob, {... options ...});

# Not a full validation, but just a quick check for a reasonable header:
my $is_likely_sereal = scalar_looks_like_sereal($some_string);
# or:
$is_likely_sereal = $decoder->looks_like_sereal($some_string);

DESCRIPTION

This library implements a deserializer for an efficient, compact-output, and feature-rich binary protocol called Sereal. Its sister module Sereal::Encoder implements an encoder for this format. The two are released separately to allow for independent and safer upgrading.

This distribution requires Perl 5.14 or newer.

This decoder supports Sereal protocol versions 1 through 7 and rejects unsupported protocol versions. Tags and compression formats that require a newer protocol version than the version declared in a document are rejected.

The protocol specification and many other bits of documentation can be found in the github repository. Right now, the specification is at https://github.com/Sereal/Sereal/blob/master/sereal_spec.pod, there is a discussion of the design objectives in https://github.com/Sereal/Sereal/blob/master/README.pod, and the output of our benchmarks can be seen at https://github.com/Sereal/Sereal/wiki/Sereal-Comparison-Graphs.

CLASS METHODS

get_sereal_checksum

my $tag = Sereal::Decoder::get_sereal_checksum($packet);
my $hex = defined($tag) ? unpack('H*', $tag) : undef;
my $tag = Sereal::Decoder::get_sereal_checksum($stream, $offset);

Returns the raw checksum bytes stored in the envelope, or undef for an unsigned packet. It checks framing and declared lengths, including the presence of the declared payload, but does not authenticate, decompress, decode, or run callbacks. Malformed framing raises an exception. It can inspect an unknown checksum type, whose tag length need not be 8 or 16 bytes. Retrieving a tag does not establish its validity. Bytes following the declared packet are ignored.

validate_sereal_checksum

my $valid = Sereal::Decoder::validate_sereal_checksum($packet);
my $valid = Sereal::Decoder::validate_sereal_checksum($packet, $key16);
my $valid = Sereal::Decoder::validate_sereal_checksum($stream, $key16, $offset);

Verifies packet authentication without decompressing or decoding its contents. Returns true for a valid tag and false for a mismatch or an unsigned packet. Malformed framing, unknown checksum types, missing keys for checksummed input, and keys of a size other than 16 bytes raise exceptions. An omitted key uses $Sereal::DefaultKey; an explicitly undefined key does not supply a key. The packet selects the SipHash variant. This function always verifies, regardless of checksum_policy or its environment default.

A valid checksum authenticates the protected bytes, not their validity as a Sereal value or their suitability for your application. Validation performs one MAC pass over the protected bytes. Decoding the packet afterward with verification enabled repeats that work. Both checksum functions support custom opcodes for direct calls and the same behavior through XSUB calls via code references. They can also be imported from the Sereal wrapper.

prepare_checksum_key

my $key16 = Sereal::Decoder::prepare_checksum_key($key);
my $key16 = Sereal::Decoder::prepare_checksum_key($key, $opt_key);
my $empty_key16 = Sereal::Decoder::prepare_checksum_key();

Returns a deterministic 16-byte normalization of a byte string using SipHash-2-4-128. It first computes SipHash-2-4-128 of Out damned spot! followed by the optional $opt_key, keyed by Call me Ishmael., then uses that result as the key to SipHash-2-4-128 of Luke, I am your father. followed by $key. Both arguments default to the empty string. This only normalizes the length; it does not add entropy or replace a password KDF.

siphash24, siphash24_128 and siphash36_128

my $tag8  = Sereal::Decoder::siphash24($bytes, $key16);
my $tag16 = Sereal::Decoder::siphash24_128($bytes, $key16);
my $tag16 = Sereal::Decoder::siphash36_128($bytes, $key16);
my $tag16_default = Sereal::Decoder::siphash36_128($bytes);

Compute a raw SipHash tag for a byte string and an exactly 16-byte key. siphash24() returns the eight-byte SipHash-2-4 result, siphash24_128() returns the 16-byte SipHash-2-4-128 result, and siphash36_128() returns the 16-byte SipHash-3-6-128 result. Each helper accepts an omitted key, which defaults to sixteen null bytes. Explicit keys must be exactly 16 bytes. These low-level helpers are provided for tooling that needs to inspect or verify Sereal checksum envelopes. The returned values are binary strings.

In SipHash-c-d, c is the compression round count per message block and d is the round count in each finalization stage. The 128 suffix specifies the tag size. See the authors' SipHash variant documentation.

new

Constructs a decoder object. Optionally takes a hash reference of options. Defaults are given below.

General Options

legacy_float_byteorder

Selects the source byte order for floating-point values in protocol versions 1 through 6. Accepted values are 'native' (the default), 'little', and 'big'. Set this when reading protocol v1 through v6 data from a machine with a different, known byte order. It has no effect on protocol v7.

This option does not convert between different native extended formats. See "FLOATING-POINT INTEROPERABILITY" for details.

allow_unknown_long_double

Allows protocol v7 native extended values with an unidentified layout to be read using the local long double representation. This option is off by default. Enable it only for data from a matching Perl build and ABI. Storage sizes must still match, and known format or byte-order mismatches remain errors.

my $decoder = Sereal::Decoder->new({ allow_unknown_long_double => 1 });

See "FLOATING-POINT INTEROPERABILITY" for the limits of this mode.

incremental

If set to a non-zero value (default: 0), then Sereal::Decoder will destructively parse Sereal documents out of a variable. Every time a Sereal document is successfully parsed it is removed from the front of the string it is parsed from. Input and output must be different scalars in this mode. When used with an offset, removal includes the prefix before that offset.

This means you can do this:

while (length $buffer) {
    my $data= decode_sereal($buffer,{incremental=>1});
}

alias_smallint

If set to a true value then Sereal::Decoder will share integers from -16 to 15 (encoded as either SRL_HDR_NEG and SRL_HDR_POS) as read-only aliases to a common SV.

The result of this may be significant space savings in data structures with many integers in the specified range. The cost is more memory used by the decoder and a very modest speed penalty when deserializing.

Note this option changes the structure of the dumped data. Use with caution.

See also the "alias_varint_under" option.

alias_varint_under

If set to a true positive integer smaller than 16 then this option is similar to setting "alias_smallint" and causes all integers from -16 to 15 to be shared as read-only aliases to the same SV, except that this treatment ALSO applies to SRL_HDR_VARINT. If set to a value larger than 16 then this applies to all varints varints under the value set. (In general SRL_HDR_VARINT is used only for integers larger than 15, and SRL_HDR_NEG and SRL_HDR_POS are used for -16 to -1 and 0 to 15 respectively.)

In simple terms if you want to share values larger than 16 then you should use this option, if you want to share only values in the -16 to 15 range then you should use the "alias_smallint" option instead.

The result of this may be significant space savings in data structures with many integers in the desire range. The cost is more memory used by the decoder and a very modest speed penalty when deserializing.

Note this option changes the structure of the dumped data. Use with caution.

use_undef

If set to a true value then this any undef value to be deserialized as PL_sv_undef. This may change the structure of the data structure being dumped, do not enable this unless you know what you are doing.

set_readonly

If set to a true value then the output will be completely readonly (deeply).

set_readonly_scalars

If set to a true value then scalars in the output will be readonly (deeply). References won't be readonly.

These options control authentication, object handling, input validation, and resource limits. See "ROBUSTNESS" for guidance on untrusted input.

checksum_key

A 16-byte key used to verify checksummed input. If omitted, the configured default key is used; explicit undef suppresses that default. See "CHECKSUM CONFIGURATION" for package and environment defaults.

The packet selects the checksum algorithm. checksum_policy controls verification and key-length validation.

Supplying multiple key options warns if their values are identical and throws an exception if they differ.

checksum_mode

Accepted for configurations shared with Encoder. Values are 'none' or 0, 'v1' or 1, 'v2' or 2, and 'v3' or 3. The packet determines the algorithm; use checksum_policy to control verification. This option and mode defaults do not change Decoder behavior.

checksum_policy

Controls checksum verification and whether unsigned input is accepted. The default is 'verify-if-available'. Accepted values are:

'verify-if-available' - verify when the checksum type is supported and a key is configured; reject a failed verification. Otherwise warn and decode without authentication.
'die' - reject checksummed packets that cannot be verified, including unsupported checksum types, missing keys and invalid tags. Unsigned packets are accepted.
'require_key' - require a key at construction and reject every packet that is unsigned or cannot be verified. 'require' and 'require_v1' are aliases.
'require_v2' - require a key and accept only verified packets using checksum mode v2 or v3.
'require_v3' - require a key and accept only verified packets using checksum mode v3.
'ignore' - skip verification silently, even when a key is configured.
'ignore-warn' - skip verification and warn for each checksummed packet.

The v1, v2 and v3 names refer to checksum modes. Policies other than the 'require' policies accept unsigned packets. 'die' and the 'require' policies require exactly 16 key bytes; other policies pad or truncate keys with a warning. Required checksum modes are checked before decompression or body decoding. See "PERFORMANCE" for verification costs.

checksum_key_v1

Compatibility alias for checksum_key. It can verify every supported checksum type; the suffix does not restrict accepted input.

checksum_key_v2

Compatibility alias for checksum_key. It can verify every supported checksum type; the suffix does not restrict accepted input.

checksum_key_v3

Compatibility alias for checksum_key. It can verify every supported checksum type; the suffix does not restrict accepted input.

refuse_objects

Rejects objects with an exception. This option is off by default. See "ROBUSTNESS" for object-handling considerations.

no_bless_objects

Decodes ordinary serialized objects without blessing them. This option is off by default. See no_thaw_objects to prevent THAW callbacks.

no_thaw_objects

Returns frozen objects as argument arrays blessed into Sereal::Decoder::THAW_args instead of calling THAW. This option is off by default. See "FREEZE/THAW CALLBACK MECHANISM" for the array layout.

thaw_allow_classes

Restricts which classes may invoke THAW. Accepts an array reference of class names, a hash reference with true values for allowed names, or a predicate $cb->($classname) called once per frozen object and returning true to allow the call. Without this option, all classes are allowed.

For a denied class, see "thaw_deny_action". This option has no effect when refuse_objects or no_thaw_objects prevents thawing. See "FREEZE/THAW CALLBACK MECHANISM" for callback behavior.

thaw_deny_action

Action when thaw_allow_classes denies a THAW call: 'croak' throws an exception (the default); 'raw' returns the argument array blessed into Sereal::Decoder::THAW_args, as no_thaw_objects does.

refuse_snappy

Rejects Snappy-compressed documents. This option is off by default.

refuse_zstd

Rejects Zstd-compressed documents. This option is off by default.

validate_utf8

Rejects invalid UTF-8 byte sequences in text strings. This option is off by default. Enable it when input must contain valid UTF-8.

max_recursion_depth

Maximum recursion depth, default 10000. The limit is approximate and depends on the structures encountered. Deep recursion can exhaust the C stack; raising this limit can cause a crash. See "ROBUSTNESS".

max_num_hash_entries

Maximum entries in a hash or hash-based object. Zero means unlimited (the default). Input exceeding the limit is rejected. See "ROBUSTNESS" for resource limits.

max_num_array_entries

Maximum entries in an array. Zero means unlimited (the default). Input exceeding the limit is rejected. See "ROBUSTNESS" for resource limits.

max_string_length

Maximum encoded string length in bytes. Zero means unlimited (the default). Input exceeding the limit is rejected. See "ROBUSTNESS" for resource limits.

max_uncompressed_size

Maximum uncompressed document size in bytes. Zero means unlimited (the default). Input exceeding the limit is rejected. See "ROBUSTNESS" for resource limits.

undef_deadly_policy

Controls the diagnostic UNDEF_DEADLY value introduced in protocol version 6. 'undef' decodes it silently as ordinary undef (the default); 'warn' warns and decodes it; 'die' throws an exception. An omitted or undefined option selects 'undef'; other values are rejected at construction. use_undef and reference tracking apply as for ordinary undef.

unknown_option_policy

Controls unrecognized constructor keys: 'warn' warns and continues (the default), 'die' throws an exception, and 'ignore' ignores them. Invalid policy values throw an exception.

SEREAL_UNKNOWN_OPTION_POLICY and SEREAL_DECODER_UNKNOWN_OPTION_POLICY initialize $Sereal::UnknownOptionPolicy and $Sereal::Decoder::UnknownOptionPolicy at module load if undefined. At construction, the first defined value wins: the option, the Decoder-specific package variable, then the shared package variable. Otherwise 'warn' is used.

INSTANCE METHODS

decode

Given a byte string of Sereal data, the decode call deserializes that data structure. The result can be obtained in one of two ways: decode accepts a second parameter, which is a scalar to write the result to, AND decode will return the resulting data structure.

The two are subtly different in case of data structures that contain references to the root element. In that case, the return value will be a (non-recursive) copy of the reference. The pass-in style is more correct. In other words,

$decoder->decode($sereal_string, my $out);
# is almost the same but safer than:
my $out = $decoder->decode($sereal_string);

This is an unfortunate side-effect of perls standard copy semantics of assignment. Possibly one day we will have an alternative to this.

Caller-supplied outputs are cleared before parsing begins. This releases their previous values and may run DESTROY; the input and output scalars remain retained during the reset.

Once parsing begins, application callbacks, including THAW methods, must not modify the input scalar. The decoder reads directly from its buffer. Input and output may otherwise use the same scalar; the decoder copies the input for this case. Incremental mode requires separate input and output.

decode_with_header

Given a byte string of Sereal data, the decode_with_header call deserializes that data structure as decode would do, however it also decodes the optional user data structure that can be embedded into a Sereal document, inside the header (see Sereal::Encoder::encode).

It accepts an optional second parameter, which is a scalar to write the body to, and an optional third parameter, which is a scalar to write the header to.

Regardless of the number of parameters received, decode_with_header returns an ArrayRef containing the deserialized header, and the deserialized body, in this order.

See decode for the subtle difference between the one, two and three parameters versions.

If there is no header in a Sereal document, corresponding variable or return value will be set to undef.

decode_only_header

Given a byte string of Sereal data, the decode_only_header deserializes only the optional user data structure that can be embedded into a Sereal document, inside the header (see Sereal::Encoder::encode).

It accepts an optional second parameter, which is a scalar to write the header to.

Regardless of the number of parameters received, decode_only_header returns the resulting data structure.

See decode for the subtle difference between the one and two parameters versions.

If there is no header in a Sereal document, corresponding variable or return value will be set to undef.

decode_with_offset

Same as the decode method, except as second parameter, you must pass an integer offset into the input string, at which the decoding is to start. The optional "pass-in" style scalar (see decode above) is relegated to being the third parameter.

decode_only_header_with_offset

Same as the decode_only_header method, except as second parameter, you must pass an integer offset into the input string, at which the decoding is to start. The optional "pass-in" style scalar (see decode_only_header above) is relegated to being the third parameter.

decode_with_header_and_offset

Same as the decode_with_header method, except as second parameter, you must pass an integer offset into the input string, at which the decoding is to start. The optional "pass-in" style scalars (see decode_with_header above) are relegated to being the third and fourth parameters.

bytes_consumed

After using the various decode methods documented previously, bytes_consumed can return the number of bytes from the body of the input string that were actually consumed by the decoder. That is, if you append random garbage to a valid Sereal document, decode will happily decode the data and ignore the garbage. If that is an error in your use case, you can use bytes_consumed to catch it.

my $out = $decoder->decode($sereal_string);
if (length($sereal_string) != $decoder->bytes_consumed) {
  die "Not all input data was consumed!";
}

Chances are that if you do this, you're violating UNIX philosophy in "be strict in what you emit but lenient in what you accept".

You can also use this to deserialize a list of Sereal documents that is concatenated into the same string (code not very robust...):

my @out;
my $pos = 0;
eval {
  while (1) {
    push @out, $decoder->decode_with_offset($sereal_string, $pos);
    $pos += $decoder->bytes_consumed;
    last if $pos >= length($sereal_string)
         or not $decoder->bytes_consumed;
  }
};

As mentioned, only the bytes consumed from the body are considered. So the following example is correct, as only the header is deserialized:

my $header = $decoder->decode_only_header($sereal_string);
my $count = $decoder->bytes_consumed;
# $count is 0

decode_from_file

Sereal::Decoder->decode_from_file($file);
$decoder->decode_from_file($file);

Read and decode the file specified. If called in list context and incremental mode is enabled then decodes all packets contained in the file and returns a list, otherwise decodes the first (or only) packet in the file. Accepts an optinal "target" variable as a second argument.

looks_like_sereal

Performs some rudimentary check to determine if the argument appears to be a valid Sereal packet or not. These tests are not comprehensive and a true result does not mean that the document is valid, merely that it appears to be valid. On the other hand a false result is always reliable.

The return of this method may be treated as a simple boolean but is in fact a more complex return. When the argument does not look anything like a Sereal document then the return is perl's FALSE, which has the property of being string equivalent to "" and numerically equivalent to 0. However when the argument appears to be a UTF-8 encoded protocol 3 Sereal document (by noticing that the \xF3 in the magic string has been replaced by \xC3\xB3) then it returns 0 (the number, which is string equivalent to "0"), and otherwise returns the protocol version of the document. This means you can write something like this:

$type= Sereal::Decoder->looks_like_sereal($thing);
if ($type eq '') {
    say "Not a Sereal document";
} elsif ($type eq '0') {
    say "Possibly utf8 encoded Sereal document";
} else {
    say "Sereal document version $type";
}

For reference, Sereal's magic value is a four byte string which is either =srl for protocol version 1 and 2 or =\xF3rl for protocol version 3 and later. This function checks that the magic string corresponds with the reported version number, as well as other checks, which may be enhanced in the future.

Note that looks_like_sereal() may be called as a class or object method, and may also be called as a single argument function. See the related scalar_looks_like_sereal() for a version which may ONLY be called as a function, not as a method (and which is typically much faster).

EXPORTABLE FUNCTIONS

sereal_decode_with_object

The functional interface that is equivalent to using decode. Takes a decoder object reference as first parameter, followed by a byte string to deserialize. Optionally takes a third parameter, which is the output scalar to write to. See the documentation for decode above for details.

This functional interface is marginally faster than the OO interface since it avoids method resolution overhead and, on sufficiently modern Perl versions, can usually avoid subroutine call overhead. See Sereal::Performance for a discussion on how to tune Sereal for maximum performance if you need to.

sereal_decode_with_header_with_object

The functional interface that is equivalent to using decode_with_header. Takes a decoder object reference as first parameter, followed by a byte string to deserialize. Optionally takes third and fourth parameters, which are the output scalars to write to. See the documentation for decode_with_header above for details.

sereal_decode_only_header_with_object

The functional interface that is equivalent to using decode_only_header. Takes a decoder object reference as first parameter, followed by a byte string to deserialize. Optionally takes a third parameters, which outputs scalars to write to. See the documentation for decode_with_header above for details.

sereal_decode_only_header_with_offset_with_object

The functional interface that is equivalent to using decode_only_header_with_offset. Same as the sereal_decode_only_header_with_object function, except as the third parameter, you must pass an integer offset into the input string, at which the decoding is to start. The optional "pass-in" style scalar (see sereal_decode_only_header_with_object above) is relegated to being the fourth parameter.

sereal_decode_with_header_and_offset_with_object

The functional interface that is equivalent to using decode_with_header_and_offset. Same as the sereal_decode_with_header_with_object function, except as the third parameter, you must pass an integer offset into the input string, at which the decoding is to start. The optional "pass-in" style scalars (see sereal_decode_with_header_with_object above) are relegated to being the fourth and fifth parameters.

sereal_decode_with_offset_with_object

The functional interface that is equivalent to using decode_with_offset. Same as the sereal_decode_with_object function, except as the third parameter, you must pass an integer offset into the input string, at which the decoding is to start. The optional "pass-in" style scalar (see sereal_decode_with_object above) is relegated to being the third parameter.

decode_sereal

The functional interface that is equivalent to using new and decode. Expects a byte string to deserialize as first argument, optionally followed by a hash reference of options (see documentation for new()). Finally, decode_sereal supports a third parameter, which is the output scalar to write to. See the documentation for decode above for details.

This functional interface is significantly slower than the OO interface since it cannot reuse the decoder object.

decode_sereal_with_header_data

The functional interface that is equivalent to using new and decode_with_header. Expects a byte string to deserialize as first argument, optionally followed by a hash reference of options (see documentation for new()). Finally, decode_sereal supports third and fourth parameters, which are the output scalars to write to. See the documentation for decode_with_header above for details.

This functional interface is significantly slower than the OO interface since it cannot reuse the decoder object.

scalar_looks_like_sereal

The functional interface that is equivalent to using looks_like_sereal.

Note that this version cannot be called as a method. It is normally executed as a custom opcode, as such errors about its usage may be caught at compile time, and it should be much faster than looks_like_sereal.

REGULAR EXPRESSIONS

The Decoder restores the regexp pattern and its matching modifiers, including the character set modes /d, /l, /u, /a and /aa, /n (no unnamed captures), and /xx (extended whitespace handling inside character classes). It also preserves /m, /s, /i, /x and /p.

/p first shipped in Perl 5.10; the character set modifiers in Perl 5.14; /n in Perl 5.22; and /xx in Perl 5.26. A modifier unavailable on the receiving Perl causes an exception naming the modifier and the required stable Perl version. Unknown modifier bytes and conflicting character set modes are rejected. These checks apply to every Sereal protocol version.

Preserving modifiers does not freeze the regexp engine, Unicode tables, or the execution locale. Those can still change matching behavior between hosts or Perl versions.

Perl 5.14 and 5.16 have a core regexp-copy lifetime bug: a scalar copy of a regexp can become invalid if the original regexp scalar is overwritten. Keep the original alive and unchanged while its scalar copies are in use, including when reusing an explicit decoder output scalar. This also affects regexps created by Perl's own eval and qr//; Perl 5.18 and later avoid it.

Perl 5.20 also has a core cleanup issue for read-only bare regexp values in lexicals: scope cleanup can raise Modification of a read-only value attempted. This occurs with regexps created by Perl itself as well. Prefer regexp references on that Perl when using read-only values.

FLOATING-POINT INTEROPERABILITY

Perl builds and numeric formats

Most Perl builds store floating-point numbers as double, normally IEEE 754 binary64. Long-double and quadmath builds support greater precision or range. Quadmath uses IEEE binary128; long double varies by platform. Perl's Config module reports its internal floating-point type (NV) in $Config{nvtype}. Platforms can also differ in byte order.

Encoding and decoding make separate decisions. The encoder chooses a stored representation from the producer's NV type. The decoder decides whether to accept that representation, then converts accepted values to its local NV type. Input options control interpretation and acceptance; they do not change the local NV type or increase its precision or range. A successful read can therefore involve rounding or loss of range.

Reading protocol v7

Protocol v7 uses little-endian IEEE binary32, binary64, and binary128 by default. The decoder handles byte order automatically and can read binary128 without quadmath support. No special option is needed.

Values are rounded to the local NV precision when necessary, using nearest/even rounding. Overflow produces signed infinity; underflow can produce a subnormal value or signed zero. NaN payload bits are not preserved. Use a Perl build with sufficient precision and range if these losses are unacceptable.

Reading native long doubles

A long-double producer can request native storage instead of portable storage. Protocol v7 identifies the native format, byte order, and storage size. The decoder rejects known layouts that do not match its own.

Unknown layouts are rejected by default. Set allow_unknown_long_double => 1 only when producer and reader use matching Perl builds and ABIs (native memory layouts). This trusts the stored bytes rather than converting them. The storage size must still match, and known format or byte-order mismatches remain errors. Only same-build value round trips are guaranteed for unknown layouts.

Reading protocol versions 1 through 6

These versions used the producer's native byte order without identifying it in the document. Extended values could also depend on the producer's native format and storage size. The decoder preserves the historical behavior by assuming its own native representation.

When reading data from a different platform, obtain the producer's byte order and set legacy_float_byteorder => 'little' or 'big' as needed. This corrects byte order only; it cannot convert between different native extended formats or storage sizes. Do not infer byte order from apparently reasonable decoded numbers: the wrong order can also produce plausible values. This option has no effect on protocol v7.

CHECKSUM CONFIGURATION

For per-object settings, see "checksum_key", "checksum_mode", and "checksum_policy". Package and environment defaults are described below.

Shared default key

$Sereal::DefaultKey supplies a shared default when no checksum key option is present. $Sereal::Decoder::DefaultKey can override the Decoder's default. Both contain raw key bytes, not hexadecimal text. A default Decoder key is used for all three supported checksum types. Explicit key options replace the default configuration; supplying an explicit undef key also suppresses the default.

The first load of Sereal::Encoder or Sereal::Decoder initializes this shared variable from SEREAL_DEFAULT_KEY, unless application code has already set it. The Decoder-specific SEREAL_DECODER_DEFAULT_KEY initializes $Sereal::Decoder::DefaultKey the same way. Constructor key options take precedence over the Decoder-specific default, which takes precedence over the shared default. Exactly 32 hexadecimal characters are decoded to 16 bytes. Other lengths or non-hexadecimal values are passed unchanged to prepare_checksum_key(), with a warning that does not include the secret. Normalization does not add entropy or turn a weak password into a strong key. If the environment variable is absent, no default key is installed. Loading the other module does not reset an initialized default, and subsequent environment changes do not affect it.

Code can set or localize the shared variable:

local $Sereal::DefaultKey = $key16;
my $decoder = Sereal::Decoder->new({ checksum_policy => 'require_key' });

New encoders and decoders copy its current value. Existing objects retain their keys. Localizing it to undef disables the default for new objects in that scope. Cached objects, including objects returned by the Sereal wrapper's cache helpers, retain the configuration they were created with.

Default policy

SEREAL_CHECKSUM_POLICY sets the shared default; SEREAL_DECODER_CHECKSUM_POLICY sets the Decoder-specific default. At module startup they initialize $Sereal::ChecksumPolicy and $Sereal::Decoder::ChecksumPolicy when those variables are undefined.

SEREAL_CHECKSUM_POLICY=require_key perl my_application.pl

The constructor option overrides the Decoder-specific package variable, which overrides the shared package variable. If none is defined, the policy is 'verify-if-available'. Invalid values cause construction to fail. Existing objects and recursive clones retain their configured policy.

ROBUSTNESS

The project-wide security policy, supported-version guidance, and private reporting instructions are at https://github.com/Sereal/Sereal/security/policy. Report suspected vulnerabilities through https://github.com/Sereal/Sereal/security/advisories/new.

The decoder validates packet structure and bounds before using packet data, and malformed input should normally produce a Perl exception. This is a security goal, not a guarantee that every malformed packet is safe or that the implementation has no vulnerabilities. Treat packets from untrusted or attacker-influenced sources as hostile input, keep Sereal::Decoder updated, and do not use successful decoding as a substitute for validating the resulting values in the application.

Decoding can consume substantial CPU and memory even when a packet is valid. The resource limits below default to unlimited (zero): max_num_hash_entries, max_num_array_entries, max_string_length, and max_uncompressed_size. Set limits appropriate to the application when decoding data whose size or producer is not controlled. The recursion limit defaults to 10000, which can still be too high for some Perl builds; lower it when appropriate. These decoder limits do not replace process-level CPU and memory limits. In particular, a checksum does not limit the cost of decoding a packet from someone who knows the key.

Protocol version 6 checksums authenticate only packets that use a checksum envelope. The default policy verifies when the matching key is configured; without one it warns and decodes without authentication. Use checksum_policy => 'require_key' with a matching key when every packet must be authenticated; it rejects unsigned packets and failed verification. 'die' rejects checksummed packets that cannot be verified but still accepts unsigned packets. Verification runs before decompression. Checksums are not encryption, do not prevent replay, and do not make data safe to deserialize when the key holder is malicious or compromised. Packets without a checksum have no cryptographic integrity check provided by Sereal.

Object handling can execute application code or cause later destructor code to run. Use refuse_objects when objects are not expected. For frozen objects, no_thaw_objects prevents THAW calls, while thaw_allow_classes can restrict which classes may run them. Do not assume that disabling module autoloading makes callbacks safe: already loaded classes can still have their THAW methods invoked unless these controls are used. See "FREEZE/THAW CALLBACK MECHANISM" for details.

FREEZE/THAW CALLBACK MECHANISM

Some objects do not lend themselves naturally to naive perl datastructure level serialization. For instance XS code might use a hidden structure that would not get serialized, or an object may contain volatile data like a filehandle that would not be reconstituted properly. To support cases like this Sereal supports a FREEZE and THAW api. When objects are serialized their FREEZE method is asked for a replacement representation, and when objects are deserialized their THAW method is asked to convert that replacement back to something useful.

For security reasons decoding an object will NOT autoload any modules to support THAW, however if the classes and methods are preloaded it will invoke THAW as required and an exception will be thrown if the class has not been loaded. It is possible to disable THAW in the decoder by using the no_thaw_objects option, which when true causes frozen objects to be blessed into a special utility class "Sereal::Decoder::THAW_args" for debugging. The objects themselves are an array, whose last member will contain the class name the arguments are for.

Prior to v4.024 the decoder had issues with frozen representations that contained other objects and did not define a specific order that items would be thawed and blessed, making it impractical to put an object inside of the frozen representation of another object.

As of v4.024 frozen representations may contain other objects, and the order in which they are thawed is defined to be LIFO, thus the arguments to a THAW method will themselves be thawed (if need be) before the call to the containing objects THAW method. Thawing occurs only after all simple objects have been blessed into their appropriate object form.

Each frozen argument array must belong to a single OBJECT_FREEZE or OBJECTV_FREEZE declaration. Conflicting declarations are rejected rather than replacing the class associated with that array. Further references to the same frozen object may use REFP normally.

The FREEZE/THAW mechanism is inspired by the equivalent mechanism in CBOR::XS. The general mechanism is documented in the A GENERIC OBJECT SERIALIATION PROTOCOL section of Types::Serialiser. Similar to CBOR using CBOR, Sereal uses the string Sereal as a serializer identifier for the callbacks.

PERFORMANCE

See Sereal::Performance for general performance and tuning information.

Checksum verification makes an additional pass over the protected header and encoded payload. Its CPU cost grows with their size and depends on the SipHash variant. Compressed payloads are verified before decompression. With a configured key, all verifying policies perform the same verification work for a given packet. Ignore policies skip authentication but still validate the envelope framing.

Reusing a decoder avoids repeated configuration work; each packet still requires verification. Warnings and their logging can also add per-packet cost.

THREAD-SAFETY

Sereal::Decoder is thread-safe on Perl 5.14 and higher. This means "thread-safe" in the sense that if you create a new thread, all Sereal::Decoder objects will become a reference to undef in the new thread. This might change in a future release to become a full clone of the decoder object.

DEVELOPMENT TESTING

The Git repository includes an ASan and libFuzzer harness for Decoder in Perl/author_tools/fuzz_decoder. It builds Decoder with sanitizer and coverage instrumentation and uses Sereal::Encoder to generate seed packets. It is a repository development tool, distributed separately from the CPAN modules, and is not run by make test.

See the harness README for requirements and build and run instructions.

BUGS, CONTACT AND SUPPORT

For reporting bugs, please use the github bug tracker at http://github.com/Sereal/Sereal/issues.

For support and discussion of Sereal, there are two Google Groups:

Announcements around Sereal (extremely low volume): https://groups.google.com/forum/?fromgroups#!forum/sereal-announce

Sereal development list: https://groups.google.com/forum/?fromgroups#!forum/sereal-dev

AUTHORS AND CONTRIBUTORS

Yves Orton <demerphq@gmail.com>

Damian Gryski

Steffen Mueller <cpan@steffen-mueller.net>

Rafaël Garcia-Suarez

Ævar Arnfjörð Bjarmason <avar@cpan.org>

Tim Bunce

Daniel Dragan <bulkdd@cpan.org> (Windows support and bugfixes)

Zefram

Borislav Nikolov

Ivan Kruglov <ivan.kruglov@yahoo.com>

Eric Herman <eric@freesa.org>

Some inspiration and code was taken from Marc Lehmann's excellent JSON::XS module due to obvious overlap in problem domain.

ACKNOWLEDGMENT

This module was originally developed for Booking.com. With approval from Booking.com, this module was generalized and published on CPAN, for which the authors would like to express their gratitude.

COPYRIGHT AND LICENSE

The license for the code in this distribution is the following, with the exceptions listed below:

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

Except portions taken from Marc Lehmann's code for the JSON::XS module, which is licensed under the same terms as this module. (Many thanks to Marc for inspiration, and code.)

Also except the code for Snappy compression library, whose license is reproduced below and which, to the best of our knowledge, is compatible with this module's license. The license for the enclosed Snappy code is:

Copyright 2011, Google Inc.
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

  * Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
  * Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
  * Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

To install Sereal::Decoder, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Sereal::Decoder

CPAN shell

perl -MCPAN -e shell
install Sereal::Decoder

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	Go to GitHub issues (only if GitHub is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

SYNOPSIS

DESCRIPTION

CLASS METHODS

get_sereal_checksum

validate_sereal_checksum

prepare_checksum_key

siphash24, siphash24_128 and siphash36_128

new

General Options

legacy_float_byteorder

allow_unknown_long_double

incremental

alias_smallint

alias_varint_under

use_undef

set_readonly

set_readonly_scalars

Security and Policy Related Options

checksum_key

checksum_mode

checksum_policy

checksum_key_v1

checksum_key_v2

checksum_key_v3

refuse_objects

no_bless_objects

no_thaw_objects

thaw_allow_classes

thaw_deny_action

refuse_snappy

refuse_zstd

validate_utf8

max_recursion_depth

max_num_hash_entries

max_num_array_entries

max_string_length

max_uncompressed_size

undef_deadly_policy

unknown_option_policy

INSTANCE METHODS

decode

decode_with_header

decode_only_header

decode_with_offset

decode_only_header_with_offset

decode_with_header_and_offset

bytes_consumed

decode_from_file

looks_like_sereal

EXPORTABLE FUNCTIONS

sereal_decode_with_object

sereal_decode_with_header_with_object

sereal_decode_only_header_with_object

sereal_decode_only_header_with_offset_with_object

sereal_decode_with_header_and_offset_with_object

sereal_decode_with_offset_with_object

decode_sereal

decode_sereal_with_header_data

scalar_looks_like_sereal

REGULAR EXPRESSIONS

FLOATING-POINT INTEROPERABILITY

Perl builds and numeric formats

Reading protocol v7

Reading native long doubles

Reading protocol versions 1 through 6

CHECKSUM CONFIGURATION

Shared default key

Default policy

ROBUSTNESS

FREEZE/THAW CALLBACK MECHANISM

PERFORMANCE

THREAD-SAFETY

DEVELOPMENT TESTING

BUGS, CONTACT AND SUPPORT

AUTHORS AND CONTRIBUTORS

ACKNOWLEDGMENT

COPYRIGHT AND LICENSE

Module Install Instructions

Keyboard Shortcuts