/ 
Linux-Event-0.117
/ Linux::Event::Framer::Varint

NAME

Linux::Event::Framer::Varint - Frame messages with an unsigned LEB128 payload length

SYNOPSIS

package CompactStream;

use parent 'Linux::Event::IO::Sock::Stream';
use Linux::Event::Framer 'Varint',
    max_frame => 1_048_576;

sub on_message ($self, $message) {
    ...
}

DESCRIPTION

Linux::Event::Framer::Varint prefixes every message with its payload length encoded as canonical unsigned LEB128.

Small lengths use fewer prefix bytes.

For example:

payload length 0      prefix 00
payload length 1      prefix 01
payload length 127    prefix 7f
payload length 128    prefix 80 01

The encoded number is the payload length in bytes.

The payload follows immediately after the variable-width prefix.

WHAT IS LEB128?

LEB128 stores an integer in groups of seven data bits.

The high bit of each prefix byte indicates whether another prefix byte follows.

For example, decimal 128 is encoded as:

80 01

The first byte has its continuation bit set, so the parser knows another prefix byte follows.

The second byte completes the length.

Applications normally do not need to encode or decode this themselves. send and the native inbound parser handle it automatically.

DECLARING THE FRAMER

Declare Varint after the ordered-byte parent class:

package CompactStream;

use parent 'Linux::Event::IO::Sock::Stream';
use Linux::Event::Framer 'Varint';

The same framing policy can also be used with ordered-byte Pipe and TTY subclasses.

RECEIVING

Linux::Event reads enough prefix bytes to decode one canonical unsigned LEB128 length.

It then waits until the declared number of payload bytes are available.

For example, a 128-byte payload begins with:

80 01

If Linux receives:

first read:   80
second read:  01 <some payload bytes>
later reads: <remaining payload bytes>

the message is not delivered until all 128 payload bytes are present.

The application does not need to preserve partial prefix or payload state.

VARIABLE PREFIX WIDTH

Unlike LengthPrefix, the number of prefix bytes is not fixed.

Smaller values require less wire space.

For example:

5       05
127     7f
128     80 01
256     80 02
16_383  ff 7f
16_384  80 80 01

This makes Varint useful for protocols where small messages are common and a fixed four-byte length prefix would be unnecessarily large.

ZERO-LENGTH MESSAGES

The canonical encoding of zero is one byte:

00

Therefore:

$self->send('');

produces:

00

with no payload bytes following it.

Inbound 00 likewise represents one empty message.

CANONICAL ENCODING

Linux::Event accepts only the canonical unsigned LEB128 representation of a length.

For example, zero must be encoded as:

00

An overlong representation such as:

80 00

is rejected even though it could mathematically decode to the same numeric value.

This gives every payload length one normal wire representation.

PREFIX LENGTH AND OVERFLOW

Linux::Event bounds the Varint prefix parser.

The native parser accepts an unsigned LEB128 wire value of at most 64 bits and uses no more than ten prefix bytes.

Malformed prefixes that:

  • continue for too many bytes

  • overflow the supported unsigned value

  • exceed the native Perl unsigned integer range

  • use a noncanonical overlong representation

are rejected as framing errors.

The parser does not continue consuming an arbitrary number of continuation bytes.

INCLUDING THE PREFIX

include_prefix

By default, on_message receives only the payload.

For example, a 128-byte payload beginning on the wire with:

80 01

is delivered as exactly those 128 payload bytes.

To include the encoded Varint prefix in the inbound message:

use Linux::Event::Framer 'Varint',
    include_prefix => 1;

the callback receives:

prefix bytes + payload bytes

For a 128-byte payload, the delivered message therefore begins with:

80 01

and has a total size of 130 bytes.

include_prefix affects inbound delivery only.

LIMITING PAYLOAD SIZE

max_frame

An optional max_frame limits the decoded payload length:

use Linux::Event::Framer 'Varint',
    max_frame => 1_048_576;

When the prefix has been decoded, Linux::Event checks the declared payload length immediately.

If it exceeds max_frame, a framing error is reported without waiting for the oversized payload to arrive.

send also rejects payloads larger than max_frame.

max_frame must be a non-negative integer.

MAX_FRAME AND MAX_BUFFER

max_frame limits the protocol payload.

The ordered-byte resource's max_buffer separately limits native input storage.

A complete Varint frame includes:

variable-width prefix
payload

so the resource buffer limit also protects the complete framed representation.

SENDING

send calculates the payload byte length and prepends its canonical unsigned LEB128 encoding.

For example:

$self->send('x' x 127);

starts the wire frame with:

7f

while:

$self->send('x' x 128);

starts it with:

80 01

The application supplies only the payload.

RAW WRITE

write bypasses framing:

$self->write($bytes);

For example:

$self->write("\x80\x01" . ('x' x 128));

writes that exact wire representation.

Ordinary Varint protocol output should normally use send so the length prefix is guaranteed to match the payload.

INCLUDE_PREFIX DOES NOT CHANGE SENDING

include_prefix affects only inbound delivery.

For example:

use Linux::Event::Framer 'Varint',
    include_prefix => 1;

still means:

$self->send('x' x 128);

creates:

80 01 <128 payload bytes>

The application should not prepend the Varint itself before calling send.

BYTE LENGTH, NOT CHARACTER COUNT

The Varint represents payload bytes.

This framer does not define text encoding or serialization.

If the application sends text, it is responsible for converting that text into the protocol's intended byte encoding before calling send.

MULTIPLE MESSAGES MAY ARRIVE AT ONCE

One kernel read may contain several complete Varint-framed messages.

For example, conceptually:

<length><payload><length><payload><length><payload>

can produce several consecutive on_message callbacks from one native input drain.

Kernel read boundaries do not become application message boundaries.

MALFORMED INPUT

Linux::Event rejects malformed Varint prefixes rather than attempting to repair them.

Examples include:

  • an overlong encoding such as 80 00 for zero

  • a prefix that continues beyond the supported maximum width

  • a numeric overflow

  • a decoded length larger than max_frame

  • a complete frame that violates the ordered-byte buffer limit

These use the normal ordered-byte framing error path.

ERROR BEHAVIOR

Malformed or oversized inbound framing produces a Linux::Event::Error with type:

framing

The resource then follows its ordinary framing-error lifecycle.

Outbound send rejects a payload that violates max_frame rather than emitting an invalid frame.

WHEN TO USE VARINT

Varint framing is useful when:

  • the protocol specifies unsigned LEB128 lengths

  • message sizes vary substantially

  • small messages are common

  • saving prefix bytes on small messages is useful

  • payload contents must remain binary-safe

If the protocol uses a fixed one-, two-, or four-byte integer length instead, use Linux::Event::Framer::LengthPrefix.

If it specifically uses a four-byte network-order length, use Linux::Event::Framer::U32BE.

PERFORMANCE MODEL

Inbound Varint decoding runs in Linux::Event's native ordered-byte parser.

The parser maintains partial prefix and payload state without repeatedly crossing into Perl.

Canonical-form validation, overflow checks, max_frame enforcement, and frame boundary detection happen before semantic message delivery.

Outbound send has a one-byte fast path for payload lengths below 128 and generates additional LEB128 bytes only when required.

SEE ALSO

Linux::Event::Framer, Linux::Event::Framer::LengthPrefix, Linux::Event::Framer::U32BE, Linux::Event::Framer::Netstring, Linux::Event::Framer::DecimalLength, Linux::Event::IO::Sock::Stream, Linux::Event::IO::Pipe, Linux::Event::IO::TTY, docs/FRAMING.md, docs/CHOOSING-A-FRAMER.md.