/ 
Punk-Observe-0.01
/ Punk::Observe::Exec

NAME

Punk::Observe::Exec - planning and running a query

SYNOPSIS

use Punk::Observe::Exec;

my @rows = (
    { kind => 'span', t => 1, duration => 700_000_000,
      service => 'api', status => 2 },
    { kind => 'log',  t => 2, body => 'connection refused',
      severity => 17, service => 'api' },
);

my $r = Punk::Observe::Exec::run(
    'trace | where duration > 500ms | slowest 20', \@rows, {});

die "$r->{stage}: $r->{error}" unless $r->{ok};
warn 'partial answer' if $r->{meta}{truncated};

DESCRIPTION

A query is parsed by Punk::Observe::Query, planned, and then executed in steps.

It yields

A worker holds hundreds of connections. A query that scans two gigabytes synchronously stalls every one of them, and what the operator sees is that the service froze because somebody opened a dashboard.

So the executor is a resumable state machine rather than a function: it processes a bounded number of rows and returns, and the caller drives it from a timer so the event loop runs in between. Yielding is invisible in the answer - the same query over the same data gives the same result whatever the step budget.

It refuses

Estimating a query's cost and refusing it is the fourth thing the planner does, and a refusal carries what to add rather than what went wrong:

this query would scan too much - try narrowing the time range, as in
| where t > ...

A refused query with an actionable message is a better product than a thirty-second one, and far better than a timeout.

The result is honest

meta is never optional and never absent. It carries how many rows were scanned, whether a budget cut the answer short, and whether a percentile is exact.

A truncated result that looks complete is the observability equivalent of a green dashboard over dropped spans - which is the failure this whole project exists to stop. A partial answer is the correct prefix of the real one, and it says so.

THE ROW

Rows are hashrefs. kind is metric, span or log, defaulting to log.

kind       metric, span or log
t          event time, unix nanoseconds
duration   nanoseconds, spans
value      a number, metrics
body       the text, logs
severity   OTLP's 24-point scale, logs
status     the OTLP status code, spans
service    the service name
trace_hi   the trace id, high 8 bytes
trace_lo   the trace id, low 8 bytes
span_id    the span id
attrs      a hashref of attributes

There is one row shape for all three signals, which is what makes where, by and count one implementation each rather than three.

FUNCTIONS

run

my $r = Punk::Observe::Exec::run($query, \@rows, \%opts);

Parses, plans and runs a query to completion.

Options:

max_rows

The planner's budget. A query estimated to scan more than this is refused before it runs.

rows_available

How many rows the planner should believe exist, for estimating against max_rows.

step

Rows processed per step. Zero runs to completion in one step.

hard_max

An absolute ceiling on rows scanned. Reaching it truncates the answer and sets meta->{truncated}.

On failure:

{ ok => 0, stage => 'parse', error => "..." }

stage is parse or plan. The error is the actionable message, not a diagnostic.

On success:

{
  ok    => 1,
  shape => 'rows',
  rows  => [ ... ],       # when shape is rows
  groups => [ ... ],      # when shape is series or scalar
  meta  => {
    scanned_rows  => 4096,
    scanned_bytes => 262144,
    truncated     => 0,
    degraded      => 0,
    exact         => 1,
    steps         => 1,
  },
}

shape is rows for a query that yields rows, series or scalar for one that aggregates, and buckets for one that aggregates over time. A rows answer carries rows, each with t and whichever of body, service, severity, duration, value, trace_hi and trace_lo the row has. An aggregated answer carries groups, each with key, value and count.

A buckets answer carries bucket_ns and series, one entry per group, each with a key and its points in time order:

{
  shape     => 'buckets',
  bucket_ns => '60000000000',
  series    => [
    { key => 'api', points => [ [ $t, $value, $count ], ... ] },
  ],
}

$t is the instant the bucket starts, as a decimal string like every other instant here. A bucket with no rows is absent rather than present and zero - see "Bucketing over time" in Punk::Observe::Query.

A query asking for more series than can be drawn is refused rather than answered short, because a chart that simply stops reads as a service that went quiet. The refusal carries meta, so how much was scanned before it gave up is still visible.

Read meta before trusting the answer. truncated means a budget cut it short, degraded means a source could not be read in full, and exact is false when a percentile was estimated rather than computed.

steps

my $r = Punk::Observe::Exec::steps($query, \@rows, $step);

Drives the executor one step at a time and reports the cursor after each, so that yielding can be observed rather than assumed.

{ cursors => [ 100, 200, 300, 300 ], steps => 4, scanned => 300, rows => 12 }

cursors is the scan position after each step. With a step budget below the row count it must have more than one entry, and its last value must equal scanned. Parse and plan failures are fatal here rather than returned.

join

my $rows = Punk::Observe::Exec::join(\@left, \@right, $edge);

The cross-signal join: every row of @right whose trace identifier appears in @left. Rows without trace_hi and trace_lo on either side are skipped.

$edge is traces, logs or spans, and anything else is fatal.

This is not a general relational join, deliberately. There are exactly three named edges, and they are the three that physically exist in the data:

metric -> traces   the exemplar's trace id
traces -> logs     LogRecord.trace_id
traces -> spans    the same trace, already contiguous

A general join across three columnar stores with no shared key would need a planner nobody can predict and a cost model nobody can debug. Refusing it is the design.

SEE ALSO

Punk::Observe, Punk::Observe::Query