NAME
Punk::Observe::Query - one query language over traces, metrics and logs
SYNOPSIS
metric http.server.duration
| where service = "api" and http.route = "/checkout"
| rate(5m) by http.response.status_code
| p95
log {service="api"}
| where severity >= error
| search "connection refused"
| count by service
trace
| where duration > 500ms and status = "error"
| slowest 20DESCRIPTION
A source, then stages. Every stage takes a stream of rows and returns a stream of rows, so where, by, count and the rest are one implementation each - they work on a log because a log line is a row, and on a span because a span is a row.
The signals differ only in which columns exist.
THE CROSS-SIGNAL PIPELINE
Three stages re-key the stream, and they are the reason this is one language rather than three:
metric http.server.duration | p99 by http.route
| exemplars # the trace ids recorded alongside the spike
| traces # those traces, in full
| logs # every log line correlated by trace_idA spike, to the traces that caused it, to the lines those traces logged, in one expression.
| traces needs a trace identifier on its rows. Logs and spans have one and | exemplars produces one; a bare metric stream does not. So metric x | traces is a parse error, and the error says to add | exemplars first - this is the feature, so it has to be discoverable by typing rather than by reading this page.
GRAMMAR
query := source selector? pipeline?
source := 'metric' NAME | 'log' | 'trace' | 'spans'
selector := '{' cmp (',' cmp)* '}' # sugar for a leading where
pipeline := ('|' stage)+
stage := 'where' expr
| 'search' STRING
| agg ('by' field (',' field)*)?
| 'bucket' '(' DURATION ')' agg? ('by' field (',' field)*)?
| 'rate' '(' DURATION ')' agg? ('by' field (',' field)*)?
| 'top' INT 'by' agg
| 'slowest' INT
| 'limit' INT
| 'sort' field ('asc' | 'desc')?
| 'exemplars' | 'traces' | 'logs' | 'spans'
agg := 'count' | 'sum' | 'avg' | 'min' | 'max'
| 'p50' | 'p90' | 'p95' | 'p99' | 'distinct'
expr := expr ('and' | 'or') expr | 'not' expr | '(' expr ')' | cmp
cmp := field OP value
OP := '=' | '!=' | '<' | '<=' | '>' | '>=' | '=~' | '!~'
value := STRING | NUMBER | DURATION | SEVERITYnot binds tightest, then and, then or.
An aggregate carries its own grouping: count by service is one stage.
Bucketing over time
bucket(1m) cuts the range into equal spans and aggregates within each, so the answer is a series over time rather than one number. Without an aggregate it counts, which is the histogram of arrivals:
log | bucket(1m) count by severity
metric http.server.duration | bucket(5m) p95 by http.routerate(5m) is the same stage with the answer divided by the span, so it reads per second and does not change when the window widens. Only count and sum are divided: a percentile per second is not a quantity, and dividing one would report a service getting faster because somebody chose a wider bucket.
Boundaries are aligned to the epoch, not to the query. A bucket covers the same span whoever asks for it, so panning a chart does not move the boundaries, and two panels over slightly different ranges agree about the minutes they share.
A bucket with nothing in it is absent rather than zero. For a count zero would be right; for a percentile it would be invented, because no samples is undefined rather than nought. What a gap means belongs to the caller, which is the only party that knows the range that was asked for.
Durations
First-class tokens, not function calls: 500ms, 1.5s, 5m, 2h, 7d, 1w. Also ns and us. There is deliberately no unit for a month, because 1m meaning a month somewhere would be a trap nobody recovers from.
Severities
trace, debug, info, warn, error, fatal, case-insensitive. severity >= error is a numeric comparison on OTLP's twenty-four point scale, not a string match on a level name.
Columns
every source t, service, and any attribute
metric value
log body, severity, trace_id, span_id
trace, spans duration, name, status, kind, trace_id, span_idUsing a column that belongs to another signal is a parse error naming it, never an empty result. An empty result for a nonsensical query is the worst outcome available, because it looks like an answer.
A name that is not a reserved column is an attribute, and attributes are accepted on any source.
Values are quoted
where service = "api", not where service = api. A bare word is ambiguous with a column reference, and accepting it would turn a mistyped column name into a comparison that silently never matches. The error says so.
FUNCTIONS
Parsing only. To plan and run a query, see "run" in Punk::Observe::Exec.
parse
my $q = Punk::Observe::Query::parse($source);Parses a query and returns its syntax tree.
On failure:
{ ok => 0, error => "...", offset => 24 }offset is the byte position in $source where the parse stopped, so an interface can point at the character rather than repeat the query back.
On success:
{
ok => 1,
source => 'metric',
name => 'http.server.duration',
selector => { ... },
stages => [ { kind => 'where', expr => { ... } }, ... ],
}source is metric, log, trace or spans. name is present only for a metric source, and selector only where one was given - it is sugar for a leading where and parses to the same expression shape.
Each stage carries kind, and then whichever of these the stage has:
expr an expression tree, for where and the selector
text the literal, for search
agg the aggregate name, for agg and top
window the window in nanoseconds, for rate
n the count, for top, slowest and limit
desc present and true for a descending sort
fields the grouping or sort fields, as an arrayrefkind is one of where, search, by, agg, rate, top, slowest, limit, sort, exemplars, traces, logs or spans.
An expression is a tree of comparisons joined by and, or and not. A duration literal has already become nanoseconds and a severity name has already become its number on OTLP's twenty-four point scale, so a consumer never parses either again.
parse_free_cycles
my $ok = Punk::Observe::Query::parse_free_cycles($source, $n);Parses and frees $n times, returning how many parses succeeded. The syntax tree is bump-allocated and released in one go, and this is what asserts that holds under repetition: memory that grows across a few hundred thousand cycles is a leak in the parser, on the one path a hostile caller can drive at will.