measure

Replaces the input with metrics describing the input.

measure [real_time=bool, cumulative=bool]

Description

The measure operator yields metrics for each received batch of events or bytes using the following schema, respectively:

Events Metrics

type tenzir.measure.events = record{
  timestamp: time,
  events: uint64,
  schema_id: string,
  schema: string,
}

Bytes Metrics

type tenzir.measure.bytes = record{
  timestamp: time,
  bytes: uint64,
}

real_time = bool (optional)

Whether to emit metrics immediately with every batch, rather than buffering until the upstream operator stalls, i.e., is idle or waiting for further input.

The is especially useful when measure should emit data without latency.

cumulative = bool (optional)

Whether to emit running totals for the events and bytes fields rather than per-batch statistics.

Examples

Get the number of bytes read incrementally for a file

load_file "input.json"
measure

{timestamp: 2023-04-28T10:22:10.192322, bytes: 16384}
{timestamp: 2023-04-28T10:22:10.223612, bytes: 16384}
{timestamp: 2023-04-28T10:22:10.297169, bytes: 16384}
{timestamp: 2023-04-28T10:22:10.387172, bytes: 16384}
{timestamp: 2023-04-28T10:22:10.408171, bytes: 8232}

Get the number of events read incrementally from a file

load_file "eve.json"
read_suricata
measure

{
  timestamp: 2023-04-28T10:26:45.159885,
  events: 65536,
  schema_id: "d49102998baae44a",
  schema: "suricata.dns"
}
{
  timestamp: 2023-04-28T10:26:45.812321,
  events: 412,
  schema_id: "d49102998baae44a",
  schema: "suricata.dns"
}

Get the total number of events in a file, grouped by schema

load_file "eve.json"
read_suricata
measure
summarize schema, events=sum(events)

{schema: "suricata.dns", events: 65948}