Reference

A short map of RollupDB 0.4.1. The full contracts, with every limit and edge case, are in the docs folder of the code repository.

Commands

Stream Definitions

Changing a Stream

update-stream adds the stream’s next version ("version": 2, then 3). A new version may change each tier’s retain_seconds and grace_seconds, max_groups and statistics, and it may add a sketch to a stream that has none. The name, effective_at, dimensions and tier widths stay, since the retained buckets were built with them; anything else is refused.

The change covers the whole history at once. Every bucket keeps its full exact state, so a statistic added later works on old data too. Shorter retention applies at the next rollup, and what’s already expired stays gone. A sketch added later covers buckets that start after the change; percentiles over older ranges come back as not_retained.

Readings and Queries

A reading is {"time": "...", "value": 12.5, "dimensions": {"service": "web"}}. Time is RFC 3339, the value is a finite number, and a reading later than its bucket’s grace window is refused and counted, never quietly merged somewhere else.

A query names a stream and a time range aligned to the finest tier, with an optional filter on dimensions and up to 32 quantiles as fractions (0.99, not 99). The answer says which resolutions it used, what it covered and what’s missing. Ranges that cut through a coarse bucket are refused rather than guessed.

Exact and Approximate

Count, sum, min, max and mean are exact, and count is a true integer. Variance is kept in a form that merges correctly and is accurate to floating-point limits. Percentiles come from a KLL sketch: approximate, with the error model returned in every answer. In the recorded test the worst observed rank error was 0.5%, against a model of 1.3%. The test results

Automatic Rollups

rollupdb serve rolls up every stream once a minute, and --advance-every changes that (0 turns it off). An embedding gets the same with db.RunRetention(ctx, every, report). advance FILE without a stream name, db.AdvanceAll and POST /v1/advance do one pass over every stream.

SQL

Each stream is a view named after it (or stream_ plus its name, if the name is taken by one of the file’s own tables), one row per retained bucket per group, with columns start, end, width_seconds, tier, one per dimension, count, sum, min, max, mean, m2 and sketch. Buckets never overlap, so sum(count), sum(sum), min(min) and max(max) are correct over any rows, and three aggregates merge the rest: rollup_variance(count, mean, m2), rollup_variance_sample(count, mean, m2) and rollup_quantile(sketch, rank).

SELECT service, substr(start, 1, 10) AS day,
       sum(count) AS readings, sum(sum) / sum(count) AS mean_ms,
       rollup_quantile(sketch, 0.99) AS p99_ms
FROM latency WHERE start >= '2026-10-01T00:00:00Z'
GROUP BY service, day ORDER BY service, day;

One statement at a time, starting with SELECT, WITH or VALUES, read-only, at most 10,000 rows. SQL sees what’s retained now; for coverage and gaps, use a query. The SQL notes

HTTP

rollupdb serve listens on a loopback address only, with no authentication; it’s for programs on the same machine.

POST bodies need Content-Type: application/json. Errors come back as {"error": {"code": "...", "message": "..."}}.

Go Library

db, err := rollupdb.Create(ctx, "metrics.rollupdb") // or rollupdb.Open
err = db.CreateStream(ctx, definition)
result, err := db.Ingest(ctx, batch)
answer, err := db.Query(ctx, query)
rolled, err := db.Advance(ctx, "latency")
all, err := db.AdvanceAll(ctx)              // or db.RunRetention(ctx, time.Minute, nil)
rows, err := db.SQL(ctx, "SELECT sum(count) FROM latency")
err = db.UpdateStream(ctx, nextVersion)
err = db.Snapshot(ctx, "backup.rollupdb")
err = db.Close()

WithClock injects a clock for replay and tests. Use one owning process per file.

The Clock

RollupDB trusts the host clock to decide what’s late and what’s old enough to roll up. If the clock jumps more than 366 days past a stream’s last write, ingest and advance refuse with ErrClockJump (HTTP 409 clock_jump) and write nothing, so a wrong clock can’t expire history early. If a stream really sat idle that long, raise the limit once with ROLLUPDB_MAX_CLOCK_STEP (a Go duration such as 9000h, or 0 for none) or WithMaxClockStep.