v0.4.0 is out, with SQL and automatic rollups

Keep years of statistics without keeping years of raw events

RollupDB is an aggregate database in one file. You say what has to survive, counts, sums, extremes, averages, variance and percentiles, and it folds seconds into minutes, hours and days as the data ages. The raw events go. The history stays.

How it works Quick start
Readings arrive into one-second buckets. As they age, RollupDB folds them into minute, hour and day buckets, each keeping count, sum, min, max, mean, variance and a percentile sketch, and the raw events are not stored.

15.4 MB

Five years of history from 2.8 million readings that took 207 MB as raw JSON lines

+0.37%

Growth in file size when the same five years got twice as many readings

Exact

Count, sum, min, max and mean after five years of rollups, checked against an outside reference

0

Acknowledged writes lost in 15 runs that killed the process mid-write

From the recorded test run in the code repository, on a 4-CPU Linux runner. Sizes are completed snapshot files.

Articles

project

Why the Project Is Called RollupDB

A rollup is what monitoring people have called this for decades. You take sixty one-second readings and roll them up into one minute. Then sixty minutes into an hour, and …

Try it in five minutes

One binary, no server and no CGo. The quick start creates a database, defines a stream, loads ten thousand readings and asks for their percentiles.

Quick start