Storage
Source: Packages/ORIXMonitorCore/Sources/ORIXMonitorCore/Storage/TimeSeriesStore.swift.
TimeSeriesStore is an actor around one SQLite database. It keeps numeric history in three retention tiers, answers downsampled range queries, reports gaps, and holds an event log (alerts, bookmarks, system and network events, process spikes).
Location
| Case | Path |
|---|---|
| Normal run | ~/Library/Application Support/ORIX Monitor/telemetry.sqlite (resolved with FileManager.urls(for: .applicationSupportDirectory, in: .userDomainMask)) |
-ORIXDataDir <dir> |
<dir>/telemetry.sqlite |
| Tests, benchmarks | TimeSeriesStore(url: nil) opens a private :memory: database |
The directory is created if missing. With WAL mode SQLite also creates telemetry.sqlite-wal and telemetry.sqlite-shm. .gitignore excludes all three patterns.
If the database cannot be opened, MonitorModel keeps running without a store (storeError is shown in Settings › History).
Connection settings
Applied at open (sqlite3_open_v2 with SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_FULLMUTEX):
PRAGMA journal_mode=WAL;
PRAGMA synchronous=NORMAL;
PRAGMA auto_vacuum=INCREMENTAL;Schema
CREATE TABLE IF NOT EXISTS raw(
metric INTEGER NOT NULL, ts INTEGER NOT NULL, v REAL NOT NULL,
PRIMARY KEY(metric, ts)) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS minute(
metric INTEGER NOT NULL, ts INTEGER NOT NULL,
mn REAL, mx REAL, av REAL, n INTEGER,
PRIMARY KEY(metric, ts)) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS hour(
metric INTEGER NOT NULL, ts INTEGER NOT NULL,
mn REAL, mx REAL, av REAL, n INTEGER,
PRIMARY KEY(metric, ts)) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS events(
id INTEGER PRIMARY KEY AUTOINCREMENT, ts REAL NOT NULL, kind TEXT NOT NULL,
severity INTEGER NOT NULL, metric INTEGER, code TEXT NOT NULL, payload TEXT NOT NULL);
CREATE INDEX IF NOT EXISTS events_ts ON events(ts);| Column | Meaning |
|---|---|
metric |
MetricID.rawValue (stable integer key: 1 = cpu.total, 10 = memory.used, …, 90 = self.cpu, 91 = self.memory) |
raw.ts |
Unix time in milliseconds (rounded) |
minute.ts, hour.ts |
Unix time in seconds of the start of the bucket (floored to 60 s / 3600 s) |
v |
The value in the metric's stored unit |
mn, mx, av, n |
Minimum, maximum, arithmetic mean and number of raw samples in the bucket |
events.ts |
Unix time in seconds (floating point) |
events.kind |
EventKind: alert, finding, bookmark, system, network, process |
events.severity |
Severity: 0 info, 1 warning, 2 critical |
events.payload |
JSON object of string → string |
There is no schema-version table and no migration code: the schema is the first and only version.
Retention tiers
RetentionPolicy (seconds each tier is kept):
Tier (StoreTier) |
Content | Default retention | Settings choices |
|---|---|---|---|
raw |
Every sample as produced by the scheduler | 6 hours | 1 h, 6 h, 24 h, 3 days |
minute |
One row per metric per minute: min, avg, max, count | 14 days | 7, 14, 30, 90 days |
hour |
One row per metric per hour: min, avg, max, count | 730 days (2 years) | 90 days, 1 year, 2 years, 5 years |
Events are pruned with the hour retention.
Write path
flowchart TD
A["append(values, date)"] --> F{"value is finite?"}
F -- "no" --> X["dropped"]
F -- "yes" --> P["pendingRaw buffer"]
F -- "yes" --> M["roll minute bucket"]
F -- "yes" --> H["roll hour bucket"]
M --> C["closedBuckets when the bucket start changes"]
H --> C
P --> FL["flush()"]
C --> FL
M --> FL
H --> FL
FL --> T["one BEGIN IMMEDIATE transaction"]
T --> R["INSERT OR REPLACE raw"]
T --> AG["INSERT OR REPLACE minute and hour"]
T --> PR{"more than 1 h since last prune?"}
PR -- "yes" --> PN["prune() and incremental_vacuum"]
append(_:at:)touches no disk. Each finite value is added topendingRawand to the runningBucket(start, min, max, sum, count) of its metric in both the minute and the hour tier. NaN and infinite values are dropped (nonFiniteValuesAreDropped).rollcomputes the bucket start asfloor(time / step) × step. If the metric's current bucket has a different start, it is moved toclosedBucketsand a new one begins.- Relaunch inside a bucket. When there is no in-memory bucket for a metric (first value after launch),
loadBucketreads the row already stored for that bucket start and continues it, so a relaunch within the same minute or hour merges with the earlier partial aggregate instead of overwriting it (historySurvivesRelaunchAndPartialBucketsMerge). flush(now:)writes, in one transaction, all pending raw rows, all closed buckets, and the current open buckets (INSERT OR REPLACE, so an open bucket is rewritten with its latest aggregate at each flush). The stored average issum / count.- The engine calls
flusheveryflushInterval(30 s) and onstop().series(...)andstatistics()also flush first, so a query never misses buffered samples.
Durability
At most one flush period of raw samples (30 s by default) is held in memory. A normal quit flushes through AppDelegate.applicationShouldTerminate → MonitorModel.shutdown() → TelemetryEngine.stop(). A crash or kill loses the unflushed samples; Scripts/bench.sh notes this explicitly.
Pruning
prune(now:) runs at most once an hour (checked at the end of flush), and immediately after setRetention(_:) resets the timer:
DELETE FROM raw WHERE ts < (now − retention.raw) × 1000;
DELETE FROM minute WHERE ts < now − retention.minute;
DELETE FROM hour WHERE ts < now − retention.hour;
DELETE FROM events WHERE ts < now − retention.hour;
PRAGMA incremental_vacuum;eraseAll() clears the in-memory buffers, deletes every row of the four tables and runs VACUUM. It is reached from Settings › History › "Erase All History…" behind a confirmation dialog.
Read path and downsampling
series(_:from:to:maxPoints:tier:now:) returns a Series { metric, tier, step, points, gaps } where each SeriesPoint is { time, min, max, avg, count } and time is the start of the bucket.
Tier selection
SeriesMath.tier(from:to:now:retention:), unless the caller forces a tier:
| Condition | Tier |
|---|---|
span ≤ 2 hours and from is within raw retention |
raw |
span ≤ 7 days and from is within minute retention |
minute |
| otherwise | hour |
So a one-hour range from two days ago is answered from the minute tier (raw data that old no longer exists).
Bucket width
SeriesMath.step(span:tier:maxPoints:rawInterval:):
native = (tier == raw) ? max(rawInterval, 1) : tier.step // 60 or 3600
wanted = span / maxPoints
step = wanted > native ? ceil(wanted / native) × native : nativeThe step is never finer than the tier and always a whole multiple of the tier's native resolution. rawInterval is kept in sync with the configured sampling interval by TelemetryEngine.update(_:). Raw buckets are never narrower than one second.
Examples (from stepNeverFinerThanTier): 1 h on the minute tier at 600 points → 60 s; 24 h on the minute tier at 600 points → 180 s; 2 h raw at a 2 s interval and 600 points → 12 s.
Aggregation
Raw tier:
SELECT (ts / :stepMs) * :stepMs, MIN(v), MAX(v), AVG(v), COUNT(*)
FROM raw WHERE metric=? AND ts>=? AND ts<? GROUP BY 1 ORDER BY 1;Minute and hour tiers:
SELECT (ts / :step) * :step, MIN(mn), MAX(mx), SUM(av * n) / SUM(n), SUM(n)
FROM <tier> WHERE metric=? AND ts>=? AND ts<? GROUP BY 1 ORDER BY 1;Properties that follow, and that downsamplingPreservesExtremesAndWeightedMean checks:
- Extremes survive. The min of mins and the max of maxes are exact, so a one-sample spike is still visible at any zoom level.
- Means are sample-weighted. A bucket with 30 samples weighs 30 times a bucket with one.
Series.averageapplies the same weighting across returned points. - Sample counts are preserved, which lets reports state how many measurements a figure rests on.
Callers and their point budgets: History page 480, reports 360, findings 2000, default 600.
Gap detection
SeriesMath.gaps(points:from:to:step:tolerance:) with tolerance = 2.5:
- No point at all → one gap covering the whole range.
first.time − from > 2.5 × step→ a leading gap[from, first.time].- For consecutive points with
b.time − a.time > 2.5 × step→ a gap[a.time + step, b.time]. to − (last.time + step) > 2.5 × step→ a trailing gap. The store passesmin(to, now)so the future part of a range is not reported as missing.
Gaps are reported, never filled. Their consumers:
| Consumer | Behaviour |
|---|---|
HistoryChart |
Shades each gap with a RectangleMark; splits the line and the min–max band into segments at holes wider than 2.5 steps |
| History page statistics | "Gaps: n · total duration" |
PDFReportRenderer.chart |
Shades gaps and breaks the line with the same 2.5-step rule |
| Markdown / PDF reports | "Data gaps" section: count, total missing, longest, per metric |
| JSON / CSV | JSON includes gaps per series; CSV simply has no rows for missing buckets |
Detectors.sustained |
Ends an episode when consecutive points are more than 3 steps apart |
A gap means "no sample was stored": the app was not running, the Mac was asleep, sampling was paused, history recording was off, or the metric could not be measured (for example a rate just after a discontinuity).
Events
| Method | Purpose |
|---|---|
record(_:) |
Inserts a StoredEvent; returns the row id |
events(kinds:from:to:limit:) |
Newest first, filtered by kind and time, default limit 500 |
deleteEvent(id:) |
Removes one event (bookmark deletion) |
deleteEvents(kinds:) |
Removes all events of the given kinds (alert history "Clear History") |
Event codes written by the app are listed in telemetry. The finding kind exists in EventKind but nothing writes events of that kind: findings are recomputed from history on demand and are not persisted.
Statistics
statistics() returns StoreStatistics { rawRows, minuteRows, hourRows, events, fileBytes, earliest }. fileBytes is the size of the database file plus its -wal file; earliest is the oldest hour bucket. Shown in Settings › History and used for the History page subtitle ("Collected locally since …").
Limitations
- No encryption. The database is a plain SQLite file protected by the user account's file permissions. It contains numeric series and event payloads (process names in
process.spike, interface names in network events, bookmark notes); it contains no IP addresses and no per-process history. - SQL errors are not surfaced.
exec,runandprepareignore failures after open; a failed write is silent. - Timestamps are wall-clock. A manual clock change moves where subsequent rows land. Rates are unaffected (they use the monotonic clock), but a backwards clock jump can overwrite rows with identical
(metric, ts)keys. - No migration path exists yet for future schema changes.
- Per-process data is not stored, only the process count and
process.spikeevents. - Minute and hour buckets are aligned to Unix time, not to the local time zone; hour buckets therefore start on UTC hour boundaries.