SPB Git

spb/findata

Public
3.0 KB

# FinData OS — STATUS (updated 2026-11-01, build session)

Legend: ✅ done and verified · 🟡 done, not yet verified in production · ⛔ blocked · ⬜ not started

# Phase 0 — Discover ✅

  • ✅ Cluster/SSD inventory, node choice M3U96a with rationale (docs/DISCOVERY.md §1)
  • ✅ Vendor identified (FirstRate Data), auth model, capability matrix, license summary, sanitized probes (docs/evidence/phase0-probes.md)
  • ✅ Minimal authorized test requests (last_update ×7 types, ticker listings, one 15.8 MB daily 1-min archive, splits/dividends, fundamentals JSON)
  • ✅ Capacity model and disk budget (≤ 250 GB curated, 60 GB hard stop)

# Phase 1 — Foundation

  • ✅ Monorepo (uv workspace, 8 packages), migrations (findata migrate), secrets in .env only, redaction filter
  • ✅ Authentication: owner password (argon2) + TOTP MFA, sessions, CSRF, scoped API keys; tests green
  • 🟡 Deployment via mld to M3U96a, TLS via MacLustr Tunnel, health checks — see "Production" below
  • ✅ CI-equivalent: make test (37 tests), make lint (ruff clean), pip-audit (no known vulnerabilities, 2026-11-01)

# Phase 2 — Market pipeline (US stocks, 1-minute)

  • ✅ Connector (discover/capabilities/fetch_range/normalize/validate/checkpoint), immutable raw archives, delta Parquet per session, atomic versioned publication, supersede without duplicates (tested), 5-minute derivation (tested vs manual aggregation and DST), coverage + quality anomalies
  • 🟡 Real incremental run on the node (see Production)

# Phase 3 — Fundamentals

  • ✅ FirstRate fundamentals connector (verified live on AAPL during discovery), long-format PIT facts, available_at policy, revision/restatement tracking, as-of query hides later filings and later-observed restatements (tested), earnings events
  • ⬜ SEC EDGAR connector: parsing implemented + fixture-tested; pending configuration (SEC_EDGAR_USER_AGENT)
  • ⚠️ Limitation: vendor gives no revision history → restatements before first ingestion are unknown

# Phase 4 — Platform

  • ✅ REST API /api/v1 (38+ routes, OpenAPI), cursor pagination, Arrow/CSV/JSON, async Parquet exports, sandboxed SQL, coverage/quality/health/admin, Prometheus metrics
  • ✅ Python SDK (bars via Arrow pages, exports, fundamentals, SQL) — tested against a live uvicorn thread
  • 🟡 Web console (Next.js static export) — built by a sub-agent; see Production for verification

# Phase 5 — Scale

  • 🟡 Representative multi-year backfill (bounded letter range) — to run on the node
  • 🟡 Backup + restore drill — to run on the node
  • ⬜ Load benchmark

# Phase 6 — AI and polish

  • ✅ Assistant endpoint implemented (Anthropic SDK, model proposes SQL only; FinData validates/executes/cites versions) — pending configuration until ANTHROPIC_API_KEY is set and a capability/cost test is recorded here; OpenAI provider not implemented
  • ⬜ Security review pass on production

# Production (to be filled with real outputs)

(see bottom of this file after deployment)

Files · 0467d449e66a5e62fb10ba4fcbf2f09fc3d1ef38
Menu

Theme