SPB Git
spboucher.ai

spb/findata

Public
History of services/ingest/src Clear filter
page 1

  1. Mon, Nov 2, 2026 6

    1. scheduler: daily dedupe keys fire once (quality/gc were re-enqueued every tick); backup scope documented
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 2 files +26 −23
    2. publish: chained O(delta) snapshot id; backfill batches 20 symbols per version; heartbeat-based stale requeue; symbol index
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 4 files +58 −18
    3. options parser: ragged lines tolerated, malformed files skipped
      Simon-Pierre Boucher · 1 file +1 −1
    4. parser: tolerate ragged vendor lines; skip unparseable files instead of failing the archive
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +5 −1
    5. fundamentals: remember 404 listings in the checkpoint so resumes do not retry them
      Simon-Pierre Boucher · 1 file +3 −1
    6. health: catalog-based lake usage and 60 s cached evaluation (probes were timing out → mld heal restart loop)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +4 −1

  2. Sun, Nov 1, 2026 15

    1. manifests: compact per-version deltas (full lists reconstructed from lake_files); restore drill verifies against the restored DB
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 3 files +42 −16
    2. memory: release Arrow buffers between symbols, smaller DuckDB limit, faster stale requeue
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 4 files +21 −7
    3. gc: remove stale .part downloads
      Simon-Pierre Boucher · 1 file +3
    4. macOS TCC workaround: run PM2 processes through sshd (Full Disk Access) for the external SSD; SIGHUP graceful stop; EIO-safe logging
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +1
    5. lake/client: EINTR-safe mkdir on the external volume; cap per-symbol anomaly rows in backfills
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 2 files +16 −4
    6. cli: worker --role, plan-all, backfill-all, options, fundamentals --all-active
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +51 −3
    7. full FirstRate catalog: cold lake root (external SSD), 57 new datasets, index/contracts/options parsers, options task, bulk workers, complete download plan (798 jobs), extended scheduler
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 11 files +315 −72
    8. restore drill: credentials via environment, redacted errors
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +15 −5
    9. compaction: single read of deltas, in-memory slicing; gc of orphan curated files
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 3 files +41 −26
    10. derive: base layout per symbol/year for backfill-driven 5m aggregation
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +33 −1
    11. api: listen on all interfaces behind a source-network guard (tunnel reachability); cli closes the pool
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +8
    12. sandbox: accept EXPLAIN; assistant test green (37 tests)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +2 −1
    13. AI assistant (config-gated, validated SQL only), EXPLAIN in sandbox
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +2 −1
    14. lint config, infra (mld manifest, alerts, bootstrap), docs (architecture, dictionary, operations, security)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 15 files +63 −42
    15. FinData OS: phase 0–4 foundation (discovery, lakehouse, ingest, API, SDK, tests)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 16 files +1,404
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere