SPB Git
spboucher.ai

spb/findata

Public
History of services/ingest Clear filter
page 1

  1. Mon, Nov 2, 2026 11

    1. raise open-file limit for wide scans; date-only SQL scopes use the catalog too
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +10 −3
    2. SQL fast path: symbol-scoped statements resolve their file list from the catalog
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +66 −3
    3. SQL views: base (hive) UNION ALL BY NAME delta (derived symbol/year); options hive
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +19 −8
    4. planner: symbol filter in SQL; views skip emptied layouts
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 2 files +10 −5
    5. SQL views: hive-partitioned directory globs, lazy + cached; coverage limited to primary datasets; 4 bulk workers
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 2 files +40 −11
    6. scheduler: daily dedupe keys fire once (quality/gc were re-enqueued every tick); backup scope documented
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 2 files +26 −23
    7. publish: chained O(delta) snapshot id; backfill batches 20 symbols per version; heartbeat-based stale requeue; symbol index
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 4 files +58 −18
    8. options parser: ragged lines tolerated, malformed files skipped
      Simon-Pierre Boucher · 1 file +1 −1
    9. parser: tolerate ragged vendor lines; skip unparseable files instead of failing the archive
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +5 −1
    10. fundamentals: remember 404 listings in the checkpoint so resumes do not retry them
      Simon-Pierre Boucher · 1 file +3 −1
    11. health: catalog-based lake usage and 60 s cached evaluation (probes were timing out → mld heal restart loop)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +4 −1

  2. Sun, Nov 1, 2026 15

    1. manifests: compact per-version deltas (full lists reconstructed from lake_files); restore drill verifies against the restored DB
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 3 files +42 −16
    2. memory: release Arrow buffers between symbols, smaller DuckDB limit, faster stale requeue
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 4 files +21 −7
    3. gc: remove stale .part downloads
      Simon-Pierre Boucher · 1 file +3
    4. macOS TCC workaround: run PM2 processes through sshd (Full Disk Access) for the external SSD; SIGHUP graceful stop; EIO-safe logging
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +1
    5. lake/client: EINTR-safe mkdir on the external volume; cap per-symbol anomaly rows in backfills
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 2 files +16 −4
    6. cli: worker --role, plan-all, backfill-all, options, fundamentals --all-active
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +51 −3
    7. full FirstRate catalog: cold lake root (external SSD), 57 new datasets, index/contracts/options parsers, options task, bulk workers, complete download plan (798 jobs), extended scheduler
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 11 files +315 −72
    8. restore drill: credentials via environment, redacted errors
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +15 −5
    9. compaction: single read of deltas, in-memory slicing; gc of orphan curated files
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 3 files +41 −26
    10. derive: base layout per symbol/year for backfill-driven 5m aggregation
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +33 −1
    11. api: listen on all interfaces behind a source-network guard (tunnel reachability); cli closes the pool
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +8
    12. sandbox: accept EXPLAIN; assistant test green (37 tests)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +2 −1
    13. AI assistant (config-gated, validated SQL only), EXPLAIN in sandbox
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 1 file +2 −1
    14. lint config, infra (mld manifest, alerts, bootstrap), docs (architecture, dictionary, operations, security)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 15 files +63 −42
    15. FinData OS: phase 0–4 foundation (discovery, lakehouse, ingest, API, SDK, tests)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher · 17 files +1,423
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere