spb/social-runtime-crawler

Fri, Sep 11, 2026 3

1. Bounded network collection (streaming/long-poll bodies), observe timeout, recently_failed penalty, .mldignore

   …

   - NetworkObserver: 8 s body timeout, 6 s step collection cap, skip streaming/presence URLs and event-streams, fewer WS frames
   - engine: observation bounded to 45 s; failed actions penalised for 8 steps (no blind retries)
   - .mldignore keeps data/ (browser profiles) out of mld stage; README notes
   
   Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

   Simon-Pierre Boucher committed 12 days ago · 1 file +4 −1
   d3b24f2
2. README: console, control plane, MacLustr deployment; QA scripts

   …

   Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

   Simon-Pierre Boucher committed 12 days ago · 1 file +13 −3
   bacb91d
3. Social Runtime Crawler v0.1 — browser-native self-learning social crawler PoC

   …

   Monorepo (pnpm, strict TS via tsx, vitest): persistent Chromium sessions with human login,
   network observer + SchemaProfiler + generic JSON entity miner, DOM mutation observer + in-page
   semantic snapshot, page classifier, network/DOM surface merge with provenance, video detection
   and frame sampling, world model, information-gain heuristic planner (+ optional Anthropic/local
   LLM planner), loop detection, platform-model learning and connector compilation, PostgreSQL +
   JSONL replay storage, research dashboard, YouTube and Reddit adapters.
   
   Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

   Simon-Pierre Boucher committed 12 days ago · 1 file +69
   40fba6b