spb/social-runtime-crawler

Fri, Sep 11, 2026 2

1. Research console (Next 16) + control-plane API + mld deployment for www.socialcrawl.co

   …

   - apps/dashboard: SocialCrawl console — observatory, sessions console (decisions, page state,
     entities, runtime APIs, media, world-model graph, live events), entity evidence pages,
     runtime API explorer, connectors, missions launcher; passcode gate + authenticated API proxy.
   - apps/api: node:http control plane (overview, sessions, entities search, schemas, platforms,
     SSE stream, job runner spawning worker processes), token-protected mutations.
   - worker: bounded action execution (60 s), PLAY_VIDEO no longer awaits play(); --job linking.
   - storage: jsonb sanitiser, linkJob; util: unicode thousands separators, stopwords.
   - deploy/social-runtime-crawler.mld.json: PM2 api 8350 + web 8351, MacLustr Tunnel route.
   
   Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

   Simon-Pierre Boucher committed 12 days ago · 1 file +5 −2
   d16c8f9
2. Social Runtime Crawler v0.1 — browser-native self-learning social crawler PoC

   …

   Monorepo (pnpm, strict TS via tsx, vitest): persistent Chromium sessions with human login,
   network observer + SchemaProfiler + generic JSON entity miner, DOM mutation observer + in-page
   semantic snapshot, page classifier, network/DOM surface merge with provenance, video detection
   and frame sampling, world model, information-gain heuristic planner (+ optional Anthropic/local
   LLM planner), loop detection, platform-model learning and connector compilation, PostgreSQL +
   JSONL replay storage, research dashboard, YouTube and Reddit adapters.
   
   Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

   Simon-Pierre Boucher committed 12 days ago · 1 file +29
   40fba6b