SPB Git forge
spboucher.ai ↗

spb/websensor

Public
33commits 1branches 0releases
3.4 MBsize
maindefault branch
10 days agolast push
TypeScript 55.4% Python 43.2% SQL 1.2%

History of deploy/websensor.mld.json.example · clear filter

  1. Sun, Sep 13, 2026 4

    1. deploy: engine fetch concurrency 48, factory 10 workers in the example manifest
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 11 days ago · 1 file +1 −1
    2. factory: continuous worker pool (10 workers, one seed each) instead of lock-step batches; per-organization wall-clock deadline (120 s); test updated for canada.ca departments
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 11 days ago · 1 file +3 −2
    3. deploy: anchor the coverage exclusion to the repo root (it hid apps/web/src/app/coverage and config/coverage); core: more multi-label suffixes (canada.ca, admin.ch, *.co)
      …
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 11 days ago · 1 file +4 −2
    4. WebSensor 0.3 — Source Factory & Global Observation Coverage Score
      …
      - Deep discovery (robots → sitemaps → feeds incl. official sub-domains → navigation-classified pages → status providers → GitHub org → EDGAR → Hugging Face → OpenAPI → posture), every candidate validated, wildcard DNS / anti-bot detection, content de-duplication, expected change frequency (median gap)
- Factory: coverage universes → factory_seeds → scored candidates (explainable) → promotion caps → SHADOW sensors (polled, never published) → evidence-based acceptance/rejection; separate websensor-factory process with Redis heartbeat; CLI expand/factory seed|run|evaluate|stats|export|requeue; YAML export for graduation
- Migration 0007 (sources.origin/sector, factory_seeds, candidate lifecycle columns, factory_daily); pipeline keeps SHADOW status and skips events for shadow sensors
- Coverage: sector universes schema + validator (--dns), registrableDomain(), GET /api/v1/coverage[/:sector], pages /coverage and /coverage/[sector], homepage rail, stats exclude shadow sensors
- Admin API /api/v1/admin/factory* + /ops Source Factory panel (funnel, sectors, candidates, accept/reject)
- Docs: COVERAGE.md, CLAUDE.md, deploy README, ARCHITECTURE, README; manifest gains websensor-factory + factory seed hook; version 0.3.0

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 11 days ago · 1 file +194 −123

  2. Tue, Sep 8, 2026 1

    1. WebSensor v0.1.0 — real-time web intelligence platform
      …
      Core (taxonomy, SSRF policy, canonical extraction, diff engines, heuristics, scoring, adaptive schedule),
db (SQL migrations + Drizzle), content-addressed zstd blob store, connectors (http, rss/atom, sitemap,
statuspage, github, jsonlist, discovery, scrapfly fallback), engine (scheduler, pipeline, Claude
interpretation, entities, novelty/clustering, silent changes, metrics), Fastify gateway (REST + WebSocket +
proxy), Next.js 16 frontend, 200-source registry, tests, docs, mld manifest.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 16 days ago · 1 file +128