-
Sun, Sep 13, 2026 7
-
coverage: governments & public institutions of the world — 3 313 members in 18 institutional roles (head of government, portal, parliament, central bank, finance, foreign affairs, statistics, health, energy, transport, competition, financial regulator, cybersecurity agency, gazette, open data, election authority, supreme court, international bodies) across 195 jurisdictions, 2 395 Wikidata QIDs; core: bund.de suffix
…
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
core: registrableDomain treats generic second-level labels under ccTLDs (gov.py, go.cr, gouv.ht, ac.ir, co.zw…) as public suffixes — institutions there are distinct organizations
…
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
discovery: plain sitemaps enter shadow at tier C (news sitemaps stay B) — 168 shadow sitemaps at 15 min were the heaviest new load
…
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
factory: continuous worker pool (10 workers, one seed each) instead of lock-step batches; per-organization wall-clock deadline (120 s); test updated for canada.ca departments
…
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
coverage: science/healthcare final revisions (THE 2026 top 400, Wikidata QIDs, verified press/pipeline hints); core: academic multi-label suffixes (edu.mo, ac.ir, edu.qa…)
…
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
deploy: anchor the coverage exclusion to the repo root (it hid apps/web/src/app/coverage and config/coverage); core: more multi-label suffixes (canada.ca, admin.ch, *.co)
…
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
WebSensor 0.3 — Source Factory & Global Observation Coverage Score
…
- Deep discovery (robots → sitemaps → feeds incl. official sub-domains → navigation-classified pages → status providers → GitHub org → EDGAR → Hugging Face → OpenAPI → posture), every candidate validated, wildcard DNS / anti-bot detection, content de-duplication, expected change frequency (median gap) - Factory: coverage universes → factory_seeds → scored candidates (explainable) → promotion caps → SHADOW sensors (polled, never published) → evidence-based acceptance/rejection; separate websensor-factory process with Redis heartbeat; CLI expand/factory seed|run|evaluate|stats|export|requeue; YAML export for graduation - Migration 0007 (sources.origin/sector, factory_seeds, candidate lifecycle columns, factory_daily); pipeline keeps SHADOW status and skips events for shadow sensors - Coverage: sector universes schema + validator (--dns), registrableDomain(), GET /api/v1/coverage[/:sector], pages /coverage and /coverage/[sector], homepage rail, stats exclude shadow sensors - Admin API /api/v1/admin/factory* + /ops Source Factory panel (funnel, sectors, candidates, accept/reject) - Docs: COVERAGE.md, CLAUDE.md, deploy README, ARCHITECTURE, README; manifest gains websensor-factory + factory seed hook; version 0.3.0 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
-
Fri, Sep 11, 2026 1
-
WebSensor 0.2 — signal intelligence upgrade
…
Registry: 7 depth fragments (40–46: AI frontier, cloud infrastructure, cybersecurity, finance & markets, governments, science & health, transport/telecom/sports/news) → 3 167 organizations / 6 320 validated sensors (was 2 681 / 4 052); seeds carry country, language and first_party; sync infers country/language by TLD and a 0–3 scheduling priority. Core: semantic diff (noise classes vs pricing/policy/product/personnel, field-level before → after with % deltas), WebSensor Signal Score with explainable reasons, impact / velocity / anomaly scores, breaking state, entity rank, ~40 new event classes (zero_day, active_exploitation, supply_chain_attack, guidance, buyback, sanction, legislation, outbreak, grounding, service_shutdown…), search syntax (entity: type: after:7d silent: importance:>70 country:), shared registry schema. Engine: noise never becomes an event, routine firehose batches damped, fingerprint idempotency, ambiguous-alias rule for entity resolution (+ relink-entities CLI), silent-change bar (first-party · eligible type · importance ≥ 45), clusters with propagation timeline, first-party/external counts, velocity, lead time and breaking/developing/confirmed states, server-side alerts with HMAC-signed webhooks, raw-body retention with reference counting, priority-aware scheduler with per-domain circuit breaker and Redis heartbeat, UTF-16 BOM transcoding. API: new filters (signal_min, group, first_party, confirmed, country, language, change_class, q syntax, order=signal), /breaking /pulse /radar /clusters/:slug /entities/rank /countries /categories/:channel /sensors/:id/snapshots, owner routes (bookmarks, saved views, notifications, custom URL monitors with SSRF checks), admin routes behind WS_ADMIN_TOKEN (ops, sensor/source actions, connector test, YAML/JSON bulk import), TTL cache, WebSocket protocol 2 with stream ids and replay. Web: terminal-style design system (badges, density modes, skeletons, sparklines, heatmaps), live system strip, ⌘K command palette, intelligence drawer, live feed v2 (URL-synced filters, pause, "↑ N new events", replay), pages /live /pulse /radar /entity/[id] /cluster/[slug] /country /bookmarks /monitors /ops, redesigned breaking / silent / explore / event / source / sensor / search / alerts / watchlists / health / api docs, OG image per event, sitemap, loading skeletons. Migrations 0005 (additive schema) and 0006 (backfills). Docs updated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
-
Tue, Sep 8, 2026 4
-
registry: 22 site-class fragments (2 681 organizations, 4 052 validated sensors) + 10 new connector families
…
- config/sources.d/*.yaml merged on top of the founding file by apps/engine/src/seeds.ts (`extend: true` adds sensors to an existing source); classes: open source, central banks & finance, energy/climate/weather, telecom & internet infra, retail/consumer/travel, gaming & entertainment, universities & research, international orgs/NGOs/standards, sports, crypto, transport/aviation/space, consumer safety/food/agri, housing/labour/open data, enterprise SaaS, health systems & medtech, politics/elections/courts, SEC EDGAR (254 issuers), package registries, web posture, OpenAPI, status JSON, documents & data - connectors: package (10 registries), edgar, statusjson (Instatus/incident.io/Status.io), openapi, csv, pdf (unpdf), dns, tls, headers, rdap (IANA bootstrap); fetcher keepAllHeaders - heuristics: connector-class priors + event types api_change, dns_change, domain_registration_change, crawler_policy_change; 8-K items → earnings/leadership_change/breach/acquisition - registry validator without DB (apps/engine/src/validate.ts, --probe), generators gen-edgar.py, gen-web-posture.py, prune-fragment.py; docs/registry/AUTHORING.md + docs/connectors/*.md (16) - SSRF: NAT64 addresses judged by the embedded IPv4 (IPv6-only hotspots) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
registry: +26 governments/public institutions, +45 news media (validated live); News feed channel; per-source llm switch (news = heuristics only)
…
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
discovery: page candidates must be real, distinct, non-thin documents (catch-all 200 / CAPTCHA pages rejected); SEDAR+ sensor removed
…
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-
WebSensor v0.1.0 — real-time web intelligence platform
…
Core (taxonomy, SSRF policy, canonical extraction, diff engines, heuristics, scoring, adaptive schedule), db (SQL migrations + Drizzle), content-addressed zstd blob store, connectors (http, rss/atom, sitemap, statuspage, github, jsonlist, discovery, scrapfly fallback), engine (scheduler, pipeline, Claude interpretation, entities, novelty/clustering, silent changes, metrics), Fastify gateway (REST + WebSocket + proxy), Next.js 16 frontend, 200-source registry, tests, docs, mld manifest. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
-