| 1 |
|
−# WebSensor |
| 2 |
|
− |
| 3 |
|
−**Detect What Changed. Know Why It Matters.** — https://www.websensor.io |
| 4 |
|
− |
| 5 |
|
−WebSensor is a global sensor network for the changing Web. It continuously monitors official public sources |
| 6 |
|
−(newsrooms, status pages, changelogs, regulators, registries, APIs), preserves every observation, detects raw |
| 7 |
|
−changes, filters noise, interprets meaningful changes into events, links them to entities, scores their |
| 8 |
|
−importance and confidence, clusters related events and publishes them live over WebSocket. |
| 9 |
|
− |
| 10 |
|
−``` |
| 11 |
|
−Sources → Sensors/Connectors → Fetch (conditional) → Snapshot (immutable) → Diff → Heuristics |
| 12 |
|
− → Interpretation (Claude, budgeted) → Entities → Novelty/Clustering → Importance → Event |
| 13 |
|
− → Redis stream → WebSocket gateway → www.websensor.io |
| 14 |
|
−``` |
| 15 |
|
− |
| 16 |
|
−- **2 681 organizations, 4 052 curated sensors** (2026-09-08), every one fetched and parsed by the registry validator |
| 17 |
|
− before being committed: the founding 271 (`config/sources.yaml`) plus 22 site-class fragments in |
| 18 |
|
− `config/sources.d/` — open source, central banks & finance, energy/climate/weather, telecom & internet |
| 19 |
|
− infrastructure, retail/consumer/travel, gaming & entertainment, universities & research, international |
| 20 |
|
− organizations/NGOs/standards, sports, crypto, transport/aviation/space, consumer safety/food/agriculture, |
| 21 |
|
− housing/labour/open data, enterprise SaaS, health systems & medtech, politics/elections/courts, SEC EDGAR filings |
| 22 |
|
− (254 issuers), package registries, web posture (robots.txt, TLS, DNS, headers, RDAP), OpenAPI contracts, status |
| 23 |
|
− JSON pages, documents & data files — plus validated discovery. |
| 24 |
|
−- **16 connector families**: HTTP (HTML/JSON/HEAD), RSS/Atom/JSON Feed, sitemaps, Statuspage, Instatus/incident.io/ |
| 25 |
|
− Status.io (`statusjson`), GitHub, JSON APIs (`jsonlist`), package registries (npm, PyPI, crates.io, RubyGems, NuGet, |
| 26 |
|
− Packagist, Hex, Go proxy, Homebrew, Docker Hub), SEC EDGAR submissions (8-K items decoded), OpenAPI/Swagger |
| 27 |
|
− contract fingerprints, CSV/TSV time series (FRED, ECB, NOAA…), PDF text (pdf.js), DNS records, TLS certificates, |
| 28 |
|
− HTTP security headers, RDAP registration — and a budgeted Scrapfly fallback. |
| 29 |
|
−- Authoring guide and connector cheat-sheet: `docs/registry/AUTHORING.md`; validator: |
| 30 |
|
− `node node_modules/tsx/dist/cli.mjs apps/engine/src/validate.ts <fragment.yaml>` (no database needed). |
| 31 |
|
−- Every event is auditable: event → change → snapshot A/B → fetch run → sensor → source, with raw evidence. |
| 32 |
|
−- Observed vs inferred is always labelled; silent (unannounced) changes are flagged. |
| 33 |
|
− |
| 34 |
|
−## Quick start |
| 35 |
|
−``` |
| 36 |
|
−createdb websensor && cp .env.example .env && pnpm install |
| 37 |
|
−npx tsx apps/engine/src/cli.ts sync # load the registry |
| 38 |
|
−pnpm dev:api & pnpm dev:web & pnpm dev:engine |
| 39 |
|
−open http://localhost:8260 |
| 40 |
|
−``` |
| 41 |
|
− |
| 42 |
|
−Docs: `CLAUDE.md` (repo guide), `docs/ARCHITECTURE.md`, `docs/connectors/*.md`, `deploy/README.md`, |
| 43 |
|
−API docs at `/api` on the site. Public API: `GET /api/v1/events`, `/api/v1/entities/{id}`, |
| 44 |
|
−`/api/v1/sources/{id}`, `/api/v1/domains/{domain}/timeline`, `/api/v1/changes/{id}`, `wss://…/api/v1/live`, |
| 45 |
|
−`/api/v1/feed.rss`. |
|
1 |
+<p align="center"> |
|
2 |
+ <img src="docs/brand/websensor-wordmark.svg" alt="WebSensor" width="420"> |
|
3 |
+</p> |
|
4 |
+ |
|
5 |
+<h1 align="center">WebSensor</h1> |
|
6 |
+ |
|
7 |
+<p align="center"><strong>The Web is changing. We are watching.</strong><br> |
|
8 |
+Real-time Internet change intelligence — <a href="https://www.websensor.io">www.websensor.io</a></p> |
|
9 |
+ |
|
10 |
+<p align="center"> |
|
11 |
+ <img src="docs/brand/websensor-mark.svg" alt="" width="28" valign="middle"> |
|
12 |
+ 3 167 organizations · 7 066 active sensors · 16 connector families · 94 event classes · immutable evidence · live WebSocket feed |
|
13 |
+</p> |
|
14 |
+ |
|
15 |
+--- |
|
16 |
+ |
|
17 |
+WebSensor continuously watches the public web for meaningful changes, preserves the evidence, connects related |
|
18 |
+signals into real-world events and surfaces important developments before they become obvious. It is not a |
|
19 |
+website monitor: it answers **what changed, where, when, exactly what, whether it was public or silent, how |
|
20 |
+important and unusual it is, who is responsible, which other sources confirm it, whether it is part of a larger |
|
21 |
+event, how fast the story is spreading and whether WebSensor was early**. |
|
22 |
+ |
|
23 |
+Built by **Simon-Pierre Boucher** · [contact@spboucher.ai](mailto:contact@spboucher.ai) · hosted on |
|
24 |
+[MacLustr](https://www.maclustr.io), an Apple-silicon compute cluster in Québec. |
|
25 |
+ |
|
26 |
+## Table of contents |
|
27 |
+ |
|
28 |
+1. [Brand](#brand) |
|
29 |
+2. [What WebSensor does](#what-websensor-does) |
|
30 |
+3. [Architecture](#architecture) |
|
31 |
+4. [The pipeline, step by step](#the-pipeline-step-by-step) |
|
32 |
+5. [Scoring model](#scoring-model) |
|
33 |
+6. [Event taxonomy](#event-taxonomy) |
|
34 |
+7. [Source registry](#source-registry) |
|
35 |
+8. [Connectors](#connectors) |
|
36 |
+9. [Data model](#data-model) |
|
37 |
+10. [Real-time protocol](#real-time-protocol) |
|
38 |
+11. [REST API](#rest-api) |
|
39 |
+12. [Frontend](#frontend) |
|
40 |
+13. [Repository layout](#repository-layout) |
|
41 |
+14. [Development](#development) |
|
42 |
+15. [Testing](#testing) |
|
43 |
+16. [Deployment and operations](#deployment-and-operations) |
|
44 |
+17. [Security and crawling etiquette](#security-and-crawling-etiquette) |
|
45 |
+18. [Roadmap](#roadmap) |
|
46 |
+19. [Credits](#credits) |
|
47 |
+ |
|
48 |
+## Brand |
|
49 |
+ |
|
50 |
+<p> |
|
51 |
+ <img src="docs/brand/websensor-mark.svg" alt="WebSensor mark" width="64"> |
|
52 |
+ |
|
53 |
+ <img src="apps/web/public/icons/icon-192.png" alt="PWA icon" width="64"> |
|
54 |
+</p> |
|
55 |
+ |
|
56 |
+The mark is a **pulse in the shape of a W**: the waveform is grey until the moment a change is detected, then |
|
57 |
+turns signal green and ends in the sensor reading. It is drawn once as SVG and reused everywhere: |
|
58 |
+ |
|
59 |
+| Asset | Path | Use | |
|
60 |
+|---|---|---| |
|
61 |
+| Mark (SVG) | `apps/web/src/app/icon.svg`, `apps/web/public/logo.svg`, `docs/brand/websensor-mark.svg` | favicon (SVG-capable browsers), nav, docs | |
|
62 |
+| Wordmark (SVG) | `docs/brand/websensor-wordmark.svg` | README, documents | |
|
63 |
+| `favicon.ico` | `apps/web/src/app/favicon.ico` | 16 / 32 / 48 px, legacy tabs | |
|
64 |
+| Apple touch icon | `apps/web/src/app/apple-icon.png` | 180 px | |
|
65 |
+| PWA icons | `apps/web/public/icons/icon-192.png`, `icon-512.png`, `icon-512-maskable.png` | manifest | |
|
66 |
+| Share image | `apps/web/src/app/opengraph-image.tsx` (+ `twitter-image.tsx`) | 1200 × 630, live counters from the API | |
|
67 |
+| Event share image | `apps/web/src/app/event/[slug]/opengraph-image.tsx` | per-event card with signal score and badges | |
|
68 |
+| React component | `apps/web/src/components/brand.tsx` (`Mark`, `PulseBackdrop`, `BRAND` colours) | app and OG images | |
|
69 |
+ |
|
70 |
+Colours: background `#0a0e15`, ink `#c9d1e0`, signal `#22d3a5`, line `#2d394e`. Type: Inter (UI), JetBrains Mono |
|
71 |
+(numbers, identifiers, timestamps). The product aesthetic is *Bloomberg Terminal × Linear × Vercel × Datadog × |
|
72 |
+threat-intelligence desk*: dense information, one-pixel borders, small badges, precise spacing, dark first. |
|
73 |
+ |
|
74 |
+## What WebSensor does |
|
75 |
+ |
|
76 |
+| Question | Where the answer comes from | |
|
77 |
+|---|---| |
|
78 |
+| What changed, exactly? | Semantic diff: field-level *before → after* pairs (prices, limits, versions, dates, status words, names) with % deltas, plus unified / side-by-side / semantic diff views | |
|
79 |
+| Where and when? | Sensor URL, canonical URL, `published_at` (claimed by the source), `observed_from` (previous snapshot), `detected_at`, `processed_at`, all UTC | |
|
80 |
+| Was it public or silent? | **Silent change** = first-party source · silent-eligible type (pricing, terms, policy, API, availability, docs, shutdown, feature removed, leadership, page removed…) · no matching announcement within 12 h · importance ≥ 45 | |
|
81 |
+| How important? | Importance (8 stored components) → **WebSensor Signal Score** 0–100 with an explainable reason list | |
|
82 |
+| Is it unusual? | Source activity anomaly vs a 14-day baseline; entity anomaly vs a 30-day baseline (`+1 800 % vs baseline`) | |
|
83 |
+| Who is responsible? | Entity resolution (organizations, products, models, APIs, regulators) with parent/child relations | |
|
84 |
+| Which sources confirm it? | Novelty vs the 72 h window, cross-source confirmations, `OBSERVED / INFERRED / CONFIRMED / UNCONFIRMED` | |
|
85 |
+| Part of a larger event? | Event clusters with propagation timeline, first-party vs external signals, velocity, `breaking / developing / confirmed / watching` state | |
|
86 |
+| Was WebSensor early? | **Lead time**: first-party detection → first external report | |
|
87 |
+| What should I pay attention to? | `/breaking`, `/pulse`, `/radar` (weak signals), watchlists, alert rules (web + signed webhooks) | |
|
88 |
+ |
|
89 |
+## Architecture |
|
90 |
+ |
|
91 |
+``` |
|
92 |
+ ┌──────────────────────── MacLustr node M4M64b ────────────────────────┐ |
|
93 |
+ Internet ──TLS──▶ BHS64 │ websensor-api :8260 Fastify gateway │ |
|
94 |
+ (OVH, Caddy, (wg1) │ /api/v1/* REST + WebSocket /api/v1/live (Redis pub/sub fan-out) │ |
|
95 |
+ MacLustr Tunnel) │ /* reverse proxy → websensor-web :8261 (Next.js 16) │ |
|
96 |
+ │ websensor-engine scheduler + pipeline, Prometheus :8262 │ |
|
97 |
+ │ PostgreSQL 17 `websensor` · Redis · blob store ~/websensor-data/blobs │ |
|
98 |
+ └───────────────────────────────────────────────────────────────────────┘ |
|
99 |
+``` |
|
100 |
+ |
|
101 |
+| Role | Package | Notes | |
|
102 |
+|---|---|---| |
|
103 |
+| Shared intelligence | `packages/core` | taxonomy, SSRF policy, hashing, canonical extraction, diff engines, heuristics, semantic diff, scoring, schedule, search syntax, registry schema. `@websensor/core/client` is the browser-safe subset | |
|
104 |
+| Persistence | `packages/db` | plain-SQL forward migrations (`migrations/*.sql`, advisory-locked) + Drizzle schema | |
|
105 |
+| Evidence | `packages/store` | content-addressed zstd blob store (`sha256/ab/cd/<hash>.zst`); raw bodies, canonical forms and full diffs never live in Postgres | |
|
106 |
+| Acquisition | `packages/connectors` | connector SDK, safe fetcher, 16 connector families, discovery, budgeted Scrapfly fallback | |
|
107 |
+| Ingestion | `apps/engine` | scheduler, pipeline, registry sync, discovery, health rollups, retention, alerts, CLI, validator | |
|
108 |
+| Gateway | `apps/api` | REST, WebSocket, intelligence read-models, TTL cache, owner-scoped tools, admin API | |
|
109 |
+| Product | `apps/web` | Next.js 16 App Router, design system, live feed, drawer, palette, desks | |
|
110 |
+ |
|
111 |
+Everything is TypeScript ESM in a pnpm workspace (Node ≥ 22.15). One node runs everything today; every role is |
|
112 |
+separable (several engine processes can run concurrently thanks to `FOR UPDATE SKIP LOCKED` claims). |
|
113 |
+ |
|
114 |
+## The pipeline, step by step |
|
115 |
+ |
|
116 |
+Every sensor run goes through `apps/engine/src/pipeline.ts`: |
|
117 |
+ |
|
118 |
+1. **Schedule** — the scheduler claims due sensors by priority (0 critical … 3 low) with `FOR UPDATE SKIP LOCKED`, |
|
119 |
+ enforces global concurrency and per-host limits (higher for large public APIs such as `github.com`, |
|
120 |
+ `data.sec.gov`), applies a per-domain circuit breaker (5 consecutive failures → 10 min pause) and publishes a |
|
121 |
+ heartbeat to Redis (`ws:engine:status`). |
|
122 |
+2. **Fetch** — conditional GET (`ETag` / `If-None-Match`, `Last-Modified` / `If-Modified-Since`), manual redirects |
|
123 |
+ validated per hop by the SSRF policy, HTTP/2 → HTTP/1.1 pinning on NGHTTP2 errors, a second attempt with a |
|
124 |
+ browser identity on 403/resets, size and time limits, gzip and UTF-16 BOM handling. `304` = cheap check. |
|
125 |
+3. **Normalize** — the connector turns the observation into a comparable representation: `text` (canonical |
|
126 |
+ HTML/plain text with navigation, footers, cookie banners, timestamps, tokens and counters stripped), `json` |
|
127 |
+ (sorted keys, volatile paths ignored) or `list` (keyed items: feed entries, sitemap URLs, incidents, releases, |
|
128 |
+ filings, records). |
|
129 |
+4. **Snapshot** — if the canonical hash changed, raw body and canonical form are stored immutably and a |
|
130 |
+ `snapshots` row records hashes, HTTP metadata, extraction confidence and the source's own timestamp. |
|
131 |
+5. **Diff** — text (paired modifications), json (path operations) or list (added / removed / modified) with |
|
132 |
+ connector guards: feed items are "new" only if never seen and < 14 days old, status pages losing an item is a |
|
133 |
+ resolution, a > 50 % sitemap shrink is a partial response, a 404 becomes `page_removed` only after separated |
|
134 |
+ confirmations. |
|
135 |
+6. **Heuristics + semantic class** — `evaluateChange` proposes an event type, magnitude, facts (prices, percents, |
|
136 |
+ versions, dates) and a signal 0–1 from ~90 typed rules and record-shape priors (KEV records, CVE ids, 8-K |
|
137 |
+ items, ClinicalTrials statuses…). `classifyChange` labels the change `cosmetic / navigation / timestamp / |
|
138 |
+ advertisement / boilerplate` (noise) or `meaningful / pricing / policy / product / personnel` (signal) and |
|
139 |
+ extracts field-level changes. A `changes` row is **always** written; noise classes never become events. |
|
140 |
+7. **Event candidate** — fingerprint idempotency (sensor + before/after canonical hashes), entity resolution |
|
141 |
+ (aliases, longest first; ordinary-word aliases such as `first` or `has` only match as exact upper-case |
|
142 |
+ acronyms), novelty vs the 72 h window (near-duplicates suppressed, redundant third-party reports suppressed), |
|
143 |
+ source anomaly, preliminary importance (routine firehose batches damped), optional **Claude interpretation** |
|
144 |
+ above `WS_LLM_MIN_IMPORTANCE` (strict JSON: type, title, summary, why it matters, observed vs inferred, |
|
145 |
+ severity, confidence, announced?), final importance + confidence + impact, silent-change bar. |
|
146 |
+8. **Cluster** — attach to an open cluster (shared entity or source, textual similarity within 6 h; same-entity |
|
147 |
+ cross-source stories within 2 h) or open one; update propagation timeline, first-party/external counts, |
|
148 |
+ velocity, lead time and breaking state. |
|
149 |
+9. **Signal score & publish** — `computeSignalScore` with stored reasons; `events` row; entity and source daily |
|
150 |
+ counters; alert evaluation (web notifications, HMAC-signed webhooks); Redis stream `ws:events` (durable, with |
|
151 |
+ stream ids) + pub/sub `ws:live`. |
|
152 |
+10. **Re-schedule** — adaptive interval from tier bounds, recency of change, change frequency, 304 ratio and errors |
|
153 |
+ (tier S 15 s–5 min · A 1–30 min · B 5 min–2 h · C 30 min–6 h · D 6–24 h). |
|
154 |
+ |
|
155 |
+Background jobs: connector health rollups (1 min), cluster state ageing (5 min), retention (30 min: raw bodies |
|
156 |
+of snapshots without an event are dropped after 21 days, canonical forms and hashes kept, event snapshots kept |
|
157 |
+forever, blobs reference-counted), run log pruning, weekly discovery. |
|
158 |
+ |
|
159 |
+## Scoring model |
|
160 |
+ |
|
161 |
+**Importance** (0–100) = 25 % intrinsic severity of the event type · 20 % source authority (tier × weight) · |
|
162 |
+15 % entity importance · 15 % novelty · 10 % magnitude · 5 % cross-source confirmation · 5 % user impact · |
|
163 |
+5 % unusualness. All eight components are stored on the event. |
|
164 |
+ |
|
165 |
+**Confidence** (0–100) blends source authenticity, extraction confidence, diff clarity, structured vs free text, |
|
166 |
+confirmations and LLM/heuristic agreement. |
|
167 |
+ |
|
168 |
+**Impact** (0–100) weights the event type's user impact, entity importance, magnitude and the largest numeric |
|
169 |
+delta among extracted fields (`$100 → $500` scores higher than `$100 → $105`). |
|
170 |
+ |
|
171 |
+**Velocity** (0–100) grows with signals per hour, unique sources and first-party signals inside a cluster. |
|
172 |
+ |
|
173 |
+**WebSensor Signal Score** (0–100) = 42 % importance · 14 % confidence · 14 % novelty · 12 % velocity · |
|
174 |
+12 % impact · 6 % anomaly, then adjusted with explainable reasons: `+ first-party evidence`, `+ N independent |
|
175 |
+confirmations`, `+ silent change`, `+ tier S source`, `- third-party report`, `- low confidence`, |
|
176 |
+`- unconfirmed`, `- routine batch from a high-volume feed`, and a hard cap of 20 for noise classes. The reason |
|
177 |
+list is shown on every event ("Why this score"). |
|
178 |
+ |
|
179 |
+**Breaking state** is not recency: `breaking` needs a strong signal, freshness (< 6 h) and either an |
|
180 |
+independent confirmation, a tier-S first-party source or signal ≥ 90; `developing` = signals accumulating; |
|
181 |
+`confirmed` = ≥ 3 sources within 24 h; `watching` otherwise; `closed` after 72 h. |
|
182 |
+ |
|
183 |
+**Entity rank** = importance × activity (24 h, 7 d) × average signal × confirmation ratio × unique sources × |
|
184 |
+acceleration vs the 30-day baseline — never raw event counts. |
|
185 |
+ |
|
186 |
+## Event taxonomy |
|
187 |
+ |
|
188 |
+94 event classes grouped for filters and desks: |
|
189 |
+ |
|
190 |
+| Group | Examples | |
|
191 |
+|---|---| |
|
192 |
+| Security | `zero_day`, `active_exploitation`, `vulnerability`, `security_advisory`, `breach`, `supply_chain_attack`, `credential_leak`, `malware_campaign`, `ransomware`, `patch_release` | |
|
193 |
+| Reliability | `outage`, `incident`, `maintenance`, `infrastructure_change`, `dns_change`, `certificate_change` | |
|
194 |
+| Product & API | `product_launch`, `model_release`, `software_release`, `service_launch`, `service_shutdown`, `feature_removed`, `API_change`, `availability_change`, `new_region`, `documentation_change` | |
|
195 |
+| Pricing & terms | `pricing_change`, `terms_change`, `policy_change`, `crawler_policy_change` | |
|
196 |
+| Corporate | `earnings`, `guidance`, `financial_filing`, `acquisition`, `merger`, `ipo`, `funding`, `capital_raise`, `dividend`, `buyback`, `bankruptcy`, `rating_change`, `insider_transaction`, `leadership_change`, `layoffs` | |
|
197 |
+| Government & legal | `government_announcement`, `regulatory_filing`, `regulatory_action`, `legislation`, `budget`, `procurement`, `sanction`, `lawsuit`, `court_decision`, `emergency_notice`, `monetary_policy`, `economic_release` | |
|
198 |
+| Science & health | `scientific_publication`, `retraction`, `dataset_release`, `clinical_trial`, `drug_approval`, `drug_warning`, `recall`, `device_recall`, `outbreak`, `space_mission`, `standard_update` | |
|
199 |
+| Transport | `accident`, `grounding`, `safety_bulletin`, `route_change`, `production_delay` | |
|
200 |
+| Sports | `sports_transaction`, `sports_result`, `suspension`, `schedule_change` | |
|
201 |
+| Web | `announcement`, `page_created`, `page_removed`, `content_change`, `domain_registration_change` | |
|
202 |
+ |
|
203 |
+Each class carries an intrinsic severity and a `usuallyAnnounced` flag used by silent-change detection. The |
|
204 |
+full list lives in `packages/core/src/taxonomy.ts`. |
|
205 |
+ |
|
206 |
+## Source registry |
|
207 |
+ |
|
208 |
+The registry is declarative YAML, merged in file-name order by `apps/engine/src/seeds.ts`: |
|
209 |
+ |
|
210 |
+- `config/sources.yaml` — the founding 271 organizations; |
|
211 |
+- `config/sources.d/10–35` — one fragment per **site class** (open source, central banks & finance, energy / |
|
212 |
+ climate / weather, telecom & internet infrastructure, retail / consumer / travel, gaming & entertainment, |
|
213 |
+ universities & research, international organizations / NGOs / standards, sports, crypto, transport / aviation / |
|
214 |
+ space, consumer safety / food / agriculture, housing / labour / open data, enterprise SaaS, health systems & |
|
215 |
+ medtech, politics / elections / courts, EDGAR filings, package registries, web posture, OpenAPI, status JSON, |
|
216 |
+ documents & data); |
|
217 |
+- `config/sources.d/40–46` — **depth** fragments (AI frontier, cloud infrastructure, cybersecurity, finance & |
|
218 |
+ markets, governments incl. Canadian provinces, science & health, transport / telecom / sports / world news); |
|
219 |
+- `config/sources.d/47+` — the long tail (open-source projects, SaaS status pages and changelogs, S&P 500 / |
|
220 |
+ Nasdaq-100 / TSX 60 EDGAR issuers and 160 US federal agencies via the Federal Register API, cities and regions, |
|
221 |
+ world governments and regulators, corporate pricing / legal / leadership pages, sports teams, entertainment, |
|
222 |
+ education, regional media, trade press, think tanks). |
|
223 |
+ |
|
224 |
+```yaml |
|
225 |
+sources: |
|
226 |
+ - id: bank-of-canada # kebab-case, unique across all files |
|
227 |
+ name: Bank of Canada |
|
228 |
+ domain: bankofcanada.ca |
|
229 |
+ categories: [finance, central-bank] |
|
230 |
+ tier: S # S 15–60 s · A 1–5 min · B 5–30 min · C 30 min–6 h · D 6–24 h |
|
231 |
+ weight: 1.4 |
|
232 |
+ country: CA # ISO 3166-1 alpha-2, EU, INT |
|
233 |
+ language: en |
|
234 |
+ products: |
|
235 |
+ - { name: Valet API, type: API } |
|
236 |
+ sensors: |
|
237 |
+ - { name: press releases, url: "https://www.bankofcanada.ca/…/feed/", type: RSS, connector: rss, tier: A } |
|
238 |
+ - { name: policy rate, url: "https://www.bankofcanada.ca/valet/observations/V39079/json?recent=3", type: REST_API, connector: jsonlist, tier: A, config: { … } } |
|
239 |
+ - id: cbc # media report ABOUT others → third-party evidence |
|
240 |
+ extend: true |
|
241 |
+ first_party: false |
|
242 |
+``` |
|
243 |
+ |
|
244 |
+Rules that never bend: |
|
245 |
+ |
|
246 |
+- **Nothing enters the registry without the validator's OK.** `apps/engine/src/validate.ts` runs every curated |
|
247 |
+ sensor through its connector (fetch + normalize) and prints `OK / WARN / FAIL`; `WARN` (empty list, thin |
|
248 |
+ client-rendered page) and `FAIL` (403, 404, parse error) sensors are pruned. Sites blocked by bot management |
|
249 |
+ are documented in `notes:`, never faked. |
|
250 |
+- One source = one organization (the entity people search for). `extend: true` adds sensors, products, aliases, |
|
251 |
+ categories, country, language or first-party flag to a source declared in an earlier file. |
|
252 |
+- Media and aggregators carry `first_party: false`; the organization's own channels are first-party and weigh |
|
253 |
+ more. High-volume feeds (news wires, arXiv, NVD, package streams, sports) run with `llm: false`. |
|
254 |
+- Country and language are inferred from unambiguous TLDs when not declared; scheduling priority is derived from |
|
255 |
+ tier and category. |
|
256 |
+ |
|
257 |
+Registry sync (`cli.ts sync`) upserts sources, sensors, organization and product entities, aliases and |
|
258 |
+`owns` relations; seed sensors removed from YAML are disabled, discovery-created ones are kept. The discovery |
|
259 |
+engine probes `robots.txt` sitemaps, `<link rel="alternate">` feeds, well-known feed paths and linked status |
|
260 |
+pages, and promotes only candidates that fetch and parse. Sensors carry a lifecycle status |
|
261 |
+(`PENDING → VALIDATED → ACTIVE / DEGRADED / DISABLED`). Bulk import (JSON or YAML) is available through the |
|
262 |
+admin API with a dry-run mode. Authoring guide: `docs/registry/AUTHORING.md`. |
|
263 |
+ |
|
264 |
+## Connectors |
|
265 |
+ |
|
266 |
+All connectors implement `fetch(endpoint)` and `normalize(endpoint, observation)`; the shared pipeline does |
|
267 |
+compare, classify, score and persist (`packages/connectors/src/types.ts`). |
|
268 |
+ |
|
269 |
+| Connector | Sensor types | Handles | |
|
270 |
+|---|---|---| |
|
271 |
+| `http` | HTML, JSON, XML, FILE, HTTP_HEADERS | canonical text extraction (`selector`, `keepChrome`), JSON with `jsonPath` / `ignoreKeys`, HEAD | |
|
272 |
+| `rss` | RSS, ATOM | RSS 2.0, Atom, RDF, JSON Feed; GitHub / GitLab / YouTube / Blogger / arXiv API feeds | |
|
273 |
+| `sitemap` | SITEMAP | sitemap index, urlset, news sitemaps (`maxUrls`, `maxChildren`) | |
|
274 |
+| `statuspage` | STATUSPAGE | Atlassian Statuspage `api/v2/summary.json` | |
|
275 |
+| `statusjson` | STATUSPAGE | Instatus, incident.io (multi-region subpages), Status.io | |
|
276 |
+| `github` | GITHUB_RELEASE, GITHUB_REPO | releases / tags / commits Atom, advisories REST | |
|
277 |
+| `jsonlist` | REST_API, JSON | keyed records from any JSON API (KEV, NVD, Federal Register, ClinicalTrials v2, USGS, HIBP, CKAN, Hugging Face…); `{now-2h}` placeholders, `{key}` / `{field.path}` URL templates | |
|
278 |
+| `package` | REST_API | npm, PyPI, crates.io, RubyGems, NuGet, Packagist, Hex, Go proxy, Homebrew, Docker Hub version streams | |
|
279 |
+| `edgar` | REST_API | SEC EDGAR company submissions → filings, 8-K items decoded | |
|
280 |
+| `openapi` | JSON | API contract fingerprints (operations, parameters, deprecations) | |
|
281 |
+| `csv` | FILE | open-data rows (`keyColumn`, `tail`, `skipRows`) — FRED, Valet, ECB, ONS, Treasury | |
|
282 |
+| `pdf` | FILE | text of official PDFs (statements, notices, license agreements) | |
|
283 |
+| `dns` | DNS | A / AAAA / NS / MX / TXT / CAA / SOA / DMARC via public resolvers | |
|
284 |
+| `tls` | TLS | certificate identity, chain, ALPN, expiry | |
|
285 |
+| `headers` | HTTP_HEADERS | security and infrastructure response headers (HSTS, CSP, CDN) | |
|
286 |
+| `rdap` | JSON | registrar, EPP status, nameservers, expiry via the IANA bootstrap | |
|
287 |
+| `discovery` | — | candidate feeds / sitemaps / status pages, every candidate fetched and parsed | |
|
288 |
+| `scrapfly` | — | budgeted anti-bot fallback, only where the source explicitly allows it | |
|
289 |
+ |
|
290 |
+Per-connector documentation: `docs/connectors/*.md`. |
|
291 |
+ |
|
292 |
+## Data model |
|
293 |
+ |
|
294 |
+PostgreSQL 17; migrations in `packages/db/migrations` (`0001_init` … `0006_backfill_states`), all additive. |
|
295 |
+ |
|
296 |
+``` |
|
297 |
+sources ─┬─ sensors ─┬─ sensor_runs |
|
298 |
+ │ ├─ snapshots ─┐ |
|
299 |
+ │ └─ changes ───┼─ events ─┬─ event_entities ─ entities ─┬─ entity_aliases |
|
300 |
+ │ │ ├─ interpretations (versioned) ├─ entity_relations |
|
301 |
+ │ │ └─ event_clusters (timeline, states) |
|
302 |
+ ├─ source_entities └─ urls / url_history |
|
303 |
+ ├─ source_daily entity_daily · metrics_daily · llm_usage · connector_health · discovery_candidates |
|
304 |
+ └─ (kind = custom, owner_token) watchlists · watchlist_items · alerts · notifications · bookmarks · saved_views |
|
305 |
+``` |
|
306 |
+ |
|
307 |
+Key columns on `events`: `event_type`, `title`, `summary`, `why_it_matters`, `importance` + `importance_components`, |
|
308 |
+`confidence`, `novelty`, `signal_score`, `velocity_score`, `impact_score`, `anomaly_score`, `score_reasons`, |
|
309 |
+`change_class`, `field_changes`, `silent_change`, `evidence_label`, `first_party`, `country`, `language`, |
|
310 |
+`fingerprint` (unique), `cluster_id`, `published_at` / `observed_from` / `detected_at` / `processed_at` / |
|
311 |
+`published_to_feed_at`, `detection_latency_ms`, `processing_latency_ms`, a generated `tsvector` for full-text |
|
312 |
+search. Raw evidence is immutable: snapshots and diffs are never rewritten; re-interpretation adds a row to |
|
313 |
+`interpretations`. |
|
314 |
+ |
|
315 |
+## Real-time protocol |
|
316 |
+ |
|
317 |
+`wss://www.websensor.io/api/v1/live` (protocol 2). One Redis subscriber fans out to every client; the engine |
|
318 |
+publishes each event once with its stream id. |
|
319 |
+ |
|
320 |
+``` |
|
321 |
+client → {"subscribe":["events:breaking","entity:org_openai","country:CA"]} |
|
322 |
+client → {"since":"1789105849242-0"} # after a reconnection: replay up to 500 missed events |
|
323 |
+client → {"ping":1} |
|
324 |
+server → {"type":"hello","protocol":2,"channels":[…]} |
|
325 |
+server → {"type":"event","sid":"1789105849242-0","channels":["events:global","events:ai"],"event":{…}} |
|
326 |
+server → {"type":"replay_done","since":"…","count":17} |
|
327 |
+server → {"type":"heartbeat","t":1789105849242} # every 25 s |
|
328 |
+``` |
|
329 |
+ |
|
330 |
+Channels: `events:global`, `events:breaking`, `events:silent`, `events:first-party`, |
|
331 |
+`events:<ai|cyber|finance|health|government|science|products|infrastructure|news>`, |
|
332 |
+`group:<security|reliability|product|commercial|corporate|government|science|transport|sports|web>`, |
|
333 |
+`country:<CC>`, `state:<breaking|developing>`, `type:<event_type>`, `entity:<id>`, `source:<id>`, |
|
334 |
+`watchlist:<id>` (server-side matching of entities, sources, keywords, categories, URLs, event types, countries). |
|
335 |
+The browser client (`apps/web/src/lib/use-live.ts`) reconnects with jittered backoff, detects half-dead sockets |
|
336 |
+with a 70 s watchdog and replays on reconnection; the feed de-duplicates by event id. |
|
337 |
+ |
|
338 |
+## REST API |
|
339 |
+ |
|
340 |
+Public, unauthenticated, 600 requests / minute / IP (`x-ratelimit-*` headers), consistent errors |
|
341 |
+`{ "error": "…", "detail"?: "…" }`, cursor pagination (`nextCursor` → `cursor`), all timestamps UTC. |
|
342 |
+Documentation with examples: https://www.websensor.io/api. |
|
343 |
+ |
|
344 |
+| Endpoint | Purpose | |
|
345 |
+|---|---| |
|
346 |
+| `GET /api/v1/events` | filters `after before category entity source domain sensor cluster importance_min confidence_min signal_min event_type group silent_change first_party confirmed country language change_class q limit cursor order=recent\|importance\|signal`; `q` accepts the search syntax `entity:openai type:pricing_change after:7d silent:true importance:>70 country:CA` | |
|
347 |
+| `GET /api/v1/events/count` · `/events/{id\|slug}` | counts; detail with related events, cluster, change, interpretation versions, snapshots, sensor reliability, page history | |
|
348 |
+| `GET /api/v1/changes/{id}` · `/snapshots/{id}[?raw=1]` · `/snapshots/compare?a=&b=` · `/sensors/{id}/snapshots` | evidence: diff, canonical or raw body, compare any two snapshots, historical memory of a page | |
|
349 |
+| `GET /api/v1/breaking` · `/pulse` · `/radar` · `/trending` · `/explore` · `/stats` | intelligence desks and aggregates (cached 5–30 s) | |
|
350 |
+| `GET /api/v1/clusters` · `/clusters/{id\|slug}` | clusters with propagation timeline, first-party / external signals, lead time | |
|
351 |
+| `GET /api/v1/entities` · `/entities/rank` · `/entities/{id}` · `/entities/{id}/timeline` | entities, WebSensor ranking, insights (35-day heatmap, baseline, anomaly, velocity, rank) | |
|
352 |
+| `GET /api/v1/sources` · `/sources/{id}` · `/sensors/{id}` | registry, source quality score, sensor polling metadata (ETag, Last-Modified, intervals) | |
|
353 |
+| `GET /api/v1/countries[/{code}]` · `/categories/{channel}` · `/domains/{domain}/timeline` · `/urls/history?url=` | country and category desks, domain and URL history | |
|
354 |
+| `GET /api/v1/search?q=` | events, clusters, entities, sources, URLs with the parsed filters echoed | |
|
355 |
+| `GET /api/v1/health/connectors` | connector health, throughput, queue depth, engine heartbeat, failing domains | |
|
356 |
+| `GET /api/v1/feed.rss` | RSS 2.0 with the same filters as `/events` | |
|
357 |
+| Owner-scoped (`X-WebSensor-Owner`) | `/watchlists`, `/alerts` (web or HMAC-signed webhook channel), `/notifications`, `/bookmarks`, `/views`, `/monitors` (custom public URLs, SSRF-checked, per-owner limit) | |
|
358 |
+| Admin (`X-WebSensor-Admin`, only when `WS_ADMIN_TOKEN` is set) | `/admin/ops`, `/admin/failures`, sensor and source actions (`run-now`, `enable`, `disable`, patch), `/admin/sensors/test` (connector dry run), `/admin/sensors/bulk`, `/admin/sources/import?dry_run=1` (JSON or YAML) | |
|
359 |
+ |
|
360 |
+## Frontend |
|
361 |
+ |
|
362 |
+Next.js 16 (App Router, React 19, Tailwind v4). Server components by default; client components only where |
|
363 |
+interaction is needed. |
|
364 |
+ |
|
365 |
+| Route | What it is | |
|
366 |
+|---|---| |
|
367 |
+| `/` | hero line, **live system strip** (sources, sensors, checks/min, events 24 h, breaking, silent, status — real values, refreshed every 10 s), live feed, right rail (breaking now, trending, anomalous), clusters, silent changes, infrastructure pulse | |
|
368 |
+| `/live` | full-width feed; filters persist in the URL (`/live?category=ai&signal_min=70&silent_change=true`); saved views | |
|
369 |
+| `/breaking` | breaking now · developing · recently confirmed · watching | |
|
370 |
+| `/silent` | the silent-change desk: previous state, current state, exact diff | |
|
371 |
+| `/pulse` | what is changing on the Internet right now | |
|
372 |
+| `/radar` | weak signals, clearly labelled as indicators | |
|
373 |
+| `/explore` | trending, breaking, silent, newly detected, most active, unusual, clusters, entities, sources, categories, countries | |
|
374 |
+| `/entity/[id]` | rank, activity, 35-day heatmap, anomaly, tabs overview / live / sources / silent / timeline (1 h → all) / related / metrics | |
|
375 |
+| `/cluster/[slug]` | web-velocity propagation timeline, first-party vs external, lead time | |
|
376 |
+| `/event/[slug]` | **what changed** first: field changes, before / after diff, why it matters (analysis), evidence, history, related signals, score transparency, OG image | |
|
377 |
+| `/source/[id]` · `/sensor/[id]` | source quality, polling metadata, sensors table; sensor history ("how this page looked over time") | |
|
378 |
+| `/category/[channel]` · `/country/[slug]` | real-time desks per category and per country | |
|
379 |
+| `/entities` · `/sources` · `/search` | rankings, registry browsing, advanced search | |
|
380 |
+| `/watchlists` · `/alerts` · `/bookmarks` · `/monitors` | anonymous owner-token tools | |
|
381 |
+| `/health` · `/ops` · `/api` · `/bot` | observability, token-gated operations dashboard, API docs, crawler policy | |
|
382 |
+ |
|
383 |
+Interaction model: click any event → **intelligence drawer** (right on desktop, full-screen sheet on mobile); |
|
384 |
+`⌘K` / `Ctrl+K` command palette (entities, sources, events, clusters, URLs, navigation, commands); three density |
|
385 |
+modes (compact / normal / comfortable) persisted; pause live; `↑ N new events` without scroll jumps; badges |
|
386 |
+`BREAKING · DEVELOPING · SILENT · FIRST PARTY · EXTERNAL · CONFIRMED · ANOMALOUS`; skeleton loaders; real 404s; |
|
387 |
+dark and light themes; audited at 375, 390, 430, 768, 1024, 1440 and 1920 px. |
|
388 |
+ |
|
389 |
+## Repository layout |
|
390 |
+ |
|
391 |
+``` |
|
392 |
+apps/ |
|
393 |
+ api/ Fastify gateway: routes.ts (public), routes-user.ts (owner), routes-admin.ts, intel.ts, live.ts, cache.ts |
|
394 |
+ engine/ scheduler.ts, pipeline.ts, cluster.ts, entities.ts, interpret.ts (Claude), alerts.ts, retention.ts, |
|
395 |
+ registry.ts, seeds.ts, discovery, metrics.ts, cli.ts, validate.ts |
|
396 |
+ web/ Next.js app: src/app (routes), src/components (ui.tsx design system, live-feed, event-drawer, |
|
397 |
+ command-palette, live-strip, brand.tsx…), src/lib (api client, format, feed-filters, use-live, owner) |
|
398 |
+packages/ |
|
399 |
+ core/ taxonomy · scoring · semantic · heuristics · diff · canonical · search · ssrf · schedule · registry-schema |
|
400 |
+ db/ schema.ts + migrations/*.sql |
|
401 |
+ store/ content-addressed blob store |
|
402 |
+ connectors/ fetcher + 16 connector families + discovery + scrapfly |
|
403 |
+config/ |
|
404 |
+ sources.yaml, sources.d/*.yaml the registry |
|
405 |
+docs/ |
|
406 |
+ ARCHITECTURE.md, PRODUCT-BRIEF.md, registry/AUTHORING.md, connectors/*.md, brand/ |
|
407 |
+scripts/ registry generators (EDGAR, Federal Register, web posture, status probe) and pruning helpers |
|
408 |
+deploy/ mld manifest example and README |
|
409 |
+tests/ fixtures (RSS before/after, Atlassian summary, KEV, sitemap…) |
|
410 |
+``` |
|
411 |
+ |
|
412 |
+## Development |
|
413 |
+ |
|
414 |
+```bash |
|
415 |
+createdb websensor && cp .env.example .env |
|
416 |
+pnpm install |
|
417 |
+pnpm db:migrate # or let the engine migrate on start |
|
418 |
+npx tsx apps/engine/src/cli.ts sync # registry → database (~6 s for 3 000 sources) |
|
419 |
+npx tsx apps/engine/src/cli.ts run-due 50 # run 50 due sensors through the pipeline |
|
420 |
+pnpm dev:api # gateway :8260 |
|
421 |
+pnpm dev:engine # scheduler + pipeline |
|
422 |
+NEXT_PUBLIC_API_URL=http://localhost:8260 pnpm dev:web # :8261 — open the site here in dev |
|
423 |
+pnpm test · pnpm typecheck · pnpm --filter @websensor/web build |
|
424 |
+``` |
|
425 |
+ |
|
426 |
+Useful CLI commands (`apps/engine/src/cli.ts`): `sync`, `discover [sourceId…]`, `probe <domain>`, |
|
427 |
+`run-once <sensorId>`, `run-due [n]`, `relink-entities [days]` (re-resolve mentioned entities after alias-rule |
|
428 |
+changes), `refresh-clusters`, `prune-blobs`, `llm-test`. Registry validator: |
|
429 |
+`node node_modules/tsx/dist/cli.mjs apps/engine/src/validate.ts <fragment.yaml> [--all] [--json report.json] |
|
430 |
+[--concurrency 6] [--probe <domain>]`; `scripts/prune-fragment.py <fragment> <report.json> [--keep-empty]` |
|
431 |
+removes failed sensors. |
|
432 |
+ |
|
433 |
+Environment (see `.env.example`): `DATABASE_URL`, `REDIS_URL`, `BLOB_STORE_DIR`, `WS_FETCH_CONCURRENCY`, |
|
434 |
+`WS_PER_HOST_CONCURRENCY` (+ `WS_PER_HOST_OVERRIDES`), `WS_MEANINGFUL_SIGNAL`, `WS_SILENT_MIN_IMPORTANCE`, |
|
435 |
+`WS_RETENTION_*`, `ANTHROPIC_API_KEY` + `WS_LLM_*` (models, daily call budget, thresholds), `SCRAPFLY_API_KEY`, |
|
436 |
+`WS_ADMIN_TOKEN`, `WS_MONITORS_PER_OWNER`, `WS_USER_AGENT`. |
|
437 |
+ |
|
438 |
+Dev tip: a copy of production data for UI work is `ssh <node> 'pg_dump websensor --data-only -t events …' | |
|
439 |
+psql websensor`, then `cli.ts sync`. |
|
440 |
+ |
|
441 |
+## Testing |
|
442 |
+ |
|
443 |
+`pnpm test` runs Vitest across the workspace: |
|
444 |
+ |
|
445 |
+- `packages/core` — diff engines, canonicalization, heuristics, **semantic classifier and field extraction** |
|
446 |
+ (source-simulator fixtures: footer-year bump, relative-time bump, pricing change with deltas, leadership change, |
|
447 |
+ site redesign burst, JSON volatile paths, new feed items), new event classes (KEV → active exploitation, |
|
448 |
+ zero-day, supply chain, guidance, buyback, emergency notice, outbreak, sports), search syntax round-trip, |
|
449 |
+ impact / velocity / signal / breaking / anomaly scoring, title generation for URL-only and batch items; |
|
450 |
+- `packages/connectors` — RSS / Atom / JSON Feed, sitemap, Statuspage, KEV, package registries, EDGAR, OpenAPI, |
|
451 |
+ CSV, DNS/TLS/RDAP normalizers on fixtures; |
|
452 |
+- `apps/engine` — alert rule matching, country / language inference, priority tiers. |
|
453 |
+ |
|
454 |
+Type safety is enforced with `pnpm typecheck` (strict TypeScript across all packages); the web app is also |
|
455 |
+linted with the React Compiler rules. |
|
456 |
+ |
|
457 |
+## Deployment and operations |
|
458 |
+ |
|
459 |
+Production runs on the MacLustr cluster, orchestrated by `mld` from the M1M32 gateway: |
|
460 |
+ |
|
461 |
+```bash |
|
462 |
+~/Desktop/cluster-skill/mld stage . websensor |
|
463 |
+~/Desktop/cluster-skill/mld deploy websensor --node M4M64b |
|
464 |
+``` |
|
465 |
+ |
|
466 |
+Post-sync hooks (fail-fast): `pnpm install --frozen-lockfile` → `createdb` + migrations → registry sync → |
|
467 |
+`next build`. Processes (PM2): `websensor-engine`, `websensor-api` (:8260, published as |
|
468 |
+https://www.websensor.io through the MacLustr Tunnel — WireGuard to the OVH gateway BHS64 with Caddy TLS), |
|
469 |
+`websensor-web` (:8261, loopback). Secrets live only in the manifest on the gateway. After a deploy that changes |
|
470 |
+alias or scoring rules: `cli.ts relink-entities 7` and `cli.ts refresh-clusters` on the node. |
|
471 |
+ |
|
472 |
+Observability: Prometheus metrics (`/api/metrics`, engine `:8262/metrics`: checks by outcome, bytes, changes by |
|
473 |
+class, events by type, LLM calls and tokens, stage latencies, suppressed candidates, queue depth, inflight), |
|
474 |
+`/health` (connectors, throughput, 304 ratio, failing domains, slowest sensors), `/ops` (engine heartbeat, queue, |
|
475 |
+storage, LLM usage, recent errors, connector dry-run, import). Retention keeps the database and blob store bounded |
|
476 |
+without ever deleting event evidence. Details: `deploy/README.md`, `docs/ARCHITECTURE.md`. |
|
477 |
+ |
|
478 |
+## Security and crawling etiquette |
|
479 |
+ |
|
480 |
+- Every URL the engine touches — seeds, discovered candidates, redirects, Scrapfly targets, webhooks, custom |
|
481 |
+ monitors — passes `assertUrlAllowed()`: private ranges, loopback, link-local, cloud metadata endpoints, |
|
482 |
+ `.maclustr.io` / `.ts.net`, single-label hosts and NAT64-embedded private IPv4 are blocked, and the HTTP |
|
483 |
+ dispatcher only connects to validated addresses (DNS-rebinding safe). |
|
484 |
+- Clear `User-Agent` (`WebSensorBot/0.2 (+https://www.websensor.io/bot; contact@spboucher.ai)`), conditional |
|
485 |
+ requests, per-host concurrency, adaptive intervals, backoff, timeouts and a per-domain circuit breaker. Official |
|
486 |
+ APIs, feeds and structured endpoints are preferred; WebSensor does not bypass authentication or anti-bot |
|
487 |
+ protections (Scrapfly is a budgeted fallback only where a source explicitly allows it). |
|
488 |
+- Webhooks: HTTPS only, public hosts only, `X-WebSensor-Signature: sha256=<HMAC>`; custom monitors: public |
|
489 |
+ URLs only, tested before activation, hard per-owner limit, never shown in public feeds. |
|
490 |
+- Admin API disabled unless `WS_ADMIN_TOKEN` is set; owner tools use an anonymous browser token; rate limiting |
|
491 |
+ on every `/api/*` route; strict input validation (zod); snapshots served raw get `nosniff` and a restrictive CSP. |
|
492 |
+ |
|
493 |
+## Roadmap |
|
494 |
+ |
|
495 |
+World map for geolocated events; e-mail / Slack / Discord / Telegram / push alert channels (the channel |
|
496 |
+abstraction is in place); accounts and API keys; embeddings-based similarity; MinIO/S3 blob driver; browser |
|
497 |
+rendering for the few high-value client-rendered pages; MCP server; more depth fragments. |
|
498 |
+ |
|
499 |
+## Credits |
|
500 |
+ |
|
501 |
+WebSensor is designed and built by **Simon-Pierre Boucher** — [contact@spboucher.ai](mailto:contact@spboucher.ai). |
|
502 |
+It runs on [MacLustr](https://www.maclustr.io), an Apple-silicon compute cluster in Québec. Source of truth: |
|
503 |
+`websensor.git` on spbgit (git.spboucher.ai). AI-generated summaries are labelled as analysis and never replace |
|
504 |
+the original evidence. |