SPB Git
spboucher.ai

spb/websensor

Public

registry: 22 site-class fragments (2 681 organizations, 4 052 validated sensors) + 10 new connector families

- config/sources.d/*.yaml merged on top of the founding file by apps/engine/src/seeds.ts (`extend: true`
  adds sensors to an existing source); classes: open source, central banks & finance, energy/climate/weather,
  telecom & internet infra, retail/consumer/travel, gaming & entertainment, universities & research,
  international orgs/NGOs/standards, sports, crypto, transport/aviation/space, consumer safety/food/agri,
  housing/labour/open data, enterprise SaaS, health systems & medtech, politics/elections/courts, SEC EDGAR
  (254 issuers), package registries, web posture, OpenAPI, status JSON, documents & data
- connectors: package (10 registries), edgar, statusjson (Instatus/incident.io/Status.io), openapi, csv, pdf
  (unpdf), dns, tls, headers, rdap (IANA bootstrap); fetcher keepAllHeaders
- heuristics: connector-class priors + event types api_change, dns_change, domain_registration_change,
  crawler_policy_change; 8-K items → earnings/leadership_change/breach/acquisition
- registry validator without DB (apps/engine/src/validate.ts, --probe), generators gen-edgar.py,
  gen-web-posture.py, prune-fragment.py; docs/registry/AUTHORING.md + docs/connectors/*.md (16)
- SSRF: NAT64 addresses judged by the embedded IPv4 (IPv6-only hotspots)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Sep 8, 2026) parent c4c4aa0 Browse files

19 changed files +19,898 −61

19 changed files
Load all 19 files Unified Split
modified CLAUDE.md +30 −9
@@ -19,23 +19,40 @@ design is `docs/PRODUCT-BRIEF.md`; this file is the working guide for the code.
19 bodies, canonical representations and full diffs live there, never in Postgres. Interface ready for S3/MinIO. 19 bodies, canonical representations and full diffs live there, never in Postgres. Interface ready for S3/MinIO.
20 - `packages/connectors` — connector SDK (`WebSensorConnector.fetch/normalize`), safe fetcher (conditional GET, 20 - `packages/connectors` — connector SDK (`WebSensorConnector.fetch/normalize`), safe fetcher (conditional GET,
21 manual redirects validated per hop, HTTP/2→1.1 pin on NGHTTP2 errors, browser-UA second attempt on 21 manual redirects validated per hop, HTTP/2→1.1 pin on NGHTTP2 errors, browser-UA second attempt on
22− 403/resets, size + time limits), connectors: `http` (HTML/JSON/text/HEAD), `rss` (RSS/Atom/RDF/JSON Feed), 22+ 403/resets, size + time limits, `keepAllHeaders`), 16 connector families: `http` (HTML/JSON/text/HEAD), `rss`
23− `sitemap` (index + news), `statuspage` (Atlassian v2 summary), `github` (releases/tags/commits Atom, 23+ (RSS/Atom/RDF/JSON Feed), `sitemap` (index + news), `statuspage` (Atlassian v2 summary), `statusjson` (Instatus /
24− advisories REST), `jsonlist` (keyed records from any JSON API; `{now-2h}` placeholders), `discovery` 24+ incident.io / Status.io), `github` (releases/tags/commits Atom, advisories REST), `jsonlist` (keyed records from
25− (robots sitemaps, `<link rel=alternate>`, well-known feed paths, linked status pages — every candidate is 25+ any JSON API; `{now-2h}` placeholders), `package` (npm/PyPI/crates/RubyGems/NuGet/Packagist/Hex/Go proxy/Homebrew/
26− fetched and parsed), `scrapfly` (fallback, budgeted, only when `fallback.scrapfly` is set on the source). 26+ Docker Hub version streams), `edgar` (SEC submissions → filings, 8-K items decoded), `openapi` (API contract
27+ fingerprints), `csv` (open-data rows, `tail`), `pdf` (pdf.js text via unpdf), `dns` (A/AAAA/NS/MX/TXT/CAA/SOA/DMARC),
28+ `tls` (certificate identity, chain, ALPN, expiry), `headers` (security/infra response headers), `rdap` (registrar,
29+ EPP status, expiry via IANA bootstrap), `discovery` (robots sitemaps, `<link rel=alternate>`, well-known feed
30+ paths, linked status pages — every candidate is fetched and parsed), `scrapfly` (fallback, budgeted, only when
31+ `fallback.scrapfly` is set on the source). `dns://` and `tls://` sensor URLs are accepted (host only).
27 - `apps/engine` — scheduler (`FOR UPDATE SKIP LOCKED` claims, global + per-host concurrency), pipeline 32 - `apps/engine` — scheduler (`FOR UPDATE SKIP LOCKED` claims, global + per-host concurrency), pipeline
28 (fetch → normalize → snapshot → diff → heuristics → change → event: entities, novelty, LLM interpretation, 33 (fetch → normalize → snapshot → diff → heuristics → change → event: entities, novelty, LLM interpretation,
29 clustering, importance/confidence, silent-change detection, publish to Redis), registry sync from 34 clustering, importance/confidence, silent-change detection, publish to Redis), registry sync from
30− `config/sources.yaml`, discovery, connector health rollups, Prometheus metrics on :8262. CLI: `src/cli.ts` 35+ `config/sources.yaml` + `config/sources.d/`, discovery, connector health rollups, Prometheus metrics on :8262.
31− (`sync`, `discover`, `probe <domain>`, `run-once <sensor>`, `run-due [n]`). 36+ CLI: `src/cli.ts` (`sync`, `discover`, `probe <domain>`, `run-once <sensor>`, `run-due [n]`); `src/validate.ts`
37+ (registry validator, see below).
32 - `apps/api` — Fastify gateway (:8260): REST `/api/v1/*`, WebSocket `/api/v1/live` (Redis pub/sub fan-out, 38 - `apps/api` — Fastify gateway (:8260): REST `/api/v1/*`, WebSocket `/api/v1/live` (Redis pub/sub fan-out,
33 channels `events:*`, `entity:*`, `source:*`, `watchlist:*`), `/api/v1/feed.rss`, `/api/health` `/api/ready` `/api/metrics`, apex→www 39 channels `events:*`, `entity:*`, `source:*`, `watchlist:*`), `/api/v1/feed.rss`, `/api/health` `/api/ready` `/api/metrics`, apex→www
34 redirect, and a reverse proxy to the Next.js app for everything else. 40 redirect, and a reverse proxy to the Next.js app for everything else.
35 - `apps/web` — Next.js 16 frontend (:8261, loopback). Pages: live, breaking, explore, sources, entities, 41 - `apps/web` — Next.js 16 frontend (:8261, loopback). Pages: live, breaking, explore, sources, entities,
36 timelines, silent changes, watchlists, alerts, event detail with diff viewer, health, API docs. 42 timelines, silent changes, watchlists, alerts, event detail with diff viewer, health, API docs.
37−- `config/sources.yaml` — the 271-organization registry (incl. governments and news media; news sources run with `llm: false`, heuristics only) (sources, curated sensors, product entities, discovery 43+- `config/sources.yaml` — the founding 271-organization registry (incl. governments and news media; news sources
38− flags, fallbacks). `docs/connectors/*.md` document each connector family. 44+ run with `llm: false`, heuristics only) + `config/sources.d/*.yaml` fragments, one per **site class** (open source,
45+ central banks & finance, energy/climate/weather, telecom & internet infrastructure, retail/consumer/travel,
46+ gaming & entertainment, universities & research, international orgs/NGOs/standards, sports, crypto,
47+ transport/aviation/space, consumer safety/food/agri, housing/labour/open data, enterprise SaaS, health systems &
48+ medtech, politics/elections/courts, EDGAR filings, package registries, web posture, OpenAPI, status JSON,
49+ documents & data). Fragments are merged by `apps/engine/src/seeds.ts` in file-name order; an entry with
50+ `extend: true` adds sensors/products/aliases to a source declared earlier. Authoring guide + connector
51+ cheat-sheet: `docs/registry/AUTHORING.md`; `docs/connectors/*.md` document each connector family.
52+- Registry validator (no DB): `node node_modules/tsx/dist/cli.mjs apps/engine/src/validate.ts [fragment.yaml] [--all]
53+ [--source id] [--connector key] [--json report.json]` runs every curated sensor through its connector and prints
54+ OK/WARN/FAIL; `--probe <domain>` is a discovery dry-run. `scripts/prune-fragment.py` removes FAIL/WARN sensors from
55+ a fragment using the JSON report; `scripts/gen-web-posture.py` regenerates the web-posture fragment.
39 56
40 ## Rules 57 ## Rules
41 - Every URL the engine touches — seeds, discovered candidates, redirects, Scrapfly targets — goes through 58 - Every URL the engine touches — seeds, discovered candidates, redirects, Scrapfly targets — goes through
@@ -53,6 +70,10 @@ design is `docs/PRODUCT-BRIEF.md`; this file is the working guide for the code.
53 - Never label inference as fact: events carry `evidence_label` (OBSERVED / INFERRED / CONFIRMED / UNCONFIRMED) 70 - Never label inference as fact: events carry `evidence_label` (OBSERVED / INFERRED / CONFIRMED / UNCONFIRMED)
54 and the interpretation keeps `observed` and `inferred` apart. Silent changes are flagged, never asserted as 71 and the interpretation keeps `observed` and `inferred` apart. Silent changes are flagged, never asserted as
55 "unannounced" without checking recent announcement-type events of the same source. 72 "unannounced" without checking recent announcement-type events of the same source.
73+- Every sensor in the registry has been fetched and parsed successfully by the validator before being committed;
74+ a fragment that fails validation is not merged. Never add sensors to the registry without running it.
75+- SSRF policy: NAT64 addresses (`64:ff9b::/96`) are judged by the embedded IPv4 address (IPv6-only networks with
76+ 464XLAT resolve every IPv4-only host that way), not blocked wholesale.
56 - Do not add sensors that are blocked with 403 by design (Akamai/Cloudflare bot management) unless the source 77 - Do not add sensors that are blocked with 403 by design (Akamai/Cloudflare bot management) unless the source
57 has `fallback.scrapfly: true` and a tier ≥ C; Scrapfly is a budgeted fallback, not the foundation. 78 has `fallback.scrapfly: true` and a tier ≥ C; Scrapfly is a budgeted fallback, not the foundation.
58 - All timestamps UTC. Ids are prefixed (`src_`, `sen_`, `snap_`, `chg_`, `evt_`, `clu_`, `ent_`…). 79 - All timestamps UTC. Ids are prefixed (`src_`, `sen_`, `snap_`, `chg_`, `evt_`, `clu_`, `ent_`…).
modified README.md +15 −3
@@ -13,9 +13,21 @@ Sources → Sensors/Connectors → Fetch (conditional) → Snapshot (immutable)
13 → Redis stream → WebSocket gateway → www.websensor.io 13 → Redis stream → WebSocket gateway → www.websensor.io
14 ``` 14 ```
15 15
16−- 271 organizations (tech, cloud, cyber, finance, government, health, science, automotive, commerce, internet, 26 governments/public institutions, 45 news media), ~370 curated sensors at launch (`config/sources.yaml`), plus validated discovery. 16+- **2 681 organizations, 4 052 curated sensors** (2026-09-08), every one fetched and parsed by the registry validator
17−- Connector families: HTTP (HTML/JSON/HEAD), RSS/Atom/JSON Feed, sitemaps, Statuspage, GitHub, JSON APIs 17+ before being committed: the founding 271 (`config/sources.yaml`) plus 22 site-class fragments in
18− (CISA KEV, NVD, Federal Register, USGS, HIBP, Google Cloud incidents…), Scrapfly fallback. 18+ `config/sources.d/` — open source, central banks & finance, energy/climate/weather, telecom & internet
19+ infrastructure, retail/consumer/travel, gaming & entertainment, universities & research, international
20+ organizations/NGOs/standards, sports, crypto, transport/aviation/space, consumer safety/food/agriculture,
21+ housing/labour/open data, enterprise SaaS, health systems & medtech, politics/elections/courts, SEC EDGAR filings
22+ (254 issuers), package registries, web posture (robots.txt, TLS, DNS, headers, RDAP), OpenAPI contracts, status
23+ JSON pages, documents & data files — plus validated discovery.
24+- **16 connector families**: HTTP (HTML/JSON/HEAD), RSS/Atom/JSON Feed, sitemaps, Statuspage, Instatus/incident.io/
25+ Status.io (`statusjson`), GitHub, JSON APIs (`jsonlist`), package registries (npm, PyPI, crates.io, RubyGems, NuGet,
26+ Packagist, Hex, Go proxy, Homebrew, Docker Hub), SEC EDGAR submissions (8-K items decoded), OpenAPI/Swagger
27+ contract fingerprints, CSV/TSV time series (FRED, ECB, NOAA…), PDF text (pdf.js), DNS records, TLS certificates,
28+ HTTP security headers, RDAP registration — and a budgeted Scrapfly fallback.
29+- Authoring guide and connector cheat-sheet: `docs/registry/AUTHORING.md`; validator:
30+ `node node_modules/tsx/dist/cli.mjs apps/engine/src/validate.ts <fragment.yaml>` (no database needed).
19 - Every event is auditable: event → change → snapshot A/B → fetch run → sensor → source, with raw evidence. 31 - Every event is auditable: event → change → snapshot A/B → fetch run → sensor → source, with raw evidence.
20 - Observed vs inferred is always labelled; silent (unannounced) changes are flagged. 32 - Observed vs inferred is always labelled; silent (unannounced) changes are flagged.
21 33
modified apps/engine/src/config.ts +2 −0
@@ -8,6 +8,8 @@ export const config = {
8 databaseUrl: env.DATABASE_URL ?? "postgres://localhost:5432/websensor", 8 databaseUrl: env.DATABASE_URL ?? "postgres://localhost:5432/websensor",
9 redisUrl: env.REDIS_URL ?? "redis://127.0.0.1:6379", 9 redisUrl: env.REDIS_URL ?? "redis://127.0.0.1:6379",
10 sourcesFile: env.WS_SOURCES_FILE ?? "./config/sources.yaml", 10 sourcesFile: env.WS_SOURCES_FILE ?? "./config/sources.yaml",
11+ /** fragments merged on top of sourcesFile (sorted by file name; `extend: true` entries add sensors to existing sources) */
12+ sourcesDir: env.WS_SOURCES_DIR ?? "./config/sources.d",
11 entitiesFile: env.WS_ENTITIES_FILE ?? "./config/entities.yaml", 13 entitiesFile: env.WS_ENTITIES_FILE ?? "./config/entities.yaml",
12 fetchConcurrency: Number(env.WS_FETCH_CONCURRENCY ?? 16), 14 fetchConcurrency: Number(env.WS_FETCH_CONCURRENCY ?? 16),
13 perHostConcurrency: Number(env.WS_PER_HOST_CONCURRENCY ?? 2), 15 perHostConcurrency: Number(env.WS_PER_HOST_CONCURRENCY ?? 2),
modified apps/engine/src/registry.ts +8 −49
@@ -1,7 +1,4 @@
1−import { readFileSync } from "node:fs"; 1+import { newId, slugify, sourceImportanceFromTier } from "@websensor/core";
2−import YAML from "yaml";
3−import { z } from "zod";
4−import { SENSOR_TYPES, TIERS, newId, slugify, sourceImportanceFromTier } from "@websensor/core";
5 import { db, discoveryCandidates, entities, entityAliases, eq, sensors, sourceEntities, sources, sql, textArray } from "@websensor/db"; 2 import { db, discoveryCandidates, entities, entityAliases, eq, sensors, sourceEntities, sources, sql, textArray } from "@websensor/db";
6 import { discoverDomain } from "@websensor/connectors"; 3 import { discoverDomain } from "@websensor/connectors";
7 import { config, log } from "./config"; 4 import { config, log } from "./config";
@@ -11,52 +8,14 @@ import { config, log } from "./config";
11 * Booleans under `discover:` only allow discovery; every candidate is validated by a real 8 * Booleans under `discover:` only allow discovery; every candidate is validated by a real
12 * fetch+parse before becoming a sensor. 9 * fetch+parse before becoming a sensor.
13 */ 10 */
14−const sensorSchema = z.object({ 11+export type { SourceSeed } from "./seeds";
15− id: z.string().optional(), 12+import { loadSeedsDetailed, type SourceSeed } from "./seeds";
16− name: z.string(),
17− url: z.string().url(),
18− type: z.enum(SENSOR_TYPES),
19− connector: z.string().default("http"),
20− tier: z.enum(TIERS).optional(),
21− interval: z.number().int().positive().optional(),
22− weight: z.number().positive().optional(),
23− config: z.record(z.string(), z.unknown()).default({}),
24−});
25 13
26−const sourceSchema = z.object({ 14+export function loadSeeds(file = config.sourcesFile, dir = config.sourcesDir): SourceSeed[] {
27− id: z.string(), 15+ const { seeds, issues, files } = loadSeedsDetailed(file, dir);
28− name: z.string(), 16+ for (const i of issues) log.error({ file: i.file, source: i.source, issue: i.message }, "invalid source seed");
29− domain: z.string(), 17+ log.info({ files: files.length, sources: seeds.length, sensors: seeds.reduce((n, s) => n + s.sensors.length, 0) }, "registry seeds loaded");
30− homepage: z.string().url().optional(), 18+ return seeds;
31− description: z.string().optional(),
32− categories: z.array(z.string()).default([]),
33− tier: z.enum(TIERS).default("B"),
34− weight: z.number().positive().default(1),
35− entity_type: z.string().default("organization"),
36− aliases: z.array(z.string()).default([]),
37− products: z.array(z.object({ id: z.string().optional(), name: z.string(), type: z.string().default("product"), aliases: z.array(z.string()).default([]) })).default([]),
38− discover: z.object({ rss: z.boolean().optional(), sitemap: z.boolean().optional(), status: z.boolean().optional(), pages: z.boolean().optional() }).default({}),
39− fallback: z.object({ firecrawl: z.boolean().optional(), scrapfly: z.boolean().optional() }).default({}),
40− sensors: z.array(sensorSchema).default([]),
41− notes: z.string().optional(),
42− enabled: z.boolean().default(true),
43− /** false = heuristics only (high-volume feeds such as news wires) */
44− llm: z.boolean().default(true),
45−});
46−export type SourceSeed = z.infer<typeof sourceSchema>;
47−
48−export function loadSeeds(file = config.sourcesFile): SourceSeed[] {
49− const raw = YAML.parse(readFileSync(file, "utf8")) as { sources: unknown[] };
50− const out: SourceSeed[] = [];
51− for (const s of raw.sources) {
52− const parsed = sourceSchema.safeParse(s);
53− if (!parsed.success) {
54− log.error({ issues: parsed.error.issues, source: (s as { id?: string }).id }, "invalid source seed");
55− continue;
56− }
57− out.push(parsed.data);
58− }
59− return out;
60 } 19 }
61 20
62 export async function syncRegistry(seeds = loadSeeds()): Promise<{ sources: number; sensors: number }> { 21 export async function syncRegistry(seeds = loadSeeds()): Promise<{ sources: number; sensors: number }> {
added apps/engine/src/seeds.ts +152 −0
@@ -0,0 +1,152 @@
1+import { existsSync, readdirSync, readFileSync, statSync } from "node:fs";
2+import { join } from "node:path";
3+import YAML from "yaml";
4+import { z } from "zod";
5+import { SENSOR_TYPES, TIERS } from "@websensor/core";
6+
7+/**
8+ * Source registry seeds. `config/sources.yaml` is the founding file; every `config/sources.d/*.yaml`
9+ * (sorted by name) is merged on top. A fragment entry may carry `extend: true` to add sensors,
10+ * products and aliases to a source declared earlier (same `id`) instead of redefining it — used to
11+ * attach connector-class sensors (EDGAR filings, TLS/DNS posture, package registries…) to existing
12+ * organizations without touching the founding file.
13+ *
14+ * This module has no database dependency so that the validator CLI can load seeds anywhere.
15+ */
16+export const sensorSchema = z.object({
17+ id: z.string().optional(),
18+ name: z.string(),
19+ url: z.string().url(),
20+ type: z.enum(SENSOR_TYPES),
21+ connector: z.string().default("http"),
22+ tier: z.enum(TIERS).optional(),
23+ interval: z.number().int().positive().optional(),
24+ weight: z.number().positive().optional(),
25+ config: z.record(z.string(), z.unknown()).default({}),
26+});
27+export type SensorSeed = z.infer<typeof sensorSchema>;
28+
29+export const productSchema = z.object({ id: z.string().optional(), name: z.string(), type: z.string().default("product"), aliases: z.array(z.string()).default([]) });
30+
31+export const sourceSchema = z.object({
32+ id: z.string().regex(/^[a-z0-9][a-z0-9-]*$/, "id must be kebab-case"),
33+ name: z.string(),
34+ domain: z.string(),
35+ homepage: z.string().url().optional(),
36+ description: z.string().optional(),
37+ categories: z.array(z.string()).default([]),
38+ tier: z.enum(TIERS).default("B"),
39+ weight: z.number().positive().default(1),
40+ entity_type: z.string().default("organization"),
41+ aliases: z.array(z.string()).default([]),
42+ products: z.array(productSchema).default([]),
43+ discover: z.object({ rss: z.boolean().optional(), sitemap: z.boolean().optional(), status: z.boolean().optional(), pages: z.boolean().optional() }).default({}),
44+ fallback: z.object({ firecrawl: z.boolean().optional(), scrapfly: z.boolean().optional() }).default({}),
45+ sensors: z.array(sensorSchema).default([]),
46+ notes: z.string().optional(),
47+ enabled: z.boolean().default(true),
48+ /** false = heuristics only (high-volume feeds such as news wires) */
49+ llm: z.boolean().default(true),
50+ /** fragment-only: merge into an already declared source instead of redefining it */
51+ extend: z.boolean().default(false),
52+});
53+export type SourceSeed = z.infer<typeof sourceSchema>;
54+
55+/** Partial schema for `extend: true` entries — only `id` is required. */
56+const extendSchema = z.object({
57+ id: z.string(),
58+ extend: z.literal(true),
59+ aliases: z.array(z.string()).default([]),
60+ products: z.array(productSchema).default([]),
61+ sensors: z.array(sensorSchema).default([]),
62+ categories: z.array(z.string()).default([]),
63+ fallback: z.object({ firecrawl: z.boolean().optional(), scrapfly: z.boolean().optional() }).optional(),
64+ notes: z.string().optional(),
65+});
66+
67+export interface SeedIssue {
68+ file: string;
69+ source: string | undefined;
70+ message: string;
71+}
72+
73+export interface LoadedSeeds {
74+ seeds: SourceSeed[];
75+ issues: SeedIssue[];
76+ files: string[];
77+ /** source id → files that declared or extended it */
78+ origins: Map<string, string[]>;
79+}
80+
81+export function listSeedFiles(file: string, dir: string): string[] {
82+ const files = existsSync(file) ? [file] : [];
83+ if (existsSync(dir) && statSync(dir).isDirectory()) {
84+ for (const f of readdirSync(dir).sort()) if (/\.ya?ml$/.test(f) && !f.startsWith("_") && !f.startsWith(".")) files.push(join(dir, f));
85+ }
86+ return files;
87+}
88+
89+export function loadSeedsDetailed(file: string, dir: string): LoadedSeeds {
90+ const files = listSeedFiles(file, dir);
91+ const byId = new Map<string, SourceSeed>();
92+ const origins = new Map<string, string[]>();
93+ const issues: SeedIssue[] = [];
94+ const touch = (id: string, f: string): void => {
95+ const arr = origins.get(id) ?? [];
96+ if (!arr.includes(f)) arr.push(f);
97+ origins.set(id, arr);
98+ };
99+ for (const f of files) {
100+ let raw: { sources?: unknown[] };
101+ try {
102+ raw = (YAML.parse(readFileSync(f, "utf8")) ?? {}) as { sources?: unknown[] };
103+ } catch (e) {
104+ issues.push({ file: f, source: undefined, message: `YAML parse error: ${(e as Error).message}` });
105+ continue;
106+ }
107+ for (const s of raw.sources ?? []) {
108+ const id = (s as { id?: string })?.id;
109+ if ((s as { extend?: boolean })?.extend === true) {
110+ const parsed = extendSchema.safeParse(s);
111+ if (!parsed.success) {
112+ issues.push({ file: f, source: id, message: parsed.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ") });
113+ continue;
114+ }
115+ const base = byId.get(parsed.data.id);
116+ if (!base) {
117+ issues.push({ file: f, source: id, message: "extend: true but no source with this id was declared before this file" });
118+ continue;
119+ }
120+ touch(base.id, f);
121+ base.aliases = [...new Set([...base.aliases, ...parsed.data.aliases])];
122+ base.products = [...base.products, ...parsed.data.products.filter((p) => !base.products.some((q) => q.name === p.name))];
123+ base.categories = [...new Set([...base.categories, ...parsed.data.categories])];
124+ base.sensors = [...base.sensors, ...parsed.data.sensors.filter((n) => !base.sensors.some((o) => o.url === n.url))];
125+ if (parsed.data.fallback) base.fallback = { ...base.fallback, ...parsed.data.fallback };
126+ if (parsed.data.notes) base.notes = [base.notes, parsed.data.notes].filter(Boolean).join(" ");
127+ continue;
128+ }
129+ const parsed = sourceSchema.safeParse(s);
130+ if (!parsed.success) {
131+ issues.push({ file: f, source: id, message: parsed.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ") });
132+ continue;
133+ }
134+ if (byId.has(parsed.data.id)) {
135+ issues.push({ file: f, source: parsed.data.id, message: "duplicate source id (use extend: true to add sensors to an existing source)" });
136+ continue;
137+ }
138+ byId.set(parsed.data.id, parsed.data);
139+ touch(parsed.data.id, f);
140+ }
141+ }
142+ // Sensor-level sanity: duplicate URLs inside a source.
143+ for (const s of byId.values()) {
144+ const seen = new Set<string>();
145+ for (const sen of s.sensors) {
146+ const k = sen.url.replace(/\/$/, "");
147+ if (seen.has(k)) issues.push({ file: "-", source: s.id, message: `duplicate sensor url ${sen.url}` });
148+ seen.add(k);
149+ }
150+ }
151+ return { seeds: [...byId.values()], issues, files, origins };
152+}
added apps/engine/src/validate.ts +175 −0
@@ -0,0 +1,175 @@
1+import { writeFileSync } from "node:fs";
2+import { resolve } from "node:path";
3+import type { SensorEndpoint, Tier } from "@websensor/core";
4+import { closeDispatcher, discoverDomain, getConnector, NormalizeError } from "@websensor/connectors";
5+import { loadSeedsDetailed, type SourceSeed } from "./seeds";
6+
7+/**
8+ * Registry validator — no database. Loads the seeds (founding file + fragments), then runs every
9+ * curated sensor through its connector (fetch + normalize) exactly as the engine would, and prints
10+ * one line per sensor:
11+ *
12+ * OK 200 list items=42 openai_news_feed https://…
13+ * WARN 200 text thin acme_pricing https://… (JS shell / interstitial)
14+ * FAIL 403 - - foo_blog_feed https://… http_403
15+ *
16+ * Usage (from the repo root):
17+ * node node_modules/tsx/dist/cli.mjs apps/engine/src/validate.ts [fragment.yaml…] [--all]
18+ * [--source id[,id…]] [--connector key] [--concurrency 8] [--json report.json] [--quiet]
19+ * node node_modules/tsx/dist/cli.mjs apps/engine/src/validate.ts --probe <domain> (discovery dry-run, no DB)
20+ *
21+ * Without a file argument the founding file is validated; with fragment paths only the sources
22+ * declared or extended by those fragments are checked (the whole registry is still loaded so that
23+ * `extend: true` resolves). Exit code 1 when any sensor FAILs or any seed is invalid.
24+ */
25+interface Result {
26+ source: string;
27+ sensor: string;
28+ connector: string;
29+ type: string;
30+ url: string;
31+ status: "OK" | "WARN" | "FAIL";
32+ http: number;
33+ mode?: string;
34+ items?: number;
35+ note: string;
36+ ms: number;
37+}
38+
39+function arg(name: string, def?: string): string | undefined {
40+ const i = process.argv.indexOf(`--${name}`);
41+ return i >= 0 ? process.argv[i + 1] : def;
42+}
43+const flag = (name: string): boolean => process.argv.includes(`--${name}`);
44+
45+async function checkSensor(src: SourceSeed, sen: SourceSeed["sensors"][number]): Promise<Result> {
46+ const t0 = Date.now();
47+ const id = sen.id ?? `${src.id}_${sen.name.toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, "")}`;
48+ const base: Omit<Result, "status" | "http" | "note" | "ms"> = { source: src.id, sensor: id, connector: sen.connector, type: sen.type, url: sen.url };
49+ let connector;
50+ try {
51+ connector = getConnector(sen.connector);
52+ } catch (e) {
53+ return { ...base, status: "FAIL", http: 0, note: (e as Error).message, ms: 0 };
54+ }
55+ const endpoint: SensorEndpoint = { id, sourceId: src.id, name: sen.name, url: sen.url, type: sen.type, tier: (sen.tier ?? src.tier) as Tier, connector: sen.connector, config: sen.config, etag: null, lastModified: null, state: null };
56+ const obs = await connector.fetch(endpoint);
57+ if (obs.error) return { ...base, status: "FAIL", http: obs.meta.status, note: `${obs.error.code}: ${obs.error.message.slice(0, 160)}`, ms: Date.now() - t0 };
58+ if (obs.meta.status >= 400) return { ...base, status: "FAIL", http: obs.meta.status, note: `http_${obs.meta.status}${obs.meta.finalUrl !== sen.url ? " → " + obs.meta.finalUrl : ""}`, ms: Date.now() - t0 };
59+ try {
60+ const norm = await connector.normalize(endpoint, obs);
61+ const items = norm.items?.length;
62+ const thin = Boolean((norm.extra as { thin?: boolean } | undefined)?.thin);
63+ let status: Result["status"] = "OK";
64+ let note = "";
65+ if (norm.mode === "list" && (items ?? 0) === 0) {
66+ status = "WARN";
67+ note = "empty list";
68+ } else if (thin || norm.extractionConfidence < 0.5) {
69+ status = "WARN";
70+ note = `thin (confidence ${norm.extractionConfidence})`;
71+ } else if (norm.mode === "text" && (norm.text?.length ?? 0) < (sen.type === "FILE" ? 20 : 200)) {
72+ status = "WARN";
73+ note = `short text (${norm.text?.length ?? 0} chars)`;
74+ }
75+ if (["http", "rss", "sitemap", "statuspage", "github", "jsonlist", "statusjson", "openapi", "csv", "pdf", "headers"].includes(sen.connector) && obs.meta.finalUrl && obs.meta.finalUrl.replace(/\/$/, "") !== sen.url.replace(/\/$/, "")) note = `${note ? note + "; " : ""}redirect → ${obs.meta.finalUrl}`;
76+ return { ...base, status, http: obs.meta.status, mode: norm.mode, items, note, ms: Date.now() - t0 };
77+ } catch (e) {
78+ const msg = e instanceof NormalizeError ? `${e.code}: ${e.message}` : (e as Error).message;
79+ return { ...base, status: "FAIL", http: obs.meta.status, note: msg.slice(0, 200), ms: Date.now() - t0 };
80+ }
81+}
82+
83+async function main(): Promise<void> {
84+ const probe = arg("probe");
85+ if (probe) {
86+ const r = await discoverDomain(probe, { probePages: true });
87+ if (!r.length) console.log(`no validated endpoint found for ${probe}`);
88+ for (const e of r) console.log(`${e.type.padEnd(10)} ${e.connector.padEnd(10)} ${e.value.toFixed(2)} ${String(e.itemCount ?? "").padStart(5)} ${e.url} (${e.evidence}${e.title ? " · " + e.title : ""})`);
89+ return;
90+ }
91+ const files = process.argv.slice(2).filter((a) => !a.startsWith("--") && /\.ya?ml$/.test(a) && (process.argv[process.argv.indexOf(a) - 1] ?? "").startsWith("--") === false);
92+ const baseFile = process.env.WS_SOURCES_FILE ?? "./config/sources.yaml";
93+ const dir = process.env.WS_SOURCES_DIR ?? "./config/sources.d";
94+ const loaded = loadSeedsDetailed(baseFile, dir);
95+ for (const i of loaded.issues) console.error(`SEED ${i.file} ${i.source ?? "-"}: ${i.message}`);
96+
97+ const wanted = new Set(files.map((f) => resolve(f)));
98+ const onlySources = arg("source")?.split(",").filter(Boolean);
99+ const onlyConnector = arg("connector");
100+ let targets = loaded.seeds;
101+ if (!flag("all") && wanted.size) targets = targets.filter((s) => (loaded.origins.get(s.id) ?? []).some((f) => wanted.has(resolve(f))));
102+ else if (!flag("all") && !wanted.size) targets = targets.filter((s) => (loaded.origins.get(s.id) ?? []).some((f) => resolve(f) === resolve(baseFile)));
103+ if (onlySources) targets = targets.filter((s) => onlySources.includes(s.id));
104+
105+ // For fragment validation, only the sensors that the fragment contributed are relevant.
106+ const jobs: { src: SourceSeed; sen: SourceSeed["sensors"][number] }[] = [];
107+ for (const src of targets) for (const sen of src.sensors) if (!onlyConnector || sen.connector === onlyConnector) jobs.push({ src, sen });
108+ if (wanted.size && !flag("all")) {
109+ const fragmentUrls = new Set<string>();
110+ const YAML = (await import("yaml")).default;
111+ const { readFileSync } = await import("node:fs");
112+ for (const f of wanted) {
113+ const raw = YAML.parse(readFileSync(f, "utf8")) as { sources?: { sensors?: { url: string }[] }[] };
114+ for (const s of raw.sources ?? []) for (const sen of s.sensors ?? []) fragmentUrls.add(sen.url);
115+ }
116+ for (let i = jobs.length - 1; i >= 0; i--) if (!fragmentUrls.has(jobs[i]!.sen.url)) jobs.splice(i, 1);
117+ }
118+
119+ const concurrency = Number(arg("concurrency", "8"));
120+ const quiet = flag("quiet");
121+ const results: Result[] = [];
122+ // Per-host politeness: never more than 2 in flight for the same host.
123+ const inflight = new Map<string, number>();
124+ const host = (u: string): string => {
125+ try {
126+ return new URL(u).hostname;
127+ } catch {
128+ return u;
129+ }
130+ };
131+ const queue = [...jobs];
132+ await Promise.all(
133+ Array.from({ length: concurrency }, async () => {
134+ while (queue.length) {
135+ const idx = queue.findIndex((j) => (inflight.get(host(j.sen.url)) ?? 0) < 2);
136+ if (idx < 0) {
137+ await new Promise((r) => setTimeout(r, 150));
138+ continue;
139+ }
140+ const job = queue.splice(idx, 1)[0]!;
141+ const h = host(job.sen.url);
142+ inflight.set(h, (inflight.get(h) ?? 0) + 1);
143+ try {
144+ const r = await checkSensor(job.src, job.sen);
145+ results.push(r);
146+ if (!quiet || r.status !== "OK") console.log(`${r.status.padEnd(5)} ${String(r.http).padStart(3)} ${(r.mode ?? "-").padEnd(4)} ${(r.items !== undefined ? `items=${r.items}` : "-").padEnd(10)} ${r.sensor.padEnd(52).slice(0, 52)} ${r.url}${r.note ? ` (${r.note})` : ""}`);
147+ } finally {
148+ inflight.set(h, (inflight.get(h) ?? 1) - 1);
149+ }
150+ }
151+ }),
152+ );
153+
154+ const ok = results.filter((r) => r.status === "OK").length;
155+ const warn = results.filter((r) => r.status === "WARN").length;
156+ const fail = results.filter((r) => r.status === "FAIL").length;
157+ const byConnector = new Map<string, { ok: number; warn: number; fail: number }>();
158+ for (const r of results) {
159+ const c = byConnector.get(r.connector) ?? { ok: 0, warn: 0, fail: 0 };
160+ c[r.status.toLowerCase() as "ok" | "warn" | "fail"]++;
161+ byConnector.set(r.connector, c);
162+ }
163+ console.log(`\n${targets.length} sources · ${results.length} sensors → OK ${ok} · WARN ${warn} · FAIL ${fail}`);
164+ for (const [c, n] of [...byConnector.entries()].sort()) console.log(` ${c.padEnd(12)} ok ${String(n.ok).padStart(4)} warn ${String(n.warn).padStart(3)} fail ${String(n.fail).padStart(3)}`);
165+ const out = arg("json");
166+ if (out) writeFileSync(out, JSON.stringify({ generatedAt: new Date().toISOString(), sources: targets.length, results }, null, 2));
167+ if (fail > 0 || loaded.issues.length > 0) process.exitCode = 1;
168+}
169+
170+main()
171+ .catch((e) => {
172+ console.error(e);
173+ process.exitCode = 1;
174+ })
175+ .finally(() => closeDispatcher());
added config/sources.d/10-open-source.yaml +1404 −0
@@ -0,0 +1,1404 @@
1+# config/sources.d/10-open-source.yaml — open-source foundations, languages, frameworks, Linux/BSD
2+# distributions and developer tools (2026-09-08). Every sensor below was fetched and parsed by
3+# `apps/engine/src/validate.ts` before being written; GitHub release/tag Atom feeds are the backbone,
4+# with the project's own news/blog feed and status page when one exists. Sources whose release stream
5+# is a firehose (canaries, nightlies, per-package monorepo tags) carry `llm: false`.
6+sources:
7+ # ───────────────────────── Foundations ─────────────────────────
8+ - id: linux-foundation
9+ name: Linux Foundation
10+ domain: linuxfoundation.org
11+ homepage: https://www.linuxfoundation.org
12+ categories: [open-source, developer, ngo]
13+ tier: B
14+ weight: 1.2
15+ aliases: [lf, the linux foundation]
16+ discover: { rss: true, sitemap: true }
17+ sensors:
18+ - { name: blog feed, url: "https://www.linuxfoundation.org/blog/rss.xml", type: RSS, connector: rss, tier: B }
19+ - id: cncf
20+ name: Cloud Native Computing Foundation
21+ domain: cncf.io
22+ homepage: https://www.cncf.io
23+ categories: [open-source, developer, cloud]
24+ tier: B
25+ weight: 1.2
26+ aliases: [cncf, cloud native computing foundation]
27+ discover: { rss: true, sitemap: true }
28+ sensors:
29+ - { name: news feed, url: "https://www.cncf.io/feed/", type: RSS, connector: rss, tier: B }
30+ - id: apache
31+ name: Apache Software Foundation
32+ domain: apache.org
33+ homepage: https://www.apache.org
34+ categories: [open-source, developer]
35+ tier: B
36+ weight: 1.2
37+ aliases: [asf, apache software foundation, apache foundation]
38+ discover: { rss: true, sitemap: false }
39+ sensors:
40+ - { name: news feed, url: "https://news.apache.org/feed/", type: RSS, connector: rss, tier: B }
41+ - id: eclipse
42+ name: Eclipse Foundation
43+ domain: eclipse.org
44+ homepage: https://www.eclipse.org
45+ categories: [open-source, developer]
46+ tier: B
47+ aliases: [eclipse foundation]
48+ products:
49+ - { name: Eclipse IDE, type: software, aliases: [eclipse ide] }
50+ - { name: Jakarta EE, type: software, aliases: [jakarta ee] }
51+ discover: { rss: true }
52+ sensors:
53+ - { name: newsroom feed, url: "https://newsroom.eclipse.org/rss.xml", type: RSS, connector: rss, tier: B }
54+ - { name: blog feed, url: "https://blogs.eclipse.org/blog/feed", type: RSS, connector: rss, tier: C }
55+ - id: openjs
56+ name: OpenJS Foundation
57+ domain: openjsf.org
58+ categories: [open-source, developer]
59+ tier: C
60+ aliases: [openjs foundation, openjsf]
61+ discover: { rss: true, sitemap: true, pages: true }
62+ notes: "No feed; blog is a Next.js shell. Sitemap is the only server-side signal."
63+ sensors:
64+ - { name: sitemap, url: "https://openjsf.org/sitemap.xml", type: SITEMAP, connector: sitemap, tier: C, config: { maxUrls: 1000 } }
65+ - id: rust-foundation
66+ name: Rust Foundation
67+ domain: rustfoundation.org
68+ categories: [open-source, developer]
69+ tier: B
70+ aliases: [rust foundation]
71+ discover: { rss: true }
72+ sensors:
73+ - { name: news feed, url: "https://rustfoundation.org/feed/", type: RSS, connector: rss, tier: B }
74+ - id: openssf
75+ name: Open Source Security Foundation
76+ domain: openssf.org
77+ categories: [open-source, developer, cyber]
78+ tier: B
79+ aliases: [openssf, open source security foundation]
80+ products:
81+ - { name: Sigstore, type: software }
82+ - { name: OpenSSF Scorecard, type: software, aliases: [scorecard] }
83+ discover: { rss: true }
84+ sensors:
85+ - { name: blog feed, url: "https://openssf.org/feed/", type: RSS, connector: rss, tier: B }
86+ - id: python
87+ extend: true
88+ aliases: [python software foundation, psf]
89+ sensors:
90+ - { name: psf blog feed, url: "https://pyfound.blogspot.com/feeds/posts/default?max-results=25", type: ATOM, connector: rss, tier: B }
91+
92+ # ───────────────────────── BSD ─────────────────────────
93+ - id: freebsd
94+ name: FreeBSD
95+ domain: freebsd.org
96+ homepage: https://www.freebsd.org
97+ categories: [open-source, developer, infrastructure]
98+ tier: B
99+ weight: 1.1
100+ aliases: [freebsd project, freebsd foundation]
101+ discover: { rss: true }
102+ sensors:
103+ - { name: news feed, url: "https://www.freebsd.org/news/feed.xml", type: RSS, connector: rss, tier: B }
104+ - { name: security advisories feed, url: "https://www.freebsd.org/security/feed.xml", type: RSS, connector: rss, tier: A }
105+ - id: openbsd
106+ name: OpenBSD
107+ domain: openbsd.org
108+ homepage: https://www.openbsd.org
109+ categories: [open-source, developer, infrastructure, cyber]
110+ tier: B
111+ weight: 1.1
112+ aliases: [openbsd project]
113+ discover: { rss: false, pages: true }
114+ notes: "No official feed; the per-release errata page (7.9 as of 2026-09) is server-rendered — bump the number at each release. Undeadly is the community OpenBSD Journal."
115+ sensors:
116+ - { name: errata 7.9, url: "https://www.openbsd.org/errata79.html", type: HTML, connector: http, tier: B }
117+ - { name: openbsd journal feed, url: "https://undeadly.org/cgi?action=rss", type: RSS, connector: rss, tier: C }
118+ - id: netbsd
119+ name: NetBSD
120+ domain: netbsd.org
121+ homepage: https://www.netbsd.org
122+ categories: [open-source, developer, infrastructure]
123+ tier: C
124+ aliases: [netbsd project, netbsd foundation]
125+ discover: { rss: true }
126+ sensors:
127+ - { name: news feed, url: "https://www.netbsd.org/changes/rss-netbsd.xml", type: RSS, connector: rss, tier: C }
128+
129+ # ───────────────────────── Linux distributions ─────────────────────────
130+ - id: linux-kernel
131+ name: Linux Kernel
132+ domain: kernel.org
133+ homepage: https://www.kernel.org
134+ categories: [open-source, developer, infrastructure]
135+ tier: A
136+ weight: 1.4
137+ aliases: [linux, the linux kernel, kernel.org, linux kernel organization]
138+ discover: { rss: true }
139+ sensors:
140+ - { name: releases api, url: "https://www.kernel.org/releases.json", type: REST_API, connector: jsonlist, tier: A, config: { itemsPath: releases, keyField: version, titleTemplate: "Linux {version} ({moniker})", dateField: released.isodate, urlField: source, compareFields: [version, released.isodate, iseol] } }
141+ - { name: releases feed, url: "https://www.kernel.org/feeds/kdist.xml", type: RSS, connector: rss, tier: A }
142+ - id: debian
143+ name: Debian
144+ domain: debian.org
145+ homepage: https://www.debian.org
146+ categories: [open-source, developer, infrastructure]
147+ tier: A
148+ weight: 1.3
149+ aliases: [debian project, debian gnu/linux]
150+ discover: { rss: true }
151+ sensors:
152+ - { name: news feed, url: "https://www.debian.org/News/news", type: RSS, connector: rss, tier: A }
153+ - { name: security advisories feed, url: "https://www.debian.org/security/dsa-long", type: RSS, connector: rss, tier: S }
154+ - id: ubuntu
155+ name: Ubuntu / Canonical
156+ domain: ubuntu.com
157+ categories: [open-source, developer, infrastructure, cloud]
158+ tier: A
159+ weight: 1.3
160+ aliases: [canonical, ubuntu linux]
161+ products:
162+ - { name: Ubuntu, type: software }
163+ - { name: Snapcraft, type: software, aliases: [snap] }
164+ - { name: LXD, type: software }
165+ discover: { rss: true, status: true }
166+ sensors:
167+ - { name: blog feed, url: "https://ubuntu.com/blog/feed", type: RSS, connector: rss, tier: B }
168+ - { name: security notices feed, url: "https://ubuntu.com/security/notices/rss.xml", type: RSS, connector: rss, tier: S, config: { maxItems: 60 } }
169+ - { name: canonical blog feed, url: "https://canonical.com/blog/feed", type: RSS, connector: rss, tier: B }
170+ - id: fedora
171+ name: Fedora Project
172+ domain: fedoraproject.org
173+ categories: [open-source, developer]
174+ tier: B
175+ aliases: [fedora, fedora linux]
176+ discover: { rss: true }
177+ sensors:
178+ - { name: fedora magazine feed, url: "https://fedoramagazine.org/feed/", type: RSS, connector: rss, tier: B }
179+ - id: red-hat
180+ name: Red Hat
181+ domain: redhat.com
182+ homepage: https://www.redhat.com
183+ categories: [open-source, enterprise, cloud, infrastructure]
184+ tier: A
185+ weight: 1.3
186+ aliases: [redhat, rhel]
187+ products:
188+ - { name: Red Hat Enterprise Linux, type: software, aliases: [rhel] }
189+ - { name: OpenShift, type: software }
190+ - { name: Ansible Automation Platform, type: software }
191+ discover: { rss: true, status: true }
192+ sensors:
193+ - { name: status, url: "https://status.redhat.com/api/v2/summary.json", type: STATUSPAGE, connector: statuspage, tier: S }
194+ - { name: blog feed, url: "https://www.redhat.com/en/rss/blog", type: RSS, connector: rss, tier: B }
195+ - id: arch-linux
196+ name: Arch Linux
197+ domain: archlinux.org
198+ categories: [open-source, developer]
199+ tier: B
200+ aliases: [arch, archlinux]
201+ discover: { rss: true }
202+ sensors:
203+ - { name: news feed, url: "https://archlinux.org/feeds/news/", type: RSS, connector: rss, tier: B }
204+ - { name: security advisories feed, url: "https://security.archlinux.org/advisory/feed.atom", type: ATOM, connector: rss, tier: A }
205+ - id: alpine
206+ name: Alpine Linux
207+ domain: alpinelinux.org
208+ categories: [open-source, developer, infrastructure]
209+ tier: B
210+ aliases: [alpine, alpine linux]
211+ discover: { rss: true }
212+ sensors:
213+ - { name: news feed, url: "https://alpinelinux.org/atom.xml", type: ATOM, connector: rss, tier: B }
214+ - id: nixos
215+ name: NixOS
216+ domain: nixos.org
217+ categories: [open-source, developer, packages]
218+ tier: B
219+ aliases: [nix, nixpkgs, nixos foundation]
220+ products:
221+ - { name: Nix, type: software }
222+ - { name: Nixpkgs, type: software }
223+ discover: { rss: true }
224+ sensors:
225+ - { name: announcements feed, url: "https://nixos.org/blog/announcements-rss.xml", type: RSS, connector: rss, tier: B }
226+ - { name: discourse announcements feed, url: "https://discourse.nixos.org/c/announcements/8.rss", type: RSS, connector: rss, tier: B }
227+ - { name: nix releases, url: "https://github.com/NixOS/nix/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: NixOS/nix, kind: releases } }
228+ - id: opensuse
229+ name: openSUSE
230+ domain: opensuse.org
231+ homepage: https://www.opensuse.org
232+ categories: [open-source, developer]
233+ tier: B
234+ aliases: [opensuse project, tumbleweed, leap]
235+ discover: { rss: true }
236+ sensors:
237+ - { name: news feed, url: "https://news.opensuse.org/feed.xml", type: RSS, connector: rss, tier: B }
238+ - id: suse
239+ name: SUSE
240+ domain: suse.com
241+ homepage: https://www.suse.com
242+ categories: [open-source, enterprise, infrastructure]
243+ tier: B
244+ aliases: [suse linux, sles]
245+ products:
246+ - { name: SUSE Linux Enterprise Server, type: software, aliases: [sles] }
247+ - { name: Rancher, type: software }
248+ discover: { rss: true }
249+ sensors:
250+ - { name: blog feed, url: "https://www.suse.com/c/feed/", type: RSS, connector: rss, tier: B }
251+
252+ # ───────────────────────── Desktop environments ─────────────────────────
253+ - id: gnome
254+ name: GNOME
255+ domain: gnome.org
256+ homepage: https://www.gnome.org
257+ categories: [open-source, developer]
258+ tier: C
259+ aliases: [gnome foundation, gnome project]
260+ discover: { rss: true, sitemap: true }
261+ notes: "Foundation site has no feed; Planet GNOME aggregates developer blogs."
262+ sensors:
263+ - { name: planet gnome feed, url: "https://planet.gnome.org/rss20.xml", type: RSS, connector: rss, tier: C, config: { maxItems: 60 } }
264+ - id: kde
265+ name: KDE
266+ domain: kde.org
267+ categories: [open-source, developer]
268+ tier: B
269+ aliases: [kde community, kde e.v., plasma]
270+ products:
271+ - { name: KDE Plasma, type: software, aliases: [plasma] }
272+ discover: { rss: true }
273+ sensors:
274+ - { name: announcements feed, url: "https://kde.org/announcements/index.xml", type: RSS, connector: rss, tier: B }
275+ - { name: news feed, url: "https://kde.org/index.xml", type: RSS, connector: rss, tier: B }
276+
277+ # ───────────────────────── Core toolchain & libraries ─────────────────────────
278+ - id: git
279+ name: Git
280+ domain: git-scm.com
281+ categories: [open-source, developer]
282+ tier: B
283+ weight: 1.2
284+ aliases: [git scm]
285+ discover: { rss: true }
286+ sensors:
287+ - { name: tags, url: "https://github.com/git/git/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: git/git, kind: tags } }
288+ - id: curl
289+ name: curl
290+ domain: curl.se
291+ categories: [open-source, developer, infrastructure, cyber]
292+ tier: B
293+ weight: 1.2
294+ aliases: [libcurl, curl project]
295+ discover: { rss: true }
296+ sensors:
297+ - { name: releases, url: "https://github.com/curl/curl/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: curl/curl, kind: releases } }
298+ - { name: vulnerabilities api, url: "https://curl.se/docs/vuln.json", type: REST_API, connector: jsonlist, tier: A, config: { keyField: id, titleTemplate: "{id}: {summary}", urlField: database_specific.www, dateField: published, summaryField: details, compareFields: [summary, database_specific.severity], maxItems: 100 } }
299+ - { name: daniel stenberg blog feed, url: "https://daniel.haxx.se/blog/feed/", type: RSS, connector: rss, tier: C }
300+ - id: openssl
301+ name: OpenSSL
302+ domain: openssl-library.org
303+ categories: [open-source, developer, infrastructure, cyber]
304+ tier: A
305+ weight: 1.3
306+ aliases: [openssl project, openssl library]
307+ discover: { rss: true }
308+ sensors:
309+ - { name: releases, url: "https://github.com/openssl/openssl/releases.atom", type: GITHUB_RELEASE, connector: github, tier: A, config: { repo: openssl/openssl, kind: releases } }
310+ - { name: news feed, url: "https://openssl-library.org/news/index.xml", type: RSS, connector: rss, tier: A, config: { maxItems: 50 } }
311+ - id: openssh
312+ name: OpenSSH
313+ domain: openssh.com
314+ homepage: https://www.openssh.com
315+ categories: [open-source, developer, infrastructure, cyber]
316+ tier: A
317+ weight: 1.2
318+ aliases: [openssh portable]
319+ discover: { rss: false, pages: true }
320+ sensors:
321+ - { name: portable tags, url: "https://github.com/openssh/openssh-portable/tags.atom", type: GITHUB_RELEASE, connector: github, tier: A, config: { repo: openssh/openssh-portable, kind: tags } }
322+ - { name: release notes, url: "https://www.openssh.org/releasenotes.html", type: HTML, connector: http, tier: B }
323+ - id: llvm
324+ name: LLVM Project
325+ domain: llvm.org
326+ categories: [open-source, developer]
327+ tier: B
328+ weight: 1.1
329+ aliases: [llvm, clang]
330+ products:
331+ - { name: Clang, type: software }
332+ - { name: LLDB, type: software }
333+ discover: { rss: true }
334+ sensors:
335+ - { name: releases, url: "https://github.com/llvm/llvm-project/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: llvm/llvm-project, kind: releases } }
336+ - { name: discourse announcements feed, url: "https://discourse.llvm.org/c/announce/46.rss", type: RSS, connector: rss, tier: B }
337+ - id: gcc
338+ name: GCC
339+ domain: gcc.gnu.org
340+ categories: [open-source, developer]
341+ tier: C
342+ aliases: [gnu compiler collection, gnu gcc]
343+ discover: { rss: false, pages: true }
344+ notes: "No feed; the gcc-mirror tags feed is dominated by basepoints. Releases page is server-rendered."
345+ sensors:
346+ - { name: releases page, url: "https://gcc.gnu.org/releases.html", type: HTML, connector: http, tier: C }
347+
348+ # ───────────────────────── Web frameworks ─────────────────────────
349+ - id: react
350+ name: React
351+ domain: react.dev
352+ categories: [open-source, developer]
353+ tier: A
354+ weight: 1.3
355+ aliases: [reactjs, react.js]
356+ discover: { rss: true }
357+ sensors:
358+ - { name: blog feed, url: "https://react.dev/rss.xml", type: RSS, connector: rss, tier: A }
359+ - { name: releases, url: "https://github.com/react/react/releases.atom", type: GITHUB_RELEASE, connector: github, tier: A, config: { repo: react/react, kind: releases } }
360+ - id: nextjs
361+ name: Next.js
362+ domain: nextjs.org
363+ categories: [open-source, developer]
364+ tier: A
365+ weight: 1.2
366+ aliases: [next.js, next js, nextjs]
367+ llm: false
368+ discover: { rss: true }
369+ notes: "Releases feed is dominated by daily canaries — heuristics only."
370+ sensors:
371+ - { name: blog feed, url: "https://nextjs.org/feed.xml", type: RSS, connector: rss, tier: A }
372+ - { name: releases, url: "https://github.com/vercel/next.js/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: vercel/next.js, kind: releases } }
373+ - id: vue
374+ name: Vue.js
375+ domain: vuejs.org
376+ categories: [open-source, developer]
377+ tier: B
378+ weight: 1.1
379+ aliases: [vue, vuejs]
380+ discover: { rss: true }
381+ sensors:
382+ - { name: blog feed, url: "https://blog.vuejs.org/feed.rss", type: RSS, connector: rss, tier: B }
383+ - { name: releases, url: "https://github.com/vuejs/core/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: vuejs/core, kind: releases } }
384+ - id: angular
385+ name: Angular
386+ domain: angular.dev
387+ categories: [open-source, developer]
388+ tier: B
389+ weight: 1.1
390+ aliases: [angularjs]
391+ discover: { rss: true }
392+ sensors:
393+ - { name: blog feed, url: "https://blog.angular.dev/feed", type: RSS, connector: rss, tier: B }
394+ - { name: releases, url: "https://github.com/angular/angular/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: angular/angular, kind: releases } }
395+ - id: svelte
396+ name: Svelte
397+ domain: svelte.dev
398+ categories: [open-source, developer]
399+ tier: B
400+ aliases: [sveltekit, svelte kit]
401+ llm: false
402+ discover: { rss: true }
403+ notes: "Monorepo releases feed mixes every package — heuristics only."
404+ sensors:
405+ - { name: blog feed, url: "https://svelte.dev/blog/rss.xml", type: RSS, connector: rss, tier: B, config: { maxItems: 40 } }
406+ - { name: releases, url: "https://github.com/sveltejs/svelte/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: sveltejs/svelte, kind: releases } }
407+ - id: astro
408+ name: Astro
409+ domain: astro.build
410+ categories: [open-source, developer]
411+ tier: B
412+ aliases: [astro build, withastro]
413+ llm: false
414+ discover: { rss: true }
415+ sensors:
416+ - { name: blog feed, url: "https://astro.build/rss.xml", type: RSS, connector: rss, tier: B }
417+ - { name: releases, url: "https://github.com/withastro/astro/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: withastro/astro, kind: releases } }
418+ - id: django
419+ name: Django
420+ domain: djangoproject.com
421+ homepage: https://www.djangoproject.com
422+ categories: [open-source, developer]
423+ tier: B
424+ weight: 1.1
425+ aliases: [django software foundation, dsf]
426+ discover: { rss: true }
427+ sensors:
428+ - { name: weblog feed, url: "https://www.djangoproject.com/rss/weblog/", type: RSS, connector: rss, tier: A }
429+ - { name: tags, url: "https://github.com/django/django/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: django/django, kind: tags } }
430+ - id: flask
431+ name: Flask / Pallets
432+ domain: palletsprojects.com
433+ categories: [open-source, developer]
434+ tier: B
435+ aliases: [pallets, pallets projects, werkzeug, jinja]
436+ products:
437+ - { name: Werkzeug, type: software }
438+ - { name: Jinja, type: software }
439+ discover: { rss: true }
440+ sensors:
441+ - { name: blog feed, url: "https://palletsprojects.com/blog/feed.xml", type: ATOM, connector: rss, tier: B }
442+ - { name: releases, url: "https://github.com/pallets/flask/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: pallets/flask, kind: releases } }
443+ - id: fastapi
444+ name: FastAPI
445+ domain: fastapi.tiangolo.com
446+ categories: [open-source, developer]
447+ tier: B
448+ aliases: [fast api]
449+ discover: { rss: false }
450+ sensors:
451+ - { name: releases, url: "https://github.com/fastapi/fastapi/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: fastapi/fastapi, kind: releases } }
452+ - id: rails
453+ name: Ruby on Rails
454+ domain: rubyonrails.org
455+ categories: [open-source, developer]
456+ tier: B
457+ weight: 1.1
458+ aliases: [rails, ror]
459+ discover: { rss: true }
460+ sensors:
461+ - { name: blog feed, url: "https://rubyonrails.org/feed.xml", type: ATOM, connector: rss, tier: B }
462+ - { name: releases, url: "https://github.com/rails/rails/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: rails/rails, kind: releases } }
463+ - id: laravel
464+ name: Laravel
465+ domain: laravel.com
466+ categories: [open-source, developer]
467+ tier: B
468+ aliases: [laravel framework]
469+ discover: { rss: true }
470+ sensors:
471+ - { name: blog feed, url: "https://laravel.com/feed", type: RSS, connector: rss, tier: B }
472+ - { name: framework releases, url: "https://github.com/laravel/framework/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: laravel/framework, kind: releases } }
473+ - id: spring
474+ name: Spring
475+ domain: spring.io
476+ categories: [open-source, developer, enterprise]
477+ tier: B
478+ weight: 1.1
479+ aliases: [spring framework, spring boot, pivotal spring]
480+ products:
481+ - { name: Spring Boot, type: software }
482+ - { name: Spring Framework, type: software }
483+ discover: { rss: true }
484+ sensors:
485+ - { name: blog feed, url: "https://spring.io/blog.atom", type: RSS, connector: rss, tier: B }
486+ - { name: spring boot releases, url: "https://github.com/spring-projects/spring-boot/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: spring-projects/spring-boot, kind: releases } }
487+
488+ # ───────────────────────── Languages & runtimes ─────────────────────────
489+ - id: dotnet
490+ name: .NET
491+ domain: dotnet.microsoft.com
492+ categories: [open-source, developer, enterprise]
493+ tier: A
494+ weight: 1.2
495+ aliases: [dotnet, .net core, asp.net, c#]
496+ products:
497+ - { name: ASP.NET Core, type: software, aliases: [asp.net] }
498+ - { name: C#, type: software, aliases: [csharp] }
499+ discover: { rss: true }
500+ sensors:
501+ - { name: releases index api, url: "https://builds.dotnet.microsoft.com/dotnet/release-metadata/releases-index.json", type: REST_API, connector: jsonlist, tier: A, config: { itemsPath: releases-index, keyField: channel-version, titleTemplate: ".NET {channel-version} → {latest-release} ({support-phase})", dateField: latest-release-date, urlField: releases.json, compareFields: [latest-release, latest-release-date, support-phase, security] } }
502+ - { name: dotnet blog feed, url: "https://devblogs.microsoft.com/dotnet/feed/", type: RSS, connector: rss, tier: B }
503+ - { name: runtime releases, url: "https://github.com/dotnet/runtime/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: dotnet/runtime, kind: releases } }
504+ - id: typescript
505+ name: TypeScript
506+ domain: typescriptlang.org
507+ homepage: https://www.typescriptlang.org
508+ categories: [open-source, developer]
509+ tier: A
510+ weight: 1.2
511+ aliases: [ts, typescript language]
512+ discover: { rss: true }
513+ sensors:
514+ - { name: blog feed, url: "https://devblogs.microsoft.com/typescript/feed/", type: RSS, connector: rss, tier: A }
515+ - { name: releases, url: "https://github.com/microsoft/TypeScript/releases.atom", type: GITHUB_RELEASE, connector: github, tier: A, config: { repo: microsoft/TypeScript, kind: releases } }
516+ - id: kotlin
517+ name: Kotlin
518+ domain: kotlinlang.org
519+ categories: [open-source, developer]
520+ tier: B
521+ weight: 1.1
522+ aliases: [kotlin language, kotlin foundation]
523+ discover: { rss: true }
524+ notes: "The GitHub releases feed is dominated by dev builds; the Kotlin blog carries the real releases."
525+ sensors:
526+ - { name: blog feed, url: "https://blog.jetbrains.com/kotlin/feed/", type: RSS, connector: rss, tier: B }
527+ - id: jetbrains
528+ name: JetBrains
529+ domain: jetbrains.com
530+ homepage: https://www.jetbrains.com
531+ categories: [developer, technology]
532+ tier: B
533+ weight: 1.1
534+ aliases: [intellij, jetbrains ides]
535+ products:
536+ - { name: IntelliJ IDEA, type: software, aliases: [intellij] }
537+ - { name: PyCharm, type: software }
538+ - { name: WebStorm, type: software }
539+ - { name: JetBrains AI Assistant, type: product, aliases: [junie] }
540+ discover: { rss: true, status: true }
541+ sensors:
542+ - { name: blog feed, url: "https://blog.jetbrains.com/feed/", type: RSS, connector: rss, tier: B }
543+ - id: swift
544+ name: Swift.org
545+ domain: swift.org
546+ homepage: https://www.swift.org
547+ categories: [open-source, developer]
548+ tier: B
549+ weight: 1.1
550+ aliases: [swift language, swift programming language]
551+ discover: { rss: true }
552+ notes: "GitHub releases feed is flooded by nightly development snapshots; the swift.org feed announces releases."
553+ sensors:
554+ - { name: blog feed, url: "https://www.swift.org/atom.xml", type: ATOM, connector: rss, tier: B }
555+ - id: deno
556+ name: Deno
557+ domain: deno.com
558+ categories: [open-source, developer, cloud]
559+ tier: B
560+ aliases: [deno land, deno deploy]
561+ products:
562+ - { name: Deno Deploy, type: service }
563+ discover: { rss: true, status: true }
564+ sensors:
565+ - { name: blog feed, url: "https://deno.com/feed", type: ATOM, connector: rss, tier: B }
566+ - { name: releases, url: "https://github.com/denoland/deno/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: denoland/deno, kind: releases } }
567+ - id: bun
568+ name: Bun
569+ domain: bun.sh
570+ categories: [open-source, developer]
571+ tier: B
572+ aliases: [bun runtime, oven]
573+ discover: { rss: true }
574+ sensors:
575+ - { name: blog feed, url: "https://bun.sh/rss.xml", type: RSS, connector: rss, tier: B }
576+ - { name: releases, url: "https://github.com/oven-sh/bun/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: oven-sh/bun, kind: releases } }
577+ - id: zig
578+ name: Zig
579+ domain: ziglang.org
580+ categories: [open-source, developer]
581+ tier: C
582+ aliases: [zig language, zig software foundation]
583+ discover: { rss: true }
584+ sensors:
585+ - { name: news feed, url: "https://ziglang.org/news/index.xml", type: RSS, connector: rss, tier: C }
586+ - { name: tags, url: "https://github.com/ziglang/zig/tags.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: ziglang/zig, kind: tags } }
587+ - id: julia
588+ name: Julia
589+ domain: julialang.org
590+ categories: [open-source, developer, science]
591+ tier: C
592+ aliases: [julia language, julialang]
593+ discover: { rss: true }
594+ sensors:
595+ - { name: blog feed, url: "https://julialang.org/feed.xml", type: RSS, connector: rss, tier: C }
596+ - { name: releases, url: "https://github.com/JuliaLang/julia/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: JuliaLang/julia, kind: releases } }
597+ - id: r-project
598+ name: R Project
599+ domain: r-project.org
600+ homepage: https://www.r-project.org
601+ categories: [open-source, developer, statistics, science]
602+ tier: C
603+ aliases: [r, r language, r foundation, cran]
604+ discover: { rss: true }
605+ sensors:
606+ - { name: blog feed, url: "https://blog.r-project.org/index.xml", type: RSS, connector: rss, tier: C }
607+
608+ # ───────────────────────── AI / ML / data science ─────────────────────────
609+ - id: jupyter
610+ name: Project Jupyter
611+ domain: jupyter.org
612+ categories: [open-source, developer, ai, science]
613+ tier: B
614+ aliases: [jupyter, jupyterlab, jupyter notebook]
615+ products:
616+ - { name: JupyterLab, type: software }
617+ discover: { rss: true }
618+ sensors:
619+ - { name: blog feed, url: "https://blog.jupyter.org/feed", type: RSS, connector: rss, tier: B }
620+ - { name: jupyterlab tags, url: "https://github.com/jupyterlab/jupyterlab/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: jupyterlab/jupyterlab, kind: tags } }
621+ - { name: notebook releases, url: "https://github.com/jupyter/notebook/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: jupyter/notebook, kind: releases } }
622+ - id: pytorch
623+ name: PyTorch
624+ domain: pytorch.org
625+ categories: [open-source, developer, ai]
626+ tier: A
627+ weight: 1.3
628+ aliases: [pytorch foundation, torch]
629+ discover: { rss: true }
630+ notes: "GitHub releases feed is polluted by viable/strict CI tags; the blog announces releases."
631+ sensors:
632+ - { name: blog feed, url: "https://pytorch.org/blog/feed/", type: RSS, connector: rss, tier: A }
633+ - id: tensorflow
634+ name: TensorFlow
635+ domain: tensorflow.org
636+ homepage: https://www.tensorflow.org
637+ categories: [open-source, developer, ai]
638+ tier: B
639+ weight: 1.1
640+ aliases: [tf, keras]
641+ discover: { rss: true }
642+ sensors:
643+ - { name: blog feed, url: "https://blog.tensorflow.org/feeds/posts/default?max-results=25", type: ATOM, connector: rss, tier: B }
644+ - { name: releases, url: "https://github.com/tensorflow/tensorflow/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: tensorflow/tensorflow, kind: releases } }
645+ - id: jax
646+ name: JAX
647+ domain: jax.dev
648+ homepage: https://docs.jax.dev
649+ categories: [open-source, developer, ai]
650+ tier: B
651+ aliases: [jax ml, google jax]
652+ discover: { rss: false }
653+ sensors:
654+ - { name: releases, url: "https://github.com/jax-ml/jax/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: jax-ml/jax, kind: releases } }
655+ - { name: changelog, url: "https://docs.jax.dev/en/latest/changelog.html", type: HTML, connector: http, tier: B }
656+ - id: langchain
657+ name: LangChain
658+ domain: langchain.com
659+ homepage: https://www.langchain.com
660+ categories: [open-source, developer, ai]
661+ tier: B
662+ weight: 1.1
663+ aliases: [langgraph, langsmith]
664+ products:
665+ - { name: LangGraph, type: software }
666+ - { name: LangSmith, type: service }
667+ llm: false
668+ discover: { rss: true, status: true }
669+ notes: "Monorepo releases feed publishes every package tag — heuristics only."
670+ sensors:
671+ - { name: blog feed, url: "https://www.langchain.com/blog/rss.xml", type: RSS, connector: rss, tier: B, config: { maxItems: 40 } }
672+ - { name: releases, url: "https://github.com/langchain-ai/langchain/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: langchain-ai/langchain, kind: releases } }
673+ - { name: langsmith pricing, url: "https://www.langchain.com/pricing", type: HTML, connector: http, tier: C }
674+ - id: vllm
675+ name: vLLM
676+ domain: vllm.ai
677+ categories: [open-source, developer, ai]
678+ tier: B
679+ aliases: [vllm project]
680+ discover: { rss: true }
681+ notes: "Blog has no feed (Next.js)."
682+ sensors:
683+ - { name: releases, url: "https://github.com/vllm-project/vllm/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: vllm-project/vllm, kind: releases } }
684+ - id: ollama
685+ name: Ollama
686+ domain: ollama.com
687+ categories: [open-source, developer, ai]
688+ tier: B
689+ weight: 1.1
690+ discover: { rss: true, pages: true }
691+ notes: "Blog has no feed; the blog index is server-rendered."
692+ sensors:
693+ - { name: releases, url: "https://github.com/ollama/ollama/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: ollama/ollama, kind: releases } }
694+ - { name: blog index, url: "https://ollama.com/blog", type: HTML, connector: http, tier: B }
695+
696+ # ───────────────────────── Infrastructure, DevOps, cloud native ─────────────────────────
697+ - id: ansible
698+ name: Ansible
699+ domain: ansible.com
700+ homepage: https://www.ansible.com
701+ categories: [open-source, developer, infrastructure]
702+ tier: B
703+ weight: 1.1
704+ aliases: [ansible core, ansible community]
705+ discover: { rss: true }
706+ sensors:
707+ - { name: community news feed, url: "https://forum.ansible.com/c/news/5.rss", type: RSS, connector: rss, tier: B }
708+ - { name: releases, url: "https://github.com/ansible/ansible/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: ansible/ansible, kind: releases } }
709+ - id: opentofu
710+ name: OpenTofu
711+ domain: opentofu.org
712+ categories: [open-source, developer, infrastructure, cloud]
713+ tier: B
714+ aliases: [open tofu]
715+ discover: { rss: true }
716+ sensors:
717+ - { name: blog feed, url: "https://opentofu.org/blog/rss.xml", type: RSS, connector: rss, tier: B }
718+ - { name: releases, url: "https://github.com/opentofu/opentofu/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: opentofu/opentofu, kind: releases } }
719+ - id: pulumi
720+ name: Pulumi
721+ domain: pulumi.com
722+ homepage: https://www.pulumi.com
723+ categories: [open-source, developer, infrastructure, cloud]
724+ tier: B
725+ aliases: [pulumi cloud]
726+ llm: false
727+ discover: { rss: true, status: true }
728+ notes: "Releases feed mixes pkg/ and sdk/ tags several times a week — heuristics only."
729+ sensors:
730+ - { name: status, url: "https://status.pulumi.com/api/v2/summary.json", type: STATUSPAGE, connector: statuspage, tier: S }
731+ - { name: blog feed, url: "https://www.pulumi.com/blog/rss.xml", type: RSS, connector: rss, tier: B, config: { maxItems: 40 } }
732+ - { name: releases, url: "https://github.com/pulumi/pulumi/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: pulumi/pulumi, kind: releases } }
733+ - id: prometheus
734+ name: Prometheus
735+ domain: prometheus.io
736+ categories: [open-source, developer, infrastructure, cloud]
737+ tier: B
738+ weight: 1.1
739+ aliases: [prometheus monitoring]
740+ discover: { rss: true }
741+ sensors:
742+ - { name: blog feed, url: "https://prometheus.io/blog/feed.xml", type: ATOM, connector: rss, tier: B }
743+ - { name: releases, url: "https://github.com/prometheus/prometheus/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: prometheus/prometheus, kind: releases } }
744+ - id: istio
745+ name: Istio
746+ domain: istio.io
747+ categories: [open-source, developer, infrastructure, cloud]
748+ tier: B
749+ aliases: [istio service mesh]
750+ discover: { rss: true }
751+ sensors:
752+ - { name: news feed, url: "https://istio.io/latest/news/feed.xml", type: RSS, connector: rss, tier: B, config: { maxItems: 40 } }
753+ - { name: releases, url: "https://github.com/istio/istio/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: istio/istio, kind: releases } }
754+ - id: envoy
755+ name: Envoy Proxy
756+ domain: envoyproxy.io
757+ homepage: https://www.envoyproxy.io
758+ categories: [open-source, developer, infrastructure, cloud]
759+ tier: B
760+ aliases: [envoy, envoyproxy]
761+ discover: { rss: true }
762+ notes: "Medium blog (blog.envoyproxy.io) does not resolve from the engine."
763+ sensors:
764+ - { name: releases, url: "https://github.com/envoyproxy/envoy/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: envoyproxy/envoy, kind: releases } }
765+ - id: helm
766+ name: Helm
767+ domain: helm.sh
768+ categories: [open-source, developer, infrastructure, cloud, packages]
769+ tier: B
770+ aliases: [helm charts]
771+ discover: { rss: true }
772+ sensors:
773+ - { name: blog feed, url: "https://helm.sh/blog/index.xml", type: RSS, connector: rss, tier: B }
774+ - { name: releases, url: "https://github.com/helm/helm/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: helm/helm, kind: releases } }
775+ - id: argo
776+ name: Argo Project
777+ domain: argoproj.github.io
778+ categories: [open-source, developer, infrastructure, cloud]
779+ tier: B
780+ aliases: [argo, argo cd, argocd, argo workflows]
781+ products:
782+ - { name: Argo CD, type: software, aliases: [argocd] }
783+ - { name: Argo Workflows, type: software }
784+ discover: { rss: true }
785+ sensors:
786+ - { name: blog feed, url: "https://blog.argoproj.io/feed", type: RSS, connector: rss, tier: B }
787+ - { name: argo cd releases, url: "https://github.com/argoproj/argo-cd/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: argoproj/argo-cd, kind: releases } }
788+ - id: nginx
789+ name: NGINX
790+ domain: nginx.org
791+ categories: [open-source, developer, infrastructure]
792+ tier: A
793+ weight: 1.2
794+ aliases: [nginx open source, f5 nginx]
795+ discover: { rss: true }
796+ sensors:
797+ - { name: releases, url: "https://github.com/nginx/nginx/releases.atom", type: GITHUB_RELEASE, connector: github, tier: A, config: { repo: nginx/nginx, kind: releases } }
798+ - { name: blog feed, url: "https://blog.nginx.org/feed", type: RSS, connector: rss, tier: B }
799+ - id: apache-httpd
800+ name: Apache HTTP Server
801+ domain: httpd.apache.org
802+ categories: [open-source, developer, infrastructure, cyber]
803+ tier: A
804+ weight: 1.1
805+ aliases: [apache httpd, httpd, apache web server]
806+ discover: { rss: false, pages: true }
807+ sensors:
808+ - { name: tags, url: "https://github.com/apache/httpd/tags.atom", type: GITHUB_RELEASE, connector: github, tier: A, config: { repo: apache/httpd, kind: tags } }
809+ - { name: 2.4 vulnerabilities, url: "https://httpd.apache.org/security/vulnerabilities_24.html", type: HTML, connector: http, tier: A }
810+ - id: caddy
811+ name: Caddy
812+ domain: caddyserver.com
813+ categories: [open-source, developer, infrastructure]
814+ tier: B
815+ aliases: [caddy server, caddy web server]
816+ discover: { rss: true }
817+ sensors:
818+ - { name: releases, url: "https://github.com/caddyserver/caddy/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: caddyserver/caddy, kind: releases } }
819+ - id: haproxy
820+ name: HAProxy
821+ domain: haproxy.org
822+ homepage: https://www.haproxy.org
823+ categories: [open-source, developer, infrastructure]
824+ tier: B
825+ aliases: [haproxy technologies]
826+ discover: { rss: true }
827+ sensors:
828+ - { name: tags, url: "https://github.com/haproxy/haproxy/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: haproxy/haproxy, kind: tags } }
829+ - { name: haproxy technologies blog feed, url: "https://www.haproxy.com/feed", type: RSS, connector: rss, tier: C }
830+
831+ # ───────────────────────── Databases & data platforms ─────────────────────────
832+ - id: sqlite
833+ name: SQLite
834+ domain: sqlite.org
835+ homepage: https://www.sqlite.org
836+ categories: [open-source, developer, infrastructure]
837+ tier: B
838+ weight: 1.2
839+ aliases: [sqlite3]
840+ discover: { rss: false, pages: true }
841+ notes: "No feed; GitHub mirror tags are dominated by bug-* branches. Server-rendered news/changes pages instead."
842+ sensors:
843+ - { name: news page, url: "https://www.sqlite.org/news.html", type: HTML, connector: http, tier: B }
844+ - { name: release history, url: "https://www.sqlite.org/changes.html", type: HTML, connector: http, tier: C }
845+ - id: mariadb
846+ name: MariaDB
847+ domain: mariadb.org
848+ categories: [open-source, developer, infrastructure]
849+ tier: B
850+ weight: 1.1
851+ aliases: [mariadb foundation, mariadb server]
852+ discover: { rss: true }
853+ sensors:
854+ - { name: foundation blog feed, url: "https://mariadb.org/feed/", type: RSS, connector: rss, tier: B }
855+ - { name: server tags, url: "https://github.com/MariaDB/server/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: MariaDB/server, kind: tags } }
856+ - id: clickhouse
857+ name: ClickHouse
858+ domain: clickhouse.com
859+ categories: [open-source, developer, infrastructure, cloud]
860+ tier: B
861+ weight: 1.1
862+ aliases: [clickhouse cloud]
863+ discover: { rss: true, status: true }
864+ sensors:
865+ - { name: status, url: "https://status.clickhouse.com/api/v2/summary.json", type: STATUSPAGE, connector: statuspage, tier: S }
866+ - { name: blog feed, url: "https://clickhouse.com/rss.xml", type: RSS, connector: rss, tier: B, config: { maxItems: 40 } }
867+ - { name: releases, url: "https://github.com/ClickHouse/ClickHouse/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: ClickHouse/ClickHouse, kind: releases } }
868+ - id: duckdb
869+ name: DuckDB
870+ domain: duckdb.org
871+ categories: [open-source, developer, infrastructure]
872+ tier: B
873+ weight: 1.1
874+ aliases: [duckdb labs, duckdb foundation, motherduck]
875+ discover: { rss: true }
876+ sensors:
877+ - { name: blog feed, url: "https://duckdb.org/feed.xml", type: ATOM, connector: rss, tier: B }
878+ - { name: releases, url: "https://github.com/duckdb/duckdb/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: duckdb/duckdb, kind: releases } }
879+ - id: neo4j
880+ name: Neo4j
881+ domain: neo4j.com
882+ categories: [open-source, developer, infrastructure, enterprise]
883+ tier: B
884+ aliases: [neo4j aura, cypher]
885+ discover: { rss: true, status: true }
886+ sensors:
887+ - { name: blog feed, url: "https://neo4j.com/blog/feed/", type: RSS, connector: rss, tier: B, config: { maxItems: 40 } }
888+ - { name: tags, url: "https://github.com/neo4j/neo4j/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: neo4j/neo4j, kind: tags } }
889+ - id: cassandra
890+ name: Apache Cassandra
891+ domain: cassandra.apache.org
892+ categories: [open-source, developer, infrastructure]
893+ tier: B
894+ aliases: [cassandra]
895+ discover: { rss: false, pages: true }
896+ sensors:
897+ - { name: tags, url: "https://github.com/apache/cassandra/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: apache/cassandra, kind: tags } }
898+ - { name: blog index, url: "https://cassandra.apache.org/_/blog.html", type: HTML, connector: http, tier: C }
899+ - id: kafka
900+ name: Apache Kafka
901+ domain: kafka.apache.org
902+ categories: [open-source, developer, infrastructure]
903+ tier: B
904+ weight: 1.1
905+ aliases: [kafka]
906+ discover: { rss: false, pages: true }
907+ sensors:
908+ - { name: releases, url: "https://github.com/apache/kafka/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: apache/kafka, kind: releases } }
909+ - { name: blog index, url: "https://kafka.apache.org/blog", type: HTML, connector: http, tier: C }
910+ - id: rabbitmq
911+ name: RabbitMQ
912+ domain: rabbitmq.com
913+ homepage: https://www.rabbitmq.com
914+ categories: [open-source, developer, infrastructure]
915+ tier: B
916+ aliases: [rabbit mq]
917+ discover: { rss: true }
918+ sensors:
919+ - { name: blog feed, url: "https://www.rabbitmq.com/blog/rss.xml", type: RSS, connector: rss, tier: B }
920+ - { name: server releases, url: "https://github.com/rabbitmq/rabbitmq-server/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: rabbitmq/rabbitmq-server, kind: releases } }
921+ - id: airflow
922+ name: Apache Airflow
923+ domain: airflow.apache.org
924+ categories: [open-source, developer, infrastructure]
925+ tier: B
926+ aliases: [airflow]
927+ llm: false
928+ discover: { rss: false, pages: true }
929+ notes: "Releases feed is dominated by provider packages — heuristics only."
930+ sensors:
931+ - { name: releases, url: "https://github.com/apache/airflow/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: apache/airflow, kind: releases } }
932+ - { name: blog index, url: "https://airflow.apache.org/blog/", type: HTML, connector: http, tier: C }
933+ - id: dbt
934+ name: dbt Labs
935+ domain: getdbt.com
936+ homepage: https://www.getdbt.com
937+ categories: [open-source, developer, infrastructure, enterprise]
938+ tier: B
939+ aliases: [dbt, dbt core, dbt cloud]
940+ products:
941+ - { name: dbt Core, type: software }
942+ - { name: dbt Cloud, type: service }
943+ discover: { rss: true, status: true }
944+ sensors:
945+ - { name: status, url: "https://status.getdbt.com/api/v2/summary.json", type: STATUSPAGE, connector: statuspage, tier: S }
946+ - { name: blog feed, url: "https://www.getdbt.com/blog/feed.xml", type: RSS, connector: rss, tier: B }
947+ - { name: dbt core releases, url: "https://github.com/dbt-labs/dbt-core/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: dbt-labs/dbt-core, kind: releases } }
948+
949+ # ───────────────────────── Applications & CMS ─────────────────────────
950+ - id: blender
951+ name: Blender
952+ domain: blender.org
953+ homepage: https://www.blender.org
954+ categories: [open-source, developer, entertainment]
955+ tier: B
956+ aliases: [blender foundation]
957+ discover: { rss: true }
958+ sensors:
959+ - { name: news feed, url: "https://www.blender.org/feed/", type: RSS, connector: rss, tier: B }
960+ - { name: tags, url: "https://github.com/blender/blender/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: blender/blender, kind: tags } }
961+ - id: vlc
962+ name: VideoLAN / VLC
963+ domain: videolan.org
964+ homepage: https://www.videolan.org
965+ categories: [open-source, developer, consumer-tech]
966+ tier: C
967+ aliases: [videolan, vlc media player]
968+ discover: { rss: false }
969+ notes: "videolan.org answers 418 to bots; releases are followed on the project GitLab tags feed."
970+ sensors:
971+ - { name: vlc tags, url: "https://code.videolan.org/videolan/vlc/-/tags?format=atom", type: ATOM, connector: rss, tier: C }
972+ - id: libreoffice
973+ name: LibreOffice / The Document Foundation
974+ domain: libreoffice.org
975+ homepage: https://www.libreoffice.org
976+ categories: [open-source, developer, consumer-tech]
977+ tier: B
978+ aliases: [the document foundation, tdf, libre office]
979+ discover: { rss: true }
980+ sensors:
981+ - { name: tdf blog feed, url: "https://blog.documentfoundation.org/feed/", type: RSS, connector: rss, tier: B }
982+ - { name: core tags, url: "https://github.com/LibreOffice/core/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: LibreOffice/core, kind: tags } }
983+ - id: wordpress
984+ name: WordPress
985+ domain: wordpress.org
986+ categories: [open-source, developer, internet]
987+ tier: A
988+ weight: 1.3
989+ aliases: [wordpress.org, wp]
990+ discover: { rss: true }
991+ sensors:
992+ - { name: news feed, url: "https://wordpress.org/news/feed/", type: RSS, connector: rss, tier: A }
993+ - { name: releases feed, url: "https://wordpress.org/news/category/releases/feed/", type: RSS, connector: rss, tier: A }
994+ - { name: tags, url: "https://github.com/WordPress/WordPress/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: WordPress/WordPress, kind: tags } }
995+ - id: drupal
996+ name: Drupal
997+ domain: drupal.org
998+ homepage: https://www.drupal.org
999+ categories: [open-source, developer, internet]
1000+ tier: B
1001+ weight: 1.1
1002+ aliases: [drupal association]
1003+ discover: { rss: true }
1004+ sensors:
1005+ - { name: security advisories feed, url: "https://www.drupal.org/security/all/rss.xml", type: RSS, connector: rss, tier: A }
1006+ - { name: news feed, url: "https://www.drupal.org/rss.xml", type: RSS, connector: rss, tier: B }
1007+ - { name: tags, url: "https://github.com/drupal/drupal/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: drupal/drupal, kind: tags } }
1008+ - id: joomla
1009+ name: Joomla
1010+ domain: joomla.org
1011+ homepage: https://www.joomla.org
1012+ categories: [open-source, developer, internet]
1013+ tier: B
1014+ aliases: [joomla!, open source matters]
1015+ discover: { rss: true }
1016+ sensors:
1017+ - { name: announcements feed, url: "https://www.joomla.org/announcements.feed?type=rss", type: RSS, connector: rss, tier: B }
1018+ - { name: security centre feed, url: "https://developer.joomla.org/security-centre.feed?type=rss", type: RSS, connector: rss, tier: A }
1019+ - { name: releases, url: "https://github.com/joomla/joomla-cms/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: joomla/joomla-cms, kind: releases } }
1020+ - id: ghost
1021+ name: Ghost
1022+ domain: ghost.org
1023+ categories: [open-source, developer, internet, media]
1024+ tier: B
1025+ aliases: [ghost cms, ghost foundation]
1026+ discover: { rss: true }
1027+ sensors:
1028+ - { name: changelog feed, url: "https://ghost.org/changelog/rss/", type: RSS, connector: rss, tier: B }
1029+ - { name: releases, url: "https://github.com/TryGhost/Ghost/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: TryGhost/Ghost, kind: releases } }
1030+ - id: matrix
1031+ name: Matrix.org
1032+ domain: matrix.org
1033+ categories: [open-source, developer, internet, standards]
1034+ tier: B
1035+ aliases: [matrix protocol, matrix.org foundation, element, synapse]
1036+ products:
1037+ - { name: Synapse, type: software }
1038+ - { name: Element, type: product }
1039+ discover: { rss: true }
1040+ sensors:
1041+ - { name: blog feed, url: "https://matrix.org/atom.xml", type: ATOM, connector: rss, tier: B, config: { maxItems: 40 } }
1042+ - { name: synapse releases, url: "https://github.com/element-hq/synapse/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: element-hq/synapse, kind: releases } }
1043+ - id: signal
1044+ name: Signal
1045+ domain: signal.org
1046+ categories: [open-source, consumer-tech, cyber]
1047+ tier: B
1048+ weight: 1.2
1049+ aliases: [signal messenger, signal foundation]
1050+ discover: { rss: true }
1051+ sensors:
1052+ - { name: blog feed, url: "https://signal.org/blog/rss.xml", type: RSS, connector: rss, tier: B }
1053+ - { name: android releases, url: "https://github.com/signalapp/Signal-Android/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: signalapp/Signal-Android, kind: releases } }
1054+ - id: tor-project
1055+ name: Tor Project
1056+ domain: torproject.org
1057+ homepage: https://www.torproject.org
1058+ categories: [open-source, cyber, internet, ngo]
1059+ tier: B
1060+ weight: 1.2
1061+ aliases: [tor, tor browser, the tor project]
1062+ products:
1063+ - { name: Tor Browser, type: software }
1064+ discover: { rss: true }
1065+ sensors:
1066+ - { name: blog feed, url: "https://blog.torproject.org/feed.xml", type: ATOM, connector: rss, tier: B }
1067+ - { name: tor tags, url: "https://gitlab.torproject.org/tpo/core/tor/-/tags?format=atom", type: ATOM, connector: rss, tier: B }
1068+ - id: wireshark
1069+ name: Wireshark
1070+ domain: wireshark.org
1071+ homepage: https://www.wireshark.org
1072+ categories: [open-source, developer, cyber]
1073+ tier: B
1074+ aliases: [wireshark foundation]
1075+ discover: { rss: false, pages: true }
1076+ notes: "Astro site without feeds; security advisories page is server-rendered."
1077+ sensors:
1078+ - { name: tags, url: "https://github.com/wireshark/wireshark/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: wireshark/wireshark, kind: tags } }
1079+ - { name: security advisories, url: "https://www.wireshark.org/security/", type: HTML, connector: http, tier: A }
1080+ - id: qemu
1081+ name: QEMU
1082+ domain: qemu.org
1083+ homepage: https://www.qemu.org
1084+ categories: [open-source, developer, infrastructure]
1085+ tier: B
1086+ aliases: [qemu project]
1087+ discover: { rss: true }
1088+ sensors:
1089+ - { name: blog feed, url: "https://www.qemu.org/feed.xml", type: ATOM, connector: rss, tier: B }
1090+ - { name: tags, url: "https://github.com/qemu/qemu/tags.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: qemu/qemu, kind: tags } }
1091+ - id: proxmox
1092+ name: Proxmox
1093+ domain: proxmox.com
1094+ homepage: https://www.proxmox.com
1095+ categories: [open-source, developer, infrastructure, enterprise]
1096+ tier: B
1097+ aliases: [proxmox ve, pve, proxmox server solutions]
1098+ products:
1099+ - { name: Proxmox VE, type: software, aliases: [pve] }
1100+ - { name: Proxmox Backup Server, type: software, aliases: [pbs] }
1101+ discover: { rss: true }
1102+ sensors:
1103+ - { name: press releases feed, url: "https://www.proxmox.com/en/about/company-details/press-releases?format=feed&type=rss", type: RSS, connector: rss, tier: B }
1104+ - { name: pve roadmap, url: "https://pve.proxmox.com/wiki/Roadmap", type: HTML, connector: http, tier: C }
1105+ - id: home-assistant
1106+ name: Home Assistant
1107+ domain: home-assistant.io
1108+ homepage: https://www.home-assistant.io
1109+ categories: [open-source, consumer-tech, developer]
1110+ tier: B
1111+ weight: 1.1
1112+ aliases: [hass, nabu casa, open home foundation]
1113+ discover: { rss: true, status: true }
1114+ sensors:
1115+ - { name: status, url: "https://status.home-assistant.io/api/v2/summary.json", type: STATUSPAGE, connector: statuspage, tier: S }
1116+ - { name: blog feed, url: "https://www.home-assistant.io/atom.xml", type: ATOM, connector: rss, tier: B }
1117+ - { name: core releases, url: "https://github.com/home-assistant/core/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: home-assistant/core, kind: releases } }
1118+ - id: obs-studio
1119+ name: OBS Studio
1120+ domain: obsproject.com
1121+ categories: [open-source, consumer-tech, media]
1122+ tier: B
1123+ aliases: [obs, obs project, open broadcaster software]
1124+ discover: { rss: true }
1125+ sensors:
1126+ - { name: blog feed, url: "https://obsproject.com/blog/rss", type: RSS, connector: rss, tier: B }
1127+ - { name: releases, url: "https://github.com/obsproject/obs-studio/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: obsproject/obs-studio, kind: releases } }
1128+
1129+ # ───────────────────────── Developer tools & editors ─────────────────────────
1130+ - id: jenkins
1131+ name: Jenkins
1132+ domain: jenkins.io
1133+ homepage: https://www.jenkins.io
1134+ categories: [open-source, developer, infrastructure, cyber]
1135+ tier: B
1136+ weight: 1.1
1137+ aliases: [jenkins ci]
1138+ discover: { rss: true }
1139+ sensors:
1140+ - { name: security advisories feed, url: "https://www.jenkins.io/security/advisories/rss.xml", type: RSS, connector: rss, tier: A }
1141+ - { name: changelog feed, url: "https://www.jenkins.io/changelog/rss.xml", type: RSS, connector: rss, tier: B }
1142+ - { name: blog feed, url: "https://www.jenkins.io/rss.xml", type: ATOM, connector: rss, tier: B }
1143+ - id: homebrew
1144+ name: Homebrew
1145+ domain: brew.sh
1146+ categories: [open-source, developer, packages]
1147+ tier: B
1148+ aliases: [brew]
1149+ discover: { rss: true }
1150+ sensors:
1151+ - { name: blog feed, url: "https://brew.sh/atom.xml", type: ATOM, connector: rss, tier: B }
1152+ - { name: releases, url: "https://github.com/Homebrew/brew/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: Homebrew/brew, kind: releases } }
1153+ - id: neovim
1154+ name: Neovim
1155+ domain: neovim.io
1156+ categories: [open-source, developer]
1157+ tier: B
1158+ aliases: [nvim]
1159+ discover: { rss: true }
1160+ sensors:
1161+ - { name: news feed, url: "https://neovim.io/news.xml", type: RSS, connector: rss, tier: B }
1162+ - { name: releases, url: "https://github.com/neovim/neovim/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: neovim/neovim, kind: releases } }
1163+ - id: vim
1164+ name: Vim
1165+ domain: vim.org
1166+ homepage: https://www.vim.org
1167+ categories: [open-source, developer]
1168+ tier: C
1169+ aliases: [vi improved]
1170+ llm: false
1171+ discover: { rss: false }
1172+ notes: "Patch tags land several times a day — heuristics only."
1173+ sensors:
1174+ - { name: tags, url: "https://github.com/vim/vim/tags.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: vim/vim, kind: tags } }
1175+ - id: microsoft
1176+ extend: true
1177+ aliases: [vs code, visual studio code]
1178+ products:
1179+ - { name: Visual Studio Code, type: software, aliases: [vs code, vscode] }
1180+ sensors:
1181+ - { name: vs code releases, url: "https://github.com/microsoft/vscode/releases.atom", type: GITHUB_RELEASE, connector: github, tier: A, config: { repo: microsoft/vscode, kind: releases } }
1182+ - { name: vs code updates feed, url: "https://code.visualstudio.com/feed.xml", type: ATOM, connector: rss, tier: B }
1183+ - id: zed
1184+ name: Zed
1185+ domain: zed.dev
1186+ categories: [open-source, developer, ai]
1187+ tier: B
1188+ aliases: [zed editor, zed industries]
1189+ discover: { rss: true }
1190+ notes: "GitHub releases feed is flooded by nightlies; zed.dev publishes a stable-releases feed."
1191+ sensors:
1192+ - { name: stable releases feed, url: "https://zed.dev/stable-releases.rss", type: RSS, connector: rss, tier: B }
1193+ - { name: blog feed, url: "https://zed.dev/blog.rss", type: RSS, connector: rss, tier: B, config: { maxItems: 40 } }
1194+ - id: godot
1195+ name: Godot Engine
1196+ domain: godotengine.org
1197+ categories: [open-source, developer, gaming]
1198+ tier: B
1199+ weight: 1.1
1200+ aliases: [godot, godot foundation]
1201+ discover: { rss: true }
1202+ sensors:
1203+ - { name: news feed, url: "https://godotengine.org/rss.xml", type: RSS, connector: rss, tier: B }
1204+ - { name: releases, url: "https://github.com/godotengine/godot/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: godotengine/godot, kind: releases } }
1205+ - id: electron
1206+ name: Electron
1207+ domain: electronjs.org
1208+ homepage: https://www.electronjs.org
1209+ categories: [open-source, developer]
1210+ tier: B
1211+ aliases: [electronjs]
1212+ llm: false
1213+ discover: { rss: true }
1214+ notes: "Releases feed publishes nightlies daily — heuristics only."
1215+ sensors:
1216+ - { name: blog feed, url: "https://www.electronjs.org/blog/rss.xml", type: RSS, connector: rss, tier: B }
1217+ - { name: releases, url: "https://github.com/electron/electron/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: electron/electron, kind: releases } }
1218+ - id: tauri
1219+ name: Tauri
1220+ domain: tauri.app
1221+ homepage: https://v2.tauri.app
1222+ categories: [open-source, developer]
1223+ tier: C
1224+ aliases: [tauri apps]
1225+ llm: false
1226+ discover: { rss: true }
1227+ notes: "Monorepo releases feed publishes every crate/package tag — heuristics only."
1228+ sensors:
1229+ - { name: blog feed, url: "https://v2.tauri.app/blog/rss.xml", type: RSS, connector: rss, tier: C }
1230+ - { name: releases, url: "https://github.com/tauri-apps/tauri/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: tauri-apps/tauri, kind: releases } }
1231+ - id: flutter
1232+ name: Flutter
1233+ domain: flutter.dev
1234+ categories: [open-source, developer]
1235+ tier: B
1236+ weight: 1.1
1237+ aliases: [flutter sdk, dart]
1238+ products:
1239+ - { name: Dart, type: software }
1240+ llm: false
1241+ discover: { rss: true }
1242+ notes: "Releases feed publishes beta cherry-picks continuously — heuristics only."
1243+ sensors:
1244+ - { name: blog feed, url: "https://medium.com/feed/flutter", type: RSS, connector: rss, tier: B }
1245+ - { name: releases, url: "https://github.com/flutter/flutter/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: flutter/flutter, kind: releases } }
1246+ - id: react-native
1247+ name: React Native
1248+ domain: reactnative.dev
1249+ categories: [open-source, developer]
1250+ tier: B
1251+ weight: 1.1
1252+ aliases: [rn]
1253+ discover: { rss: true }
1254+ sensors:
1255+ - { name: blog feed, url: "https://reactnative.dev/blog/rss.xml", type: RSS, connector: rss, tier: B, config: { maxItems: 40 } }
1256+ - { name: releases, url: "https://github.com/react/react-native/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: react/react-native, kind: releases } }
1257+ - id: expo
1258+ name: Expo
1259+ domain: expo.dev
1260+ categories: [open-source, developer, cloud]
1261+ tier: B
1262+ aliases: [expo go, eas]
1263+ products:
1264+ - { name: EAS, type: service, aliases: [expo application services] }
1265+ discover: { rss: true, status: true }
1266+ notes: "expo/expo has neither GitHub releases nor tags feeds; changelog + blog + status instead."
1267+ sensors:
1268+ - { name: status, url: "https://status.expo.dev/api/v2/summary.json", type: STATUSPAGE, connector: statuspage, tier: S }
1269+ - { name: changelog feed, url: "https://expo.dev/changelog/rss.xml", type: RSS, connector: rss, tier: B }
1270+ - { name: blog feed, url: "https://expo.dev/blog/rss.xml", type: RSS, connector: rss, tier: B }
1271+ - id: tailwind
1272+ name: Tailwind CSS
1273+ domain: tailwindcss.com
1274+ categories: [open-source, developer]
1275+ tier: B
1276+ aliases: [tailwind, tailwind labs]
1277+ discover: { rss: true }
1278+ sensors:
1279+ - { name: blog feed, url: "https://tailwindcss.com/feeds/feed.xml", type: RSS, connector: rss, tier: B }
1280+ - { name: releases, url: "https://github.com/tailwindlabs/tailwindcss/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: tailwindlabs/tailwindcss, kind: releases } }
1281+ - id: vite
1282+ name: Vite
1283+ domain: vite.dev
1284+ categories: [open-source, developer]
1285+ tier: B
1286+ aliases: [vitejs, void zero, rolldown]
1287+ llm: false
1288+ discover: { rss: true }
1289+ notes: "Monorepo releases feed mixes plugins and betas — heuristics only."
1290+ sensors:
1291+ - { name: blog feed, url: "https://vite.dev/blog.rss", type: RSS, connector: rss, tier: B }
1292+ - { name: releases, url: "https://github.com/vitejs/vite/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: vitejs/vite, kind: releases } }
1293+ - id: webpack
1294+ name: webpack
1295+ domain: webpack.js.org
1296+ categories: [open-source, developer]
1297+ tier: C
1298+ discover: { rss: true }
1299+ sensors:
1300+ - { name: blog feed, url: "https://medium.com/feed/webpack", type: RSS, connector: rss, tier: C }
1301+ - { name: releases, url: "https://github.com/webpack/webpack/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: webpack/webpack, kind: releases } }
1302+ - id: esbuild
1303+ name: esbuild
1304+ domain: esbuild.github.io
1305+ categories: [open-source, developer]
1306+ tier: C
1307+ discover: { rss: false }
1308+ sensors:
1309+ - { name: releases, url: "https://github.com/evanw/esbuild/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: evanw/esbuild, kind: releases } }
1310+ - id: rollup
1311+ name: Rollup
1312+ domain: rollupjs.org
1313+ categories: [open-source, developer]
1314+ tier: C
1315+ aliases: [rollup.js]
1316+ discover: { rss: false }
1317+ sensors:
1318+ - { name: releases, url: "https://github.com/rollup/rollup/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: rollup/rollup, kind: releases } }
1319+ - id: babel
1320+ name: Babel
1321+ domain: babeljs.io
1322+ categories: [open-source, developer]
1323+ tier: C
1324+ aliases: [babel js]
1325+ discover: { rss: true }
1326+ sensors:
1327+ - { name: blog feed, url: "https://babeljs.io/blog/rss.xml", type: RSS, connector: rss, tier: C, config: { maxItems: 30 } }
1328+ - { name: releases, url: "https://github.com/babel/babel/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: babel/babel, kind: releases } }
1329+ - id: eslint
1330+ name: ESLint
1331+ domain: eslint.org
1332+ categories: [open-source, developer]
1333+ tier: B
1334+ discover: { rss: true }
1335+ sensors:
1336+ - { name: blog feed, url: "https://eslint.org/feed.xml", type: ATOM, connector: rss, tier: B }
1337+ - { name: releases, url: "https://github.com/eslint/eslint/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: eslint/eslint, kind: releases } }
1338+ - id: prettier
1339+ name: Prettier
1340+ domain: prettier.io
1341+ categories: [open-source, developer]
1342+ tier: C
1343+ discover: { rss: true }
1344+ sensors:
1345+ - { name: blog feed, url: "https://prettier.io/blog/rss.xml", type: RSS, connector: rss, tier: C, config: { maxItems: 30 } }
1346+ - { name: releases, url: "https://github.com/prettier/prettier/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: prettier/prettier, kind: releases } }
1347+ - id: pnpm
1348+ name: pnpm
1349+ domain: pnpm.io
1350+ categories: [open-source, developer, packages]
1351+ tier: B
1352+ discover: { rss: true }
1353+ sensors:
1354+ - { name: blog feed, url: "https://pnpm.io/blog/rss.xml", type: RSS, connector: rss, tier: C }
1355+ - { name: releases, url: "https://github.com/pnpm/pnpm/releases.atom", type: GITHUB_RELEASE, connector: github, tier: B, config: { repo: pnpm/pnpm, kind: releases } }
1356+ - id: yarn
1357+ name: Yarn
1358+ domain: yarnpkg.com
1359+ categories: [open-source, developer, packages]
1360+ tier: C
1361+ aliases: [yarn berry, yarnpkg]
1362+ discover: { rss: false }
1363+ sensors:
1364+ - { name: releases, url: "https://github.com/yarnpkg/berry/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: yarnpkg/berry, kind: releases } }
1365+ - id: bazel
1366+ name: Bazel
1367+ domain: bazel.build
1368+ categories: [open-source, developer]
1369+ tier: C
1370+ aliases: [bazelbuild]
1371+ discover: { rss: true }
1372+ sensors:
1373+ - { name: blog feed, url: "https://blog.bazel.build/feed.xml", type: RSS, connector: rss, tier: C }
1374+ - { name: releases, url: "https://github.com/bazelbuild/bazel/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: bazelbuild/bazel, kind: releases } }
1375+ - id: cmake
1376+ name: CMake / Kitware
1377+ domain: cmake.org
1378+ categories: [open-source, developer]
1379+ tier: C
1380+ aliases: [kitware]
1381+ discover: { rss: true }
1382+ sensors:
1383+ - { name: releases, url: "https://github.com/Kitware/CMake/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: Kitware/CMake, kind: releases } }
1384+ - { name: kitware blog feed, url: "https://www.kitware.com/feed/", type: RSS, connector: rss, tier: C }
1385+ - id: gradle
1386+ name: Gradle
1387+ domain: gradle.org
1388+ categories: [open-source, developer]
1389+ tier: C
1390+ aliases: [gradle build tool, develocity]
1391+ discover: { rss: true }
1392+ notes: "blog.gradle.org has no feed in 2026."
1393+ sensors:
1394+ - { name: releases, url: "https://github.com/gradle/gradle/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: gradle/gradle, kind: releases } }
1395+ - id: maven
1396+ name: Apache Maven
1397+ domain: maven.apache.org
1398+ categories: [open-source, developer, packages]
1399+ tier: C
1400+ aliases: [maven]
1401+ discover: { rss: false, pages: true }
1402+ sensors:
1403+ - { name: releases, url: "https://github.com/apache/maven/releases.atom", type: GITHUB_RELEASE, connector: github, tier: C, config: { repo: apache/maven, kind: releases } }
1404+ - { name: release history, url: "https://maven.apache.org/docs/history.html", type: HTML, connector: http, tier: C }
added config/sources.d/11-central-banks-finance.yaml +1724 −0
Large diff (1,724 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/12-energy-climate-weather.yaml +1542 −0
Large diff (1,542 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/13-telecom-internet-infra.yaml +1712 −0
Large diff (1,712 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/14-retail-consumer-travel.yaml +1621 −0
Large diff (1,621 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/15-gaming-entertainment.yaml +1194 −0
Large diff (1,194 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/16-universities-research.yaml +1768 −0
Large diff (1,768 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/17-international-orgs-ngos-standards.yaml +2072 −0
Large diff (2,072 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/18-sports.yaml +1673 −0
Large diff (1,673 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/19-crypto-web3.yaml +1221 −0
Large diff (1,221 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/20-transport-logistics-aviation-space.yaml +2164 −0
Large diff (2,164 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/21-consumer-safety-food-agri.yaml +770 −0
Large diff (770 lines) is not loaded by default to keep the page fast.Load diff
added config/sources.d/22-housing-labour-opendata.yaml +651 −0
Large diff (651 lines) is not loaded by default to keep the page fast.Load diff

Diff truncated — file too large.

Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere