SPB Git forge

spb/websensor

Public
33commits 1branches 0releases
3.4 MBsize
maindefault branch
10 days agolast push
TypeScript 55.4% Python 43.2% SQL 1.2%
1.8 KB · 20 lines markdown
Rendered Raw Blame History
1# Connector: Package registries (`package`)2**Purpose**: the version streams of the software supply chain — one connector, ten registries: npm, PyPI,3crates.io, RubyGems, NuGet (flat container), Packagist (p2), Hex, Go module proxy, Homebrew formulae, Docker Hub4tags. A new version, a moved `latest` tag, a yanked release, a rebuilt Docker tag (same name, new digest) or a5Homebrew revision bump becomes a list event on the project entity (`software_release`).6**Sensor types**: REST_API, JSON. **Config**: `{ registry, name, maxItems? (50), url? }` — the API URL is derived7(`registryApiUrl`), the sensor `url` is only the human page. Names: npm `@scope/pkg`, Packagist `vendor/pkg`,8Go `github.com/owner/repo` (lower-cased for the proxy), Docker `library/nginx` or `owner/image`.9**Normalization**: keyed list — `key = version` (+ `prerelease`, `yanked`, `digest`, `publishedAt`), plus a10`latest` item whose `version` field is the registry's stable pointer; `compareFields: [version]`11(`[version, digest]` for Docker Hub, `[version, revision]` for Homebrew). Sorted newest first; `extra` carries12`latest`, `versionCount`, `description`.13**Rate limits**: npm/PyPI/crates/RubyGems/Packagist/Hex/Go proxy/Homebrew are CDN-served and unlimited for this14volume; crates.io requires a descriptive User-Agent (sent); Docker Hub anonymous API ≈ 100 req/6 h per IP → tier15C/D only. High-volume packages (canary builds every commit: Next.js, React canaries) should set `llm: false`16on the source or a `maxItems` cap.17**Known quirks**: NuGet's gz registration endpoint returns raw gzip — the flat-container index is used instead;18Homebrew exposes only the current stable version (list of one + `latest`); Go proxy `@v/list` has no dates.19**Tests**: `packages/connectors/src/connectors2.test.ts`. **Verified**: 2026-09-08 on all ten registries.20