SPB Git forge

spb/websensor

Public
33commits 1branches 0releases
3.4 MBsize
maindefault branch
10 days agolast push
TypeScript 55.4% Python 43.2% SQL 1.2%
1.4 KB · 15 lines markdown
Rendered Raw Blame History
1# Connector: HTTP response headers (`headers`)2**Purpose**: the security and infrastructure posture visible in a homepage's response headers — HSTS (and preload),3CSP, frame/content-type/referrer/permissions policies, COOP/COEP/CORP, CORS, cookie flags (names + Secure/HttpOnly/4SameSite only), `server` / `x-powered-by` / CDN markers (Cloudflare, Akamai, Fastly, CloudFront, Vercel…), cache5policy, `alt-svc`, reporting endpoints, `x-robots-tag`. A dropped HSTS header, a new CDN, a weakened CSP or a new6tracking cookie is a JSON change; `extra.securityScore` (0–7) summarizes the basics.7**Sensor type**: HTTP_HEADERS, tier D. **Config**: `method?` (GET — HEAD is often refused or answered by a different8edge), `include?` / `exclude?` header names, `headers?` (request).9**Retrieval**: the fetcher keeps all response headers (`keepAllHeaders`), the body is discarded (limit 24 MB so heavy10homepages such as cloud.google.com are not rejected). Per-request noise is scrubbed: CSP nonces, request/ray ids,11PoP identifiers, `date`, `age`, `etag`, versioned asset query strings in `link`.12**Known quirks**: bot-managed sites (openai.com, x.ai) answer 403 to any automated client — the 403 itself is13observed, not treated as an outage; some CDNs vary headers per PoP (`x-served-by` reduced to `present`).14**Tests**: covered live by the validator. **Verified**: 2026-09-08 on the founding tier S/A homepages.15