# Connector: Generic HTTP (http)

Purpose: HTML pages (pricing, news lists, security bulletins, changelogs), JSON documents, plain text and HEAD-only header monitoring. Sensor types: HTML, JSON, XML, HTTP_HEADERS, FILE. Retrieval: conditional GET (ETag / If-Modified-Since), gzip/br, manual redirects (≤ 5, SSRF-checked per hop), 12 MB limit, 25 s timeout, HTTP/2 with automatic HTTP/1.1 pin on NGHTTP2 stream errors, one browser-UA retry on 403 or TLS/connection resets, Scrapfly fallback if the source allows it. Normalization: canonicalizeHtml (removes script/style/nav/footer/aside/cookie chrome, comments, tracking params; scrubs clock times, "x minutes ago", hashes/UUIDs, tokens, visitor counters) → canonical text, headings, links, structure counts, meta dates; JSON → sorted-key canonical form (jsonPath, ignoreKeys config). Thin pages (< 40 chars, no headings: JS shells, interstitials) get extraction confidence 0.3 and never produce events (flip protection). Config: keepChrome, jsonPath, ignoreKeys, accept, headers, method: HEAD, renderJs (Scrapfly). Known quirks: Akamai/Cloudflare bot management returns 403 or resets the connection for unknown UAs (handled by the browser-UA retry, else Scrapfly). Client-rendered pages produce thin canonical text — prefer feeds/APIs. Tests: packages/core/src/core.test.ts (canonicalization, diff, heuristics). Verified: 2026-09-08 on ~85 HTML/JSON sensors (Anthropic news/pricing, Stripe changelog, Apple system status, Wikipedia ITN…).