# Connector: Sitemap (`sitemap`)
**Purpose**: page-level discovery — new URLs, removed URLs, `lastmod` bumps — including news sitemaps.
**Sensor type**: SITEMAP. **Retrieval**: conditional GET, 30 MB limit, gzip transparently decompressed
(magic-byte check), sitemap index followed (`maxChildren`, default 6; newest first when `lastmod` is present).
**Normalization**: keyed list by URL, `compareFields: lastmod`; sorted newest first; `include`/`exclude` regexes;
`maxUrls` (default 5000). A > 50 % shrink versus the previous snapshot is treated as a partial response (no
removals reported).
**Events**: `page_created` (new URL), `page_removed` (URL gone, only after confirmations), `documentation_change`
etc. via heuristics on titles/URLs.
**Known quirks**: huge multilingual indexes (BMW: 100+ children) — choose a specific child sitemap; image/video
sitemaps are skipped by discovery. **Tests**: fixtures `tests/fixtures/sitemap.xml`. **Verified**: 2026-09-08
(anthropic.com sitemap 524 URLs, BMW press sitemap).