# Tendril
**Web ingestion that reaches what other crawlers can't.**
Scrape Β· Map β running on a residential Mac, in **real WebKit**, with **deterministic, testable** extraction.
[](https://www.ten-dril.com)
**[π Live](https://www.ten-dril.com) Β· [βΆ Playground](https://www.ten-dril.com/#playground) Β· [π€ For agents](https://www.ten-dril.com/#agents) Β· [π Docs](https://www.ten-dril.com/#docs) Β· [π¦ Source](https://git.spboucher.ai/tendril)**
---
## β¦ Why Tendril
Firecrawl runs in the cloud, on datacenter IPs, in headless Chromium. Tendril runs on a **residential machine**, in **WebKit** β Safari's actual engine β behind a real IP.
| | |
|---|---|
| πΈοΈ **Browser fidelity** | WebKit on macOS ARM behind a residential IP is *not* a headless-Chromium signature. Sites that block others often don't block Tendril. |
| π **Deterministic extraction** | No LLM in the extraction path. The same page always yields the same Markdown β so you can write **golden-file tests** against it. |
| π§ **Map without rendering** | Merge sitemaps, robots directives, homepage links and `/llms.txt` into one deduplicated URL list in seconds. |
| π **Fair by default** | Identifiable bot UA + contact URL, `robots.txt` honoured by default, per-host rate limiting, SSRF-hardened. |
---
## β‘ Quickstart
```bash
# Scrape a page β clean Markdown + structured data + links
curl -s -X POST https://www.ten-dril.com/v1/scrape \
-H 'content-type: application/json' \
-d '{"url":"https://www.iana.org","formats":["markdown","structured","links"]}'
# Map a site β every URL, no rendering
curl -s -X POST https://www.ten-dril.com/v1/map \
-H 'content-type: application/json' \
-d '{"url":"https://www.iana.org","limit":50}'
```
Every response is `{ "success": boolean, "data"?: β¦, "error"?: { "code", "message" } }` with an `X-Request-Id` header.
---
## π€ Give it to your AI agent
Hand Tendril to any agent β two drop-in integrations, downloadable from the site:
| | Download | Install |
|---|---|---|
| **MCP server** β single file, no `npm install` | [`tendril-mcp.mjs`](https://www.ten-dril.com/downloads/tendril-mcp.mjs) Β· [`.zip`](https://www.ten-dril.com/downloads/tendril-mcp.zip) | `claude mcp add tendril -- node /path/to/tendril-mcp.mjs` |
| **Agent Skill** β `SKILL.md` | [`tendril-skill.zip`](https://www.ten-dril.com/downloads/tendril-skill.zip) | `unzip tendril-skill.zip -d ~/.claude/skills/` |
The MCP server exposes `tendril_scrape`, `tendril_map`, `tendril_status`. Both point at `https://www.ten-dril.com` and need no API key today.
---
## ποΈ Architecture β three-tier escalation router
Always start at the cheapest tier; escalate only on evidence of failure.
```
request βββΆ Router ββshouldEscalate()βββ
β β
ββββββββββΌββββββββββββββ β
βΌ βΌ βΌ β
βββββββββββ βββββββββββββ ββββββββββββ β
β TIER 0 β β TIER 1 β β TIER 2 β β
β undici ββΆβ WKWebView ββΆβ Safari β β β Tiers 1/2 = next phase
β HTTP β β pool β βdriver β β
ββββββ¬βββββ βββββββ¬ββββββ ββββββ¬ββββββ β
ββββββββββββββ΄βββββββββββββ΄ββββββββ
βΌ
Extraction engine (deterministic)
βΌ
markdown Β· structured Β· links
```
---
## π¦ Endpoints
| Method | Path | Purpose |
|---|---|---|
| `POST` | `/v1/scrape` | Fetch one page β Markdown, structured data, links, metadata |
| `POST` | `/v1/map` | Discover a site's URLs (sitemaps + robots + homepage + `/llms.txt`) |
| `GET` | `/v1/status` | Per-tier availability + pipeline version |
| `GET` | `/healthz` | Shallow health for uptime monitors |
---
## 𧩠What's built
| Area | Spec | Status |
|---|---|---|
| Error taxonomy (single source) | Β§15 | β
|
| `Result