Tendril — web ingestion platform (scrape/crawl/map/search) on macOS Apple Silicon: WebKit fidelity, authenticated pages, deterministic testable extraction. A self-hosted Firecrawl alternative.
-
Mon, Aug 10, 2026 3
-
feat(api): serve interactive playground + docs platform at /
…
Self-contained landing page served via @fastify/static (apps/api/public): - hero with Tendril identity (botanical/terminal theme, animated vine SVG) - live playground: scrape/map/status, formats & options, latency, rendered markdown preview, links/metadata/structured tabs, curl/python/js snippets with copy, live /v1/status pill - collapsible API reference with param tables + examples, §15 error table Explicit API routes keep priority over the static wildcard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat: add /map, robots.txt enforcement, and m3u96a deployment
…
- frontier: RFC 9309 robots.txt parser + matcher (longest-match, *,$), 24h cache with 5xx=disallow-1h, recursive sitemap/index discovery, mapSite() merging sitemaps + homepage links + /llms.txt (§10.1, §14.3) - scrape: enforce robots.txt by default (respectRobots), ERR_ROBOTS_DENIED (§26) - api: POST /v1/map, GET /v1/status, @fastify/compress (br/gzip), graceful shutdown; workspace packages now expose dist/ for production node execution - deploy: PM2 ecosystem (loopback API + ngrok www.ten-dril.com), ngrok.yml, scripts/deploy.sh + health.sh, deployed to m3u96a:8092 (§16) 96 tests green. Ingress blocked only on reserving www.ten-dril.com in the ngrok dashboard (DNS CNAME already live). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat: scaffold Tendril monorepo with Phase 1 foundations
…
Web ingestion platform (scrape/crawl/map/search) for macOS Apple Silicon, following CLAUDE.md. Phase 1 (§24) delivered and fully tested (74 tests): - shared: Result<T,E>, §15 error taxonomy (single source), types, Pino logger with redaction, URL normalization (§10.1) - router: pure shouldEscalate escalation decision (§3.4) - egress: SSRF validation with resolve-then-validate + IP pinning (§16.5) - fetcher-http: Tier 0 undici client, Safari header order, manual redirects with per-hop SSRF re-validation, meta-refresh, gzip/br (§3) - extract: deterministic pipeline (§8) — JSON-LD/OG/Twitter harvest, Readability + density fallback, Turndown rules (GFM tables, fenced code, figures, dl), link extraction - api: Fastify POST /v1/scrape + /healthz, errors.ts HTTP mapping (§15) TypeScript strict (noUncheckedIndexedAccess, exactOptionalPropertyTypes), fetchers/extractors return Result, author header on every code file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-