SPB Git

spb/worthdoing Public

Autonomous investigation agent that discovers, challenges, and ranks things genuinely worth doing — Claude + Firecrawl, Next.js 16, PostgreSQL

TypeScript 91.5% SQL 5.8% CSS 2.2%
5.6 KB

WorthDoing.ai

WorthDoing.ai

Google finds what exists. WorthDoing finds what should.

Next.js 16 TypeScript strict PostgreSQL 17 Drizzle ORM Claude Opus 5 Firecrawl v2 Tests SSE realtime License


WorthDoing.ai is an agentic web application that continuously discovers, investigates, challenges, and ranks things genuinely worth doing. It answers a different question from search engines:

What should exist, be built, researched, tested, funded, or pursued that does not exist yet — or is not being pursued enough?

It is not a thin LLM wrapper and not a fixed query → search → scrape → summarize pipeline. It is a real multi-step autonomous investigation agent: Claude repeatedly decides what it knows, what remains uncertain, what to search next, which sources to inspect, whether to reject hypotheses, and when the evidence suffices to conclude.

# How an investigation works

  1. Scout — the agent maps a domain with varied Firecrawl searches and forms falsifiable hypotheses.
  2. Investigate — targeted searches (verify / market / technical / competition), page scrapes, and evidence saved as near-verbatim quotes linked to hypotheses.
  3. Skeptic — mandatory falsification: every high-confidence hypothesis gets adversarial falsify searches. Hypotheses weaken, branch, or die. A rejected hypothesis is useful progress.
  4. Synthesize — surviving hypotheses become opportunities with a seven-dimension, evidence-cited Worth Score (demand, neglectedness, feasibility, why-now, impact, competition, risk) — with evidence confidence reported separately, never blended in.
  5. Report — a streamed, citation-grounded report per opportunity; citations resolve mechanically to saved evidence, never invented.

Every UI event corresponds to a real backend AgentEvent streamed over SSE — the live timeline never fabricates progress.

# Screenshots

Discover Live investigation
Discover page Live investigation
Opportunity report Mobile (390 px)
Opportunity report Mobile live view

Real output: the agent investigated "Find things worth doing in local AI" — 21 steps, 14 searches, 17 evidence items, one hypothesis rejected and one weakened by adversarial checks, one opportunity (Worth Score 74 at 69% evidence confidence) with a fully cited report.

# Architecture

text
Next.js 16 (App Router, port 3001)
├── src/lib/agent/          the investigation engine
│   ├── engine.ts           explicit tool loop (Claude decides → backend executes)
│   ├── tools.ts            11 tools, Zod-validated, strict schemas
│   ├── executors.ts        budget enforcement, dedup, persistence, events
│   ├── state.ts            InvestigationState + compact digest (state compression)
│   ├── synthesis.ts        streamed reports with mechanical citations
│   └── scoring.ts          evidence-weighted Worth Score
├── src/lib/firecrawl/      search / scrape / crawl / extract behind an adapter
│   └── cache.ts            canonical URLs, sha-256 content hashes, 24h freshness
├── src/lib/anthropic/      Claude adapter (streaming, cost telemetry)
├── src/lib/db/             Drizzle schema — 17 tables (Opportunity-Graph-ready)
└── src/app/                Discover · live investigation · reports · explore · history

Hard guarantees

  • Budgets are hard bounds (30 steps, 20 searches, 60 scrapes, 3 crawls, wall-time) with structured stop reasons.
  • No conclusion without traceable evidence; every score dimension carries {score, confidence, reasoning, evidenceIds}.
  • Scraped content is untrusted data, never instructions (prompt-injection defense in the system prompt and <untrusted_source> wrapping).
  • Investigations are resumable: state lives in Postgres, not in the model's context.

# Getting started

bash
cp .env.example .env       # fill in ANTHROPIC_API_KEY, FIRECRAWL_API_KEY, DATABASE_URL
pnpm install
pnpm db:migrate
pnpm dev                   # http://localhost:3001
pnpm test                  # unit tests (scoring, dedup, tool contracts)

# Deployment

Runs on node m3u96a behind an ngrok tunnel mapped to https://www.worthdoing.ai (see deploy/). SSE routes are proxy-safe: Cache-Control: no-cache, X-Accel-Buffering: no, per-event flush, Last-Event-ID replay on reconnect.


# Author

Simon-Pierre Boucher 📧 contact@spboucher.ai

Deployment: node m3u96a via ngrok → https://www.worthdoing.ai