spb/fetcha

1# Fetcha — shared engineering brief for parallel agents23You are building part of **Fetcha** (fetcha.co), "Intelligent Web Access Infrastructure": one API that routes4web requests across upstream proxy networks. Repo root: `~/Desktop/Projets/apps-web/fetcha` (pnpm + Turborepo monorepo).5The product spec is long; the essentials you need are here. Read the files you touch or depend on before writing.67## Non-negotiable rules81. **Never expose upstream provider names** (Oxylabs, Decodo, SOAX) anywhere customers can see: marketing, docs, dashboard,9   API responses, error messages. Customers see *network classes*: `datacenter`, `residential`, `isp`, `mobile`. Only the10   `/admin` area may show provider names.112. **No fake functionality.** Not available in V1 (say so explicitly, mark "coming soon" / "not yet available"):12   managed browser (`browser: true` → `BROWSER_UNAVAILABLE`), `/v1/extract`, `/v1/browser`, Stripe checkout, teams/invitations,13   webhooks delivery, OAuth login, 2FA/passkeys, `datacenter`/`isp`/`mobile` network classes (currently only `residential` is live;14   `auto` resolves to residential). Say what exists; keep the UI for future things honest ("Coming soon").153. **Business logic never lives in React components.** Use server components + server actions (`src/actions/*.ts`) or16   `src/lib/queries/*.ts` for data access. Client components only for interactivity.174. **Security:** never show plaintext API keys except at creation; redact `Authorization`/`Cookie`; never trust user HTML —18   render fetched HTML previews only inside a sandboxed `<iframe sandbox="" srcDoc=…>`.195. Every screen needs: strong hierarchy, loading state (`loading.tsx` or Suspense + `Skeleton`), empty state (`EmptyState`20   that teaches the next action), error handling, responsive layout, keyboard accessibility. Dark + light both first-class.216. Design direction: Cloudflare / Vercel / Linear / Stripe. Minimal, technical, generous whitespace, precise 13–15px type,22   tabular numbers for data (`className="tabular font-mono"`), tables/inline metrics over card grids. No gradients/neon/AI-SaaS clichés.23   Do not clone Scrapfly. Accent is blue (`text-accent`, `bg-accent`). Radii are small (rounded-md/lg).247. Do **not** edit files you don't own (listed per task). Do not modify `packages/*`, `apps/api/*`, `globals.css`,25   `layout.tsx` roots, `components/ui/*`, `lib/*` (except adding new files under `src/lib/queries/` or `src/lib/<yourarea>-*.ts`).26   If you need a shared change, write a note at the end of your report instead.278. When done, run `cd apps/web && npx tsc -p tsconfig.json --noEmit` and fix errors **in your files**. Ignore errors from other28   agents' files. Also `pnpm --filter @fetcha/web lint` if quick. Do not run `next build` (another process will).2930## Stack31Next.js 16 (App Router, `src/app`), React 19, TypeScript strict, Tailwind v4 (`@theme` tokens in `src/app/globals.css`),32Radix primitives wrapped in `src/components/ui/*`, `lucide-react` icons, `recharts` (client only), `framer-motion` available,33Drizzle ORM + Postgres via `@fetcha/db`, Better Auth (`src/lib/auth.ts`, client `src/lib/auth-client.ts`).34Path alias `@/` → `apps/web/src/`. Pages default to server components; add `"use client"` only when needed.35Route segments: `(marketing)` public pages with nav/footer, `(auth)` login/signup, `dashboard/*`, `admin/*`.36Dashboard pages must `export const dynamic = "force-dynamic"` (they read cookies/session).3738## Design tokens (Tailwind classes)39Colors: `bg-bg`, `bg-bg-subtle`, `bg-bg-muted`, `bg-bg-elevated`, `text-fg`, `text-fg-muted`, `text-fg-subtle`, `border-border`,40`border-border-strong`, `bg-accent`/`text-accent`/`bg-accent-soft`, `text-success`/`bg-success-soft`, `warning`, `danger`, `info`.41Utilities: `container-page` (max-w-6xl centered), `tabular`, `grid-bg`, `scrollbar-thin`, `animate-fade-in`, `animate-fade-up`.42Fonts: sans default; `font-mono` for ids, code, numbers.4344## Shared components (`@/components/ui/*`) — use these, don't reinvent45- `button` → `Button` (variant: default|primary|secondary|outline|ghost|danger|link; size: default|sm|xs|lg|icon|icon-sm; `loading`, `asChild`)46- `input` → `Input`, `Textarea`, `NativeSelect`, `inputClass`47- `label` → `Label`, `Hint`, `FieldError`, `Field` (grid gap wrapper)48- `card` → `Card`, `CardHeader`, `CardTitle`, `CardDescription`, `CardContent`, `CardFooter`49- `badge` → `Badge` (variant default|outline|accent|success|warning|danger|info|solid, `dot`), `StatusBadge status="success|failed|pending|active|…"`50- `table` → `Table`, `TableHeader`, `TableBody`, `TableRow`, `TableHead`, `TableCell`, `TableEmpty colSpan`51- `tabs` → `Tabs`, `TabsList variant="pill"|"underline"`, `TabsTrigger`, `TabsContent` (client)52- `dialog` → `Dialog`, `DialogTrigger`, `DialogContent size="sm|md|lg|xl"`, `DialogHeader`, `DialogTitle`, `DialogDescription`, `DialogFooter`, `DialogClose`53- `select` → Radix `Select`, `SelectTrigger`, `SelectValue`, `SelectContent`, `SelectItem`, `SelectLabel`, `SelectGroup`54- `switch` → `Switch`; `dropdown-menu` → `DropdownMenu*`; `tooltip` → `Tooltip*`, `SimpleTooltip content=`55- `skeleton` → `Skeleton`, `TableSkeleton rows cols`56- `empty-state` → `EmptyState icon title description action compact`57- `copy-button` → `CopyButton value label?`58- `code-block` → `CodeBlock code lang title? lineNumbers? maxHeight? copy?` (lang: bash|json|javascript|typescript|python|go|php|ruby|java|csharp|html|text), `InlineCode`, `highlight()`59- `page-header` → `PageHeader title description actions eyebrow`, `SectionTitle right?`60- `stat` → `Stat label value hint delta`, `StatGrid cols=2|3|4|5|6` (children are `Stat`s; renders divided grid)61- `alert` → `Alert variant="info|success|warning|danger" title action`62- `logo` → `Logo href size`, `LogoMark size`63- `theme` → `ThemeToggle`6465## Server helpers (`@/lib/*`)66- `session.ts`: `getSession()`, `getUser()`, `requireUser(next?)`, `requireAdmin()`, `isAdmin(user)`,67  `getWorkspace()` → `{ user, organization, role, projects, project /*current*/, isAdmin }` (cached per request; redirects to /login if anon),68  `requestMeta()` → `{ ip, userAgent }`, `PROJECT_COOKIE`.69- `api.ts` (server-only): `internalApi.playgroundFetch(projectId, userId, request)`, `createSession`, `listSessions`, `closeSession`,70  `usage(projectId,userId)`, `providers()` (admin: provider list + health + circuits), `probeProviders()`, `reloadProviders()`,71  `resetCircuit(key?)`, `health()`, `ready()`. Throws `InternalApiError { status, code, message, requestId, details }`.72- `format.ts`: `formatNumber`, `formatCompact`, `formatBytes`, `formatMs`, `formatUsd(n, precise?)`, `formatPercent`, `formatDate`,73  `formatDateOnly`, `timeAgo`, `truncate`, `titleCase`.74- `utils.ts`: `cn`, `slugify`, `SITE_URL`, `API_PUBLIC_URL`.75- `audit.ts`: `writeAudit({ userId, organizationId, action, target, metadata, ipAddress, userAgent })`.76- `auth-client.ts` (client): `authClient` (Better Auth React client: `signIn.email`, `signUp.email`, `signOut`, `changePassword`,77  `changeEmail`, `listSessions`, `revokeSession`, `revokeOtherSessions`, `deleteUser`, `sendVerificationEmail`, `updateUser`).78- Existing server actions: `@/actions/projects` (`createProject(formData)`, `updateProject(id, formData)`, `archiveProject(id)`,79  `selectProject(id)`; `ActionResult<T>` type), `@/actions/api-keys` (`createApiKey({name, projectId, mode, scopes, expiresInDays})` →80  `{ ok, data: { id, plaintext, prefix, name, projectId } }`, `revokeApiKey(id)`, `rotateApiKey(id)`), `@/actions/account`81  (`completeOnboarding()`, `updateOrganization(formData)`, `updateProfileName(name)`, `recordAuditFromClient(action, meta)`).8283## Data model (`@fetcha/db`, Drizzle; import `{ getDb, <table>, eq, and, desc, sql, gte, … } from "@fetcha/db"`)84Tables (TS names): `users`, `sessions`, `accounts`, `verifications`, `organizations` (plan, providerVisibility, softLimitUsd, hardLimitUsd,85suspended, ownerUserId), `organizationMembers`, `projects` (name, slug, environment, defaultCountry, defaultNetwork, logLevel,86soft/hardLimitUsd, monthlyRequestLimit, archivedAt), `apiKeys` (name, keyPrefix, last4, mode, scopes jsonb string[], expiresAt,87lastUsedAt, revokedAt, keyHash — never display), `fetchRequests` (id req_…, organizationId, projectId, apiKeyId, source api|playground|sdk,88url, finalUrl, domain, method, requestedNetwork, network, country, region, city, sessionId, browser, format, status pending|success|failed,89httpStatus, errorCode, errorMessage, attempts, latencyMs, bytesIn, bytesOut, costUsd (upstream, admin-only), priceUsd (billed),90cached, requestHeaders, responseHeaders, timing jsonb, clientIp, userAgent, createdAt, completedAt), `requestAttempts` (requestId,91attemptNo, provider ⚠ admin-only, network, country, outcome success|blocked|timeout|error|provider_error|too_large, httpStatus,92errorCode, errorDetail ⚠ admin-only, blockReason, durationMs, bytesIn, bytesOut, costUsd, routingScore, timing), `proxySessions`93(id sess_…, projectId, label, provider ⚠, network, country, status active|expired|closed, requestCount, lastUsedAt, expiresAt),94`usageEvents` (organizationId, projectId, requestId, metric request|bandwidth|residential_bandwidth|mobile_bandwidth, quantity, unit,95costUsd = customer price, upstreamCostUsd ⚠, createdAt), `subscriptions`, `billingEvents`, `providerConfigs` (id, label, enabled,96networks, pricePerGbUsd jsonb, weight, maxConcurrency), `providerHealth` (provider, network, status healthy|degraded|down|unconfigured,97latencyMs, detail, checkedAt), `domainProfiles` (domain, preferredNetwork, preferredProvider ⚠, requests, successes, blocks, captchas,98browserRequired, avgLatencyMs, routeStats jsonb, policy jsonb {order?, force_network?}, lastSeenAt), `routingMetrics` (hourly bucket,99provider, network, country, requests, successes, blocked, errors, latencySumMs, bytes, costUsd), `webhooks`, `webhookDeliveries`,100`auditLogs` (userId, organizationId, action, target, metadata, ipAddress, userAgent, createdAt), `abuseEvents`, `featureFlags`101(key, enabled, description, plans, organizationIds), `legalAcceptances`, `statusIncidents` (component, title, body, severity, startedAt, resolvedAt).102Always scope customer queries by `organizationId`/`projectId` from `getWorkspace()`.103104`@fetcha/core` exports: `PLAN_LIMITS` (per plan: label, monthly_requests, concurrency, max_timeout_ms, max_retries, networks,105retention_days, price_usd_month, included_gb, overage_per_1k_requests_usd, residential_per_gb_usd), `PLANS`, `NETWORK_CLASSES`,106`API_KEY_SCOPES` (`fetch:execute`, `browser:use`, `sessions:write`, `usage:read`), `ERROR_CODES`, `ERROR_MESSAGES`, `ERROR_HTTP_STATUS`,107`COUNTRIES` (ISO2 → name), `fetchRequestSchema` (zod), `FetchRequest`, `FetchResponseBody` types.108109## Public API surface (what docs / codegen must describe)110Base URL: `https://www.fetcha.co` (API is served under the same domain: `POST https://www.fetcha.co/v1/fetch`). Auth header:111`Authorization: Bearer fch_live_…` (test keys `fch_test_…`). Response header `X-Fetcha-Request-ID: req_…`.112- `POST /v1/fetch` body: `{ url (required), method="GET", headers, cookies, body, timeout=30000 (ms, ≤ plan max), country (ISO2),113  region, city, network="auto"|datacenter|residential|isp|mobile, session ("sess_…"), browser=false, device, locale, format="html"|text|json|raw,114  follow_redirects=true, max_redirects=10, max_response_bytes, retries (≤ plan), debug=false }`.115  Response 200: `{ request_id, success, status, url, final_url, content, content_type, headers, cookies:[{name,value,domain,path}],116  metadata: { network, country, attempts, duration_ms, bytes, session, cached, timing:{dns_ms,proxy_connect_ms,tls_ms,origin_ms,processing_ms,total_ms},117  debug?: { attempts:[{provider: "network-a"…, network, country, outcome, status, duration_ms}] } }, text? (format=text), json? (format=json) }`.118  A blocked target still returns 200 with `success:false`, `status` (e.g. 403) and `metadata.attempts` > 1.119- `POST /v1/sessions` `{ country?, region?, city?, network?="auto", ttl?=600 (60–1800 s), label? }` → `{ id:"sess_…", status, network, country,120  request_count, last_used_at, expires_at, created_at }`; `GET /v1/sessions`, `GET /v1/sessions/:id`, `DELETE /v1/sessions/:id`. `Idempotency-Key` header supported on create.121- `GET /v1/me` → `{ project, organization:{plan}, key:{scopes} }`; `GET /v1/usage` → monthly usage summary.122- `GET /health`, `GET /ready` (public, at `https://www.fetcha.co/api/health` and `/api/ready`).123- Errors: `{ "error": { "code", "message", "request_id", "details"? } }`. Codes: INVALID_API_KEY 401, EMAIL_NOT_VERIFIED 403, RATE_LIMITED 429124  (+ `Retry-After`), CONCURRENCY_LIMIT 429, INVALID_REQUEST 400, URL_NOT_ALLOWED 400, TARGET_TIMEOUT 504, TARGET_BLOCKED 502,125  TARGET_UNAVAILABLE 502, PROVIDER_UNAVAILABLE 503, NETWORK_UNAVAILABLE 400, BROWSER_UNAVAILABLE 400, BROWSER_TIMEOUT 504,126  RESPONSE_TOO_LARGE 502, TOO_MANY_REDIRECTS 502, INSUFFICIENT_CREDITS 402, USAGE_LIMIT_REACHED 402, SESSION_NOT_FOUND 404,127  SESSION_EXPIRED 410, NOT_FOUND 404, FORBIDDEN 403, INTERNAL_ERROR 500.128- Plans (from PLAN_LIMITS): Free $0 — 1,000 req/mo, 5 concurrent, 30 s timeout, 2 retries, 3-day logs, 0.5 GB residential;129  Developer $29 — 50k req, 25 concurrent, 60 s, 7 days, 5 GB incl., $0.60/1k overage, $9/GB residential;130  Growth $149 — 500k req, 100 concurrent, 90 s, 30 days, 30 GB, $0.40/1k, $7.50/GB;131  Business $599 — 5M req, 500 concurrent, 120 s, 90 days, 150 GB, $0.25/1k, $6/GB; Enterprise custom.132  Billing/checkout is NOT live yet: pricing page shows plans with "Contact sales" / "Start free"; dashboard Billing says upgrades open soon (email sales@fetcha.co).133- SDKs: `@fetcha/sdk` (JS/TS: `new Fetcha({ apiKey }).fetch({ url, country })`, `.text(url)`, `.json(url)`, `.sessions.create()`)134  and `fetcha` (Python: `Fetcha(api_key).fetch(url, country="CA")`, `.text`, `.json`, `.sessions.create(...)`). Not yet published to135  npm/PyPI — docs must say "install from source (repo `packages/sdk`, `sdk-python`) until the packages are published". CLI: not available yet.136- Routing intelligence ("auto" mode): weighted score (35 % historical success, 20 % cost, 15 % latency, 15 % network health,137  10 % geography, 5 % session stability), per-domain profiles, circuit breakers, escalation cheap→premium, retries with new IP /138  alternate network. Explain this in marketing/docs without naming providers.139- SSRF policy: only http/https; localhost, private, link-local, metadata and internal hosts blocked; every redirect validated.140- Retention: request metadata kept per plan; response bodies not stored by default; sensitive headers redacted in logs.141