# Workstream E — Usage analytics, provider management, custom endpoints

Status: complete for this phase. `pnpm typecheck`, `pnpm lint` (0 errors) and `pnpm test` (159 tests, 28 new) are green.
No dev server was started; no commit was made. One additive migration was generated **and applied locally**
(`drizzle/0002_endpoints.sql`) — it must run on prod via the mld hook (`pnpm db:migrate`).

## 1. Files

### Created
| Area | File |
| --- | --- |
| Usage API | `src/lib/usage/time.ts` (tz-aware ranges/buckets, pure), `src/lib/usage/savings.ts` (savings math, pure), `src/app/api/usage/export.csv/route.ts` |
| Usage UI | `src/components/usage/usage-dashboard.tsx`, `lazy-charts.tsx`, `kpi-tiles.tsx`, `usage-filters.tsx`, `insight-cards.tsx`, `recent-activity.tsx`, `types.ts` |
| Providers | `src/components/providers/provider-row.tsx`, `src/app/api/providers/test/route.ts` |
| Custom endpoints | `src/lib/ai/providers/custom/index.ts`, `src/lib/endpoints/service.ts`, `src/lib/endpoints/ssrf.ts`, `src/app/api/endpoints/route.ts`, `src/app/api/endpoints/[id]/route.ts`, `src/app/api/endpoints/[id]/test/route.ts`, `src/app/api/endpoints/[id]/sync/route.ts`, `src/components/endpoints/endpoint-sheet.tsx`, `endpoint-row.tsx`, `presets.ts`, `src/app/app/settings/endpoints/page.tsx` |
| Settings | `src/components/settings/section.tsx` (`SettingsSection`, `SettingsGroup`, `SettingsRow`), `src/components/settings/sections.ts` (nav) |
| Schema | `drizzle/0002_endpoints.sql` (+ `drizzle/meta/0002_snapshot.json`, journal entry) |
| Tests | `tests/unit/usage-savings.test.ts`, `usage-time.test.ts`, `endpoints-ssrf.test.ts`, `custom-endpoints.test.ts` |

### Changed
| File | Change |
| --- | --- |
| `src/lib/ai/core/types.ts` | `"custom"` appended to `PROVIDER_IDS`; new `KEYED_PROVIDER_IDS` (everything except custom). `parseModelKey` unchanged (splits on first `/` → provider `custom`). |
| `src/lib/ai/providers/index.ts` | `ADAPTERS.custom = customPlaceholderAdapter` (fails loudly, never calls a server); `PROVIDER_META.custom`. |
| `src/lib/ai/providers/env-keys.ts` | `OWNER_KEY_ENV.custom = ""`; `ownerKey()` returns `undefined` for it. |
| `src/lib/ai/registry/catalog.ts` | `CATALOGS.custom = new Map()`. |
| `src/lib/client/providers.ts` | `PROVIDERS.custom` (color `--p-custom`, links to Settings → Endpoints). `PROVIDER_ORDER` unchanged (keyed providers only). |
| `src/components/brand/provider-icon.tsx` | `case "custom"` glyph (plug). |
| `src/components/marketing/mock-data.ts` | one-line `custom` entry to keep `Record<ProviderId,string>` exhaustive (F's file — exhaustiveness only). |
| `scripts/provider-matrix.ts` | `TEST_MODELS.custom` placeholder (skipped: no owner key). |
| `src/db/schema-workspace.ts` | `custom_endpoints.discovered_models jsonb` + `discovered_at` (discovery snapshot so `/api/models` never calls the endpoint). |
| `src/app/api/models/route.ts` | additive: appends `listCustomModels(userId)`; pushes `"custom"` into `connectedProviders` when ≥ 1 endpoint is `valid`. |
| `src/lib/usage/service.ts` | rewritten (see API). `UsageRange` kept as a deprecated alias. |
| `src/app/api/usage/route.ts` | uses `usageQueryFromUrl`. |
| `src/components/usage/charts.tsx` | dataviz pass (≤ 24 px bars, 4 px data-end radius, 2 px surface gaps, 10 % area wash, hairline grid, tooltips everywhere), new `LatencyByModel` (two small multiples — never a dual axis), `Sparkline`, `ChartEmpty`; labels parsed from wall-clock bucket keys. |
| `src/app/app/usage/page.tsx` | thin server page → `<UsageDashboard />` in `Suspense` (uses `useSearchParams`). |
| `src/components/providers/add-key-dialog.tsx` | `ResponsiveDialog` (sheet on phones), paste button (clipboard API when available), show/hide, whitespace check, latency in the toast. Public API (`AddKeyDialog`, `useAddKeyDialog`) unchanged. |
| `src/app/app/settings/providers/page.tsx` | redesign (rows, statuses, Test connection with latency, More menu / ActionSheet, link card to Endpoints). |
| `src/app/app/settings/layout.tsx`, `page.tsx` | phone = section list + 48 px back bar per section; desktop = side nav; "Endpoints" added. |
| `src/app/app/settings/{account,security,appearance,data}/page.tsx` | `SettingsSection` header, 44 px targets / 16 px inputs on phones, audit log → stacked rows below `md`, session actions full-width on phones, CSV usage export link on Data. |
| `src/lib/client/types.ts` | append-only re-exports (`PublicEndpoint`, `EndpointInput`, `ManualModel`, `UsageSummary`, `SavingsOpportunity`, …). |
| `.env.example` | `ALLOW_PRIVATE_ENDPOINTS=0` documented. |

## 2. API routes

### `GET /api/usage`
Query: `range=today|7d|30d|90d|custom|all` (default 30d) · `from`,`to` = `YYYY-MM-DD` (custom, inclusive, clamped to today and 366 days; ≤ 2 days → hourly) · `tz` = IANA zone (validated, default UTC; the client sends `Intl.DateTimeFormat().resolvedOptions().timeZone`) · `provider` · `modelKey` · `projectId` (join `usage_records.conversation_id → conversations.project_id`, scoped to the user).

Response `UsageSummary` (`src/lib/usage/service.ts`):
```ts
{
  range: { key, from: string|null, to: string, bucket: "hour"|"day", tz, days },
  filters: { provider, modelKey, projectId },
  kpis: { requests, failures, stopped, errorRate, inputTokens, outputTokens, cachedTokens, reasoningTokens, totalTokens,
          costUsd, unpricedRequests, avgLatencyMs, avgTtftMs|null, avgTokensPerSec|null, avgContextTokens|null },
  series: { bucket: "YYYY-MM-DD" | "YYYY-MM-DDTHH:00", requests, failures, inputTokens, outputTokens, costUsd }[], // gap-filled in tz
  byProvider: { provider, requests, failures, inputTokens, outputTokens, costUsd }[],
  byModel: { modelKey, provider, requests, failures, inputTokens, outputTokens, cachedTokens, reasoningTokens, costUsd,
             avgLatencyMs, avgTtftMs|null, tokensPerSec|null, avgContextTokens|null }[],
  projection: { dailyAverageCost, estimatedMonthlyCost /* daily avg × 30 */, observedDays,
                previous: { costUsd, requests } | null, costTrendPct|null, requestsTrendPct|null },
  savings: SavingsOpportunity[], // top 3, see below
  recent: RecentRecord[60] (+ conversationTitle, projectId via left join),
  facets: { providers[], models: {modelKey, provider, requests}[24], projects: {id,name,icon,color}[] },
  generatedAt
}
```
tok/s = `outputTokens / max(latency − ttft, 1 ms)` on successful streaming rows. Avg context = avg input tokens of successful requests.

**Savings** (`computeSavings`, pure, tested): for every model that cost money, candidates = same provider, not deprecated, priced, covering every capability the source has among {vision, tools, structuredOutput, reasoning, files}, context ≥ 1.1 × avg prompt; same `family` preferred. The real token volume (input, cached at cached rate, output) is re-priced with `estimateCost`; the **most capable sibling that still saves ≥ 40 %** wins (Opus → Sonnet, not Haiku). Min $0.01. Each row carries `headline` ("Switching eligible tasks from Claude Opus 4 to Claude Sonnet 4 could have saved ~$17.30") and a `rationale`.

### `GET /api/usage/export.csv` — same query params → `text/csv` attachment (`polyllm-usage-<from>_<to>.csv`, ≤ 50 000 rows, formula-injection-safe quoting, `X-Row-Count` header).

### `POST /api/providers/test` `{ provider }` → `{ ok, latencyMs, modelsAvailable|null, error?, errorCode?, connections }`
Validates the **stored encrypted key** with `validateApiKey` (never returns the key), refreshes the model list on success. Rate limit `keyValidate`. Rejects `custom`.

### Endpoints (all user-scoped, key material never returned; `PublicEndpoint` carries `keyHint`, `hasKey`, `headerNames` only)
| Method | Path | Body → Response |
| --- | --- | --- |
| GET | `/api/endpoints` | → `{ endpoints: PublicEndpoint[], allowPrivate }` |
| POST | `/api/endpoints` | `{ name, baseUrl, apiKey?, headers?: Record<string,string>, modelsPath? ("" = manual only), manualModels?: ManualModel[], validate?: boolean=true }` → 201 `{ endpoint, test?: { ok, latencyMs, modelsAvailable, error? } }` |
| GET | `/api/endpoints/[id]` | → `{ endpoint }` |
| PATCH | `/api/endpoints/[id]` | partial input; `apiKey: null` clears the key, `headers: null` clears headers; changing URL/key/headers/path resets status to `unverified` → `{ endpoint }` |
| DELETE | `/api/endpoints/[id]` | → `{ ok: true }` |
| POST | `/api/endpoints/[id]/test` | → `{ ok, latencyMs, modelsAvailable, error?, errorCode?, endpoint }` |
| POST | `/api/endpoints/[id]/sync` | → `{ ok, models: PolyModel[], latencyMs, error?, endpoint }` (discovery snapshot persisted) |

`ManualModel = { id, displayName?, contextTokens?, vision?, tools?, reasoning? }` — also acts as capability override for a discovered id.
Probe = `GET {baseUrl}{modelsPath}` (accepts `{data:[{id}]}`, bare arrays, Ollama `{models:[{name}]}`); with discovery off and manual models present → a 1-token chat completion against the first manual model.

Encryption: `encryptSecret(value, { userId, provider: "custom:<endpointId>" })` → AAD `"<userId>|custom:<endpointId>"` for both the key and the headers JSON. Audit actions: `endpoint.created|updated|deleted`.

SSRF guard (`src/lib/endpoints/ssrf.ts`): http/https only, no credentials in URL, blocks localhost/*.local/*.internal/single-label hosts, IPv4 private/loopback/link-local/CGNAT/test/multicast, IPv6 ::1/ULA/link-local/mapped-v4/NAT64, plus DNS resolution of the hostname at create/update/probe. Bypass only with `ALLOW_PRIVATE_ENDPOINTS=1`. DNS rebinding after the check is out of scope.

## 3. Exact integration hook for chat and Arena (integrator wires — A's and C's files were not edited)

`src/lib/endpoints/service.ts` exports
```ts
resolveCustomEndpoint(userId, modelKey): Promise<{ adapter: AIProviderAdapter; apiKey: string; model: PolyModel; endpoint: PublicEndpoint } | null>
isCustomModelKey(key): boolean
```
Custom keys look like `custom/cep_x:llama3.1:8b` → `getModel()` returns `null` for them (they are not in the registry table) and `getDecryptedKey(userId, "custom")` returns `null`. Wire both services like this:

**`src/lib/chat/service.ts` → `prepareTurn`** — replace
```ts
const model = await getModel(input.modelKey);
if (!model) throw new ApiError(404, "Unknown model. Refresh your model list.", "MODEL_NOT_FOUND");
const apiKey = await getDecryptedKey(ctx.userId, model.provider);
if (!apiKey) throw new ApiError(400, `No API key configured for ${model.provider}. …`, "NO_PROVIDER_KEY", { provider: model.provider });
```
with
```ts
const custom = isCustomModelKey(input.modelKey) ? await resolveCustomEndpoint(ctx.userId, input.modelKey) : null;
const model = custom?.model ?? (await getModel(input.modelKey));
if (!model) throw new ApiError(404, isCustomModelKey(input.modelKey) ? "This custom endpoint no longer exists. Check Settings → Endpoints." : "Unknown model. Refresh your model list.", "MODEL_NOT_FOUND");
const apiKey = custom?.apiKey ?? (await getDecryptedKey(ctx.userId, model.provider));
if (!apiKey) throw new ApiError(400, `No API key configured for ${model.provider}. Add one in Settings → Providers.`, "NO_PROVIDER_KEY", { provider: model.provider });
```
add `adapter?: AIProviderAdapter` to `PreparedTurn`, return `adapter: custom?.adapter` from every `return { conversation, model, apiKey, … }`, and in **`runTurn`** replace `const adapter = getAdapter(turn.model.provider);` with `const adapter = turn.adapter ?? getAdapter(turn.model.provider);`. Also skip `recordProviderOutcome` when `turn.model.provider === "custom"` (there is no `provider_connections` row; it is a no-op update anyway) and keep `touchRecent(userId, model.key)` as is (works with any key).

**`src/lib/arena/service.ts`** — `createArenaSession`: for each key, `const custom = isCustomModelKey(key) ? await resolveCustomEndpoint(userId, key) : null; const m = custom?.model ?? await getModel(key); … if (!custom && !(await getDecryptedKey(userId, m.provider))) missing.push(m.provider);`. `runArenaModel`: same three-line substitution as `prepareTurn`, then `const adapter = custom?.adapter ?? getAdapter(model.provider);`.

`apiKey` for key-less endpoints is the sentinel `NO_KEY_SENTINEL` ("polyllm-no-key") — the OpenAI SDK needs a non-empty bearer; Ollama & co. ignore it. `model.pricing` is `null` → `estimateCost().known === false` → `costUsd` stays `null` in `usage_records` (the dashboard counts them as "unpriced"). `usage_records.provider` will be `"custom"` (valid `ProviderId`).

Model picker (B): `/api/models` already returns custom models with `provider: "custom"`, `family` = endpoint name, `metadata.endpointId/endpointName/endpointStatus`; `connectedProviders` includes `"custom"` when at least one endpoint is valid. If B's category chips iterate `PROVIDER_ORDER`, custom models only appear in "All"/search — add a "Custom" chip keyed on `provider === "custom"` if desired.

## 4. Coming soon / known limits (nothing fake shipped)
- Custom endpoints have **no pricing** (never invented) — their requests show `—` cost and are counted in `kpis.unpricedRequests`. A per-endpoint price sheet is a candidate follow-up.
- Discovered custom models start **text-only**; users unlock vision/tools/reasoning per id through the manual-models editor. `structuredOutput`/`files` are always off for custom endpoints (server support varies too much to assume).
- Stored **header values are never shown**; editing any header value re-saves all headers (UI explains this).
- SSRF: DNS rebinding after the check and redirects to private hosts are not followed (`redirect: "manual"` on discovery; the OpenAI SDK follows redirects for chat — acceptable risk noted).
- Usage "all time" projection uses the first recorded request as the start of the observation window.
- No `Coming soon` badges were needed.

## 5. QA checklist (integration phase, single dev server)
Usage (`/app/usage`) at 375 / 390 / 430 / 1440:
- [ ] 48 px bar with hamburger, refresh, CSV icon on phones; `PageHeader` + buttons on desktop.
- [ ] Segmented Today/7d/30d/90d/Custom scrolls horizontally without page overflow; Custom opens a bottom sheet (dialog on desktop) with two `type=date` fields (16 px), presets, validation (end before start), Apply.
- [ ] Provider/model/project chips filter everything; "Filtered by …" pills clear individually; URL query updates (deep link reload keeps state).
- [ ] KPI tiles 2-col on phones, 4-col desktop; deltas coloured by direction (cost up = red).
- [ ] Charts lazy-load with skeletons; no default recharts colours; hour labels for Today, day labels otherwise, in the browser's zone; tooltips on every plot; stacked segments show 2 px surface gaps; failure segment only when failures exist.
- [ ] Cost by provider donut folds > 6 providers into "Other"; list shows provider icons + values.
- [ ] Projection card and Savings card (or the honest "nothing to save" state); "Why this sibling?" expands; catalog link works.
- [ ] Latency by model = two side-by-side small multiples (TTFT ascending, tok/s descending).
- [ ] Recent requests: rows on phones (tap → conversation), table from `md`; export CSV downloads with the current filters.
- [ ] Empty states: no data (with "Show all time"), filters with no match ("Clear filters"), API error with Retry. Refetch keeps the previous render at 70 % opacity (no skeleton flash).

Providers (`/app/settings/providers`):
- [ ] Rows sorted connected → failed → not connected; statuses Connected / Validation failed / Not validated yet / Not connected.
- [ ] Add key opens a bottom sheet on phones (dialog on desktop) with Paste (only where `clipboard.readText` exists), show/hide, docs link; success toast shows models + latency.
- [ ] Test connection shows latency inline (green/red) and toasts; failure shows the error under "Last error".
- [ ] More → Replace / Open console / Remove (ActionSheet on phones, dropdown on desktop); Remove uses ConfirmDialog.
- [ ] "Custom endpoints" card links to `/app/settings/endpoints` with counts.

Endpoints (`/app/settings/endpoints`):
- [ ] Empty state with preset chips; Add opens a full-height sheet on phones.
- [ ] Presets fill URL/path; the private-address warning appears for localhost/LAN URLs and explains tunnel vs `ALLOW_PRIVATE_ENDPOINTS=1` (info tone when the server allows private).
- [ ] Headers rows add/remove; manual models rows with vision/tools/reasoning switches; discovery toggle hides/shows models path.
- [ ] "Save & test connection" → latency + discovered model chips; failure keeps the sheet open with the error; "Save without testing" → status "Not tested".
- [ ] Row actions: Test, More → Edit / Refresh model list / Remove. Models appear in the picker as provider "Custom" (family = endpoint name) after `refreshModels`.
- [ ] Server rejects `http://localhost:11434/v1` with 400 `PRIVATE_HOST` when `ALLOW_PRIVATE_ENDPOINTS` is unset; accepts it when set to 1.

Settings shell:
- [ ] Phone `/app/settings` shows the six-section list (≥ 60 px rows); each section has a back chevron bar; desktop `/app/settings` redirects to Account and shows the side nav with "Endpoints".
- [ ] Account: audit log as stacked rows on phones, table on desktop; inputs 44 px / 16 px on phones. Security: session buttons full-width on phones. Appearance: switches have 44 px hit areas. Data: JSON export + usage CSV link.