Verification report
All results below were produced on 2026-11-02 against the production build (pnpm build + next start, port 8540) on the operator laptop (macOS 27, Node 25.9, PostgreSQL 17.9 + pgvector 0.8.2), commit 46938d6 plus the fixes committed as the deployment commit. Nothing marked as passed was skipped. Time zone America/Toronto.
Content (import of content/ into the dev database)
| Metric | Value |
|---|---|
| Public records published | 353 (target ≥ 150) |
| In-depth (deep) records | 92 (target ≥ 50) — 100–300-word original descriptions, ≥ 3 sources, ≥ 2 alternatives, comparison notes |
| Private owner-platform records | 6 (MacLustr, AKOS, PolyLLM, SPB Git, FinData OS, UQO Research DB) — never public |
| Vendors | 300+ (one per publisher) · Categories: 8 disciplines / 71 leaves |
| Alternative links | 1,249 edges · Distinct sources cited: 1,268 · Collections published: 7 · Owner notes: 54 · Projects: 7 |
| Evidence confidence (published) | high 319 · medium 34 · low 0 |
Validator (pnpm content:validate) |
records 359 — 0 errors, 19 warnings (18 commercial records without a public pricing page, 1 shared homepage between two private records) |
| Duplicate audit | /admin/duplicates candidate pairs reviewed: name-similar pairs are distinct products (e.g. mlx/mlx-lm, claude/claude-code, openai-platform/openai-codex); 0 merges needed |
Command: pnpm content:validate && pnpm content:import --publish.
Link-validation audit (every URL field of every published record, safeFetch, UA declared)
| State | Count |
|---|---|
| ok (all URLs 2xx, or reachable but bot-protected = unverifiable) | 347 |
| redirected (official link now lands on another host, still live) | 6 |
| broken | 0 (apple-notes and automator were flagged 404 until the checker stopped sending Accept-Language, which made Apple's geo-redirect return 404; fixed and re-checked) |
| unchecked | 0 |
| Unverifiable URLs (401/403/405/429, self-redirect loops, stalls) | 25 URL fields across ~20 records (openai.com, chatgpt.com, claude.ai, godaddy.com, tableau.com, mathworks.com, realtor.ca, moodle.org, microsoft.com, milvus.io, …) — listed in Admin → Links |
Commands: worker link_sweep (342 jobs succeeded, 0 dead) + pnpm links:check --slugs … for the remainder. Duration ≈ 25 min for 353 records. |
Automated tests
| Suite | Command | Result |
|---|---|---|
| Unit + integration (vitest) | pnpm test |
25 passed / 25 (URL normalization, SSRF ranges, HTML meta, zod schema, quality score, markdown sanitizing; DB: import idempotency + revisions, public view hides private/unpublished, trigram typo search, reader-role boundary, append-only audit trigger, one-primary-category constraint, job queue dedupe/claim/backoff/dead-letter/retry/reap, argon2id + policy, login limiter) |
| End-to-end desktop + iPhone 15 (Playwright/Chromium) | E2E_OWNER_EMAIL=… E2E_OWNER_PASSWORD=… pnpm test:e2e |
22 passed / 22 (home → autocomplete → tool page; directory query/filter/sort/share URL; empty state; tool page sources/facts/compare/JSON-LD; compare tables; categories & collections; keyboard / + skip link; robots/sitemap/health; no-JavaScript SSR fallback; error report; admin login → editor save round trip → review/links/jobs/owner/audit; no owner-note leakage). Admin flows run on the desktop project only (they mutate shared records). |
| Security (black-box) | pnpm test:security |
44 passed / 44: unauthenticated admin pages redirect, 6 admin/auth APIs → 401, 5 mutating APIs → 401/403, cross-site login blocked (403), SQL/XSS/hostile payloads in search & suggest, no private fields in public HTML/JSON, sitemap excludes private paths, CSP/frame-ancestors/nosniff/referrer/no-powered-by, admin noindex, report size/honeypot (400 or 429 when rate-limited), IDOR by slug → 401, path traversal → 404 |
| Accessibility (axe-core, WCAG 2.0/2.1/2.2 A+AA + best-practice) | pnpm test:a11y |
0 serious/critical violations on 10 pages × 2 devices (fixed during hardening: tertiary-text contrast, aria-pressed on links, heading order, duplicate search landmarks, non-focusable scroll region) |
| Search ranking | pnpm eval:ranking (30 query→slug cases) |
MRR 0.756, hit@1 0.60, hit@5 0.967 (threshold 0.70) — lexical only; semantic fusion inactive without a provider key |
| Type check | pnpm typecheck |
clean |
Performance baseline (production build, laptop, warm, n = 15 sequential requests)
| Route | P50 ms | P95 ms | HTML bytes |
|---|---|---|---|
/ (ISR 5 min) |
2 | 6 | 148 k |
/tools (dynamic) |
13 | 16 | 315 k |
/tools?q=vector database |
17 | 22 | 256 k |
/tools?category=ai&source=open-source |
10 | 14 | 262 k |
/tools/duckdb (SSG) |
2 | 2 | 156 k |
/categories/analytics-engines |
7 | 9 | 107 k |
/compare?tools=duckdb,postgresql,sqlite |
5 | 7 | 70 k |
/api/suggest?q=post |
6 | 9 | 1.7 k |
| Client JS: 868 KB total chunks, 9 scripts on a tool page (largest 229 KB uncompressed, Next runtime). Budget: keep first-party client JS < 150 KB gzipped; no third-party scripts. Core Web Vitals not yet measured with Lighthouse (not installed on the laptop) — see roadmap. |
Backup / restore drill
pnpm restore:drill → encrypted dump created (AES-256-CBC, PBKDF2 200k), restored into tools_restore_drill_<ts>, compared with the source: tools 359 = 359, tool_sources 1282 = 1282, audit_log 722 = 722, owner_notes 54 = 54, content-hash digest identical → identical: true (1 s). Temporary database dropped.
Editorial spot checks (manual, during review)
- 12 records read against their official sites (duckdb, claude-code, postgresql, zotero, firstrate-data, sec-edgar, caddy, wireguard, qgis, mlx, homebrew, openalex): names, vendors, license identifiers and hosting model consistent with sources; descriptions original.
- Renames caught by research and recorded with aliases: Polygon.io → Massive, Timescale → Tiger Data, MinIO → AIStor, Kùzu archived (discontinued), AutoGen deprecated, Weights & Biases under CoreWeave, Langfuse acquired by ClickHouse, Neon under Databricks, OpenSSH
.org, Podman repo move. - Language proofreading: validator bans marketing adjectives and price figures; summaries 60–220 characters.
Gates not run / limitations
- Lighthouse/Core Web Vitals: not run (tool unavailable); latency table above is the baseline.
- Dependency audit:
pnpm audit --prod --audit-level highis part ofscripts/ci.sh; run before each release (result recorded in the deployment section below). - E2E for passkeys requires a real authenticator; covered by code review and the unit-tested WebAuthn library only.
Production deployment verification (filled in Phase 6)
See the bottom of this file.