Repository Public
1# Testing23Four suites. All of them run on a developer machine with PostgreSQL 17 and the Playwright browsers installed.45| Suite | Command | What it exercises | Needs |6|---|---|---|---|7| Unit | `pnpm test:unit` | Schema, patches, migrations, renderer (every block type, three modes), diagram layout, chart scales, SVG sanitiser, simulations (maths and rendering), templates (FR and EN), editor forms and diagram operations, quality report, outbound-address rules, pricing | nothing |8| Integration | `pnpm test` (`scripts/test.sh`) | A production build, its worker and a real database, over HTTP: passkeys, sessions, CSRF, lock-out, recovery, CMS, draft isolation, uploads, Claude pipeline, exports, audit, restart, secret scan, backup → restore | PostgreSQL, Chromium |9| Browser | `pnpm test:e2e` (`scripts/e2e.sh`) | The studio journey in Chromium with a virtual passkey authenticator; the studio at 390 px; the public site at seven widths in Chromium and in WebKit with an iPhone profile | PostgreSQL, Chromium, WebKit |10| Real API | `REAL_AI=1 scripts/test.sh` | The same build against the real Anthropic API: one generation, one edit | `ANTHROPIC_API_KEY` in `.env`; costs money |1112In the integration and browser suites the Anthropic API is replaced by `tests/fake-anthropic.ts`, a local server that speaks the streaming protocol and returns deterministic JSON (including deliberately wrong answers: invalid JSON, out-of-scope patches, a refusal, a document that claims its sources are verified). Production never loads it. Nothing else is mocked: the database, the worker, the queue, the headless browser and the file storage are the real ones.1314## Results of the run of 2026-11-17 (laptop, macOS, Node 26.13, PostgreSQL 17.9)1516| Suite | Result |17|---|---|18| Unit | 711 passed, 0 failed (17 files) |19| Integration — `tests/api.test.ts` | 53 passed, 0 failed |20| Integration — restart (`tests/restart.test.ts`) | 4 passed: publications served, sessions kept, a job left "running" by a dead worker re-run to completion, worker heartbeat |21| Secret scan | client bundle and both process logs contain neither the API key nor `APP_SECRET`; `scripts/secrets-scan.ts` clean on 255 files |22| Backup → restore | encrypted dump and storage archive written, read back, restored into an empty database and directory; row counts equal on six tables; every restored file matches its recorded SHA-256; an archive with one byte appended is refused |23| Browser | 27 passed, 0 failed (studio 13, studio on a phone 2, public pages 6 × 2 browsers) |24| Real API | 2 passed — see below |25| Lint, types | `eslint` and `tsc --noEmit` clean |2627### Real Claude run (model `claude-opus-5-5`, generation effort `high`)2829- Prompt: a short sheet on the exponential moving average with an equation, a diagram and a quiz.30- Generation: accepted on the first answer (no repair round), 12 blocks (heading, callout, paragraph, equation, diagram, cards, quiz, references), 83 s, 9 885 output tokens, **0.34 USD** (of which 0.15 USD is the first write of the system prompt to the prompt cache).31- Edit ("add a one-sentence definition box right after the title, touch nothing else"): one block added, none changed or removed, 4.7 s, **0.04 USD** (29 164 tokens read from the cache).32- Every source in the generated sheet was stored as `unverified`, as designed.3334### Production checks (https://sheet.spboucher.ai, 2026-11-17)3536- Public browser suite against the production address: 12 passed (6 in Chromium at seven widths, 6 in WebKit with an iPhone profile).37- Lighthouse 13, mobile profile, lab data, one run per page:3839| Page | Performance | Accessibility | Best practices | SEO | LCP | CLS |40|---|---|---|---|---|---|---|41| `/` | 100 | 100 | 100 | 100 | 1.0 s | 0.004 |42| `/s/melange-experts-routage-top-k` | 99 | 100 | 96 | 100 | 1.1 s | 0 |43| `/search?q=garch` | 100 | 100 | 100 | 63 | 0.8 s | 0 |4445 The search page is `noindex` on purpose, which is what lowers its SEO score. On the sheet page, "best practices" loses points for missing source maps and for a Content-Security-Policy note (the policy keeps `'unsafe-inline'`, see docs/SECURITY.md). The first measurement found two accessibility defects (contrast of secondary and accent text, a button label that did not start with its visible text) and a heading-order gap on list pages; all three were fixed and re-measured.4647### What the integration suite asserts (summary)4849- **Anonymous access**: 401 on every `/api/admin/*` and credential route (18 GET, 14 write routes); studio pages redirect to sign-in and are `noindex`; security headers present.50- **Setup**: registration refused from another origin, another RP ID or without user verification; a challenge cannot be used by another browser; replay refused; ten recovery codes, stored hashed; the setup token is single-use; bootstrap closed afterwards.51- **Sign-in**: bad signature, unknown credential, wrong origin, missing user verification and replay all refused; session rotated at each sign-in and revoked at sign-out; CSRF header and same-origin enforced; progressive lock-out per address.52- **Passkey management**: fresh verification required to add or remove; revoked passkey cannot sign in; the last passkey cannot be removed.53- **Recovery**: a code opens a session that can only enrol a passkey; doing so revokes earlier passkeys, sessions and codes.54- **CMS**: optimistic locking (409), schema validation (422) including unknown block types, unknown fields and another author name; revisions and revert; publication refused on quality errors; atomic publish; published copy unchanged while the draft moves on; slug change answers 308 from the old address; unpublish removes the sheet from page, API, sitemap and feeds.55- **Draft isolation**: no draft slug appears in the public API, search, sitemap, RSS, JSON feed or home page; preview links work only with their token, are `noindex`, and die when revoked.56- **Injected markup**: `<script>`, `onerror=` and `javascript:` links typed into a block are shown as text in the public HTML.57- **Uploads**: type decided from bytes; HTML, executables, broken JSON and oversized files refused; a hostile SVG is stored without script, handlers, foreign content or external references and is served as a sandboxed attachment; CSV becomes a typed dataset; images stay private until a published sheet uses them.58- **Claude pipeline**: brief validation; attachment handed over as data with its embedded instructions reported; idempotent job creation; author, locale and source status forced by the server; image blocks and assets from the model dropped; repair round on invalid output; clear failure when output never validates or the model declines; private addresses refused as source links; cancellation; patch-based edit with byte-identical untouched blocks, undo by revert; selection scope and locked blocks enforced; no-op and stale-version handling; translation with identical skeleton and sources; budget stop; usage ledger amounts.59- **Exports**: PDF with embedded fonts; PNG at 1× and 2× (width doubles); standalone SVG with embedded fonts and no unresolved CSS variable; JSON export re-imported without loss; reader PDF offered on the public page.60- **Audit**: sign-ins, failures, recovery, revocations, publications, uploads, settings and jobs are recorded; no secret appears in the trail.6162## Not covered by automated tests6364- **Face ID / Touch ID on real devices.** The browser suite uses Chromium's virtual platform authenticator (user verification on, and once deliberately off). Enrolment and sign-in on a real iPhone (Safari) and a real Mac (Safari, Touch ID) must be done by hand once after deployment; Firefox was not run.65- **WebKit for the studio.** WebKit has no virtual authenticator, so only the public pages run in WebKit (iPhone profile). The studio on iPhone Safari is therefore untested beyond layout at 390 px in Chromium.66- **VoiceOver.** Accessibility is checked structurally (roles, labels, headings, focus, touch-target sizes), not with a screen reader.67- **Field Core Web Vitals.** Only lab measurements exist (below); there is no real-user data yet.68- **ClamAV.** The scanner protocol code is not exercised (no clamd in the test environment).69- **Reference checking against the network** is not part of the suites (it needs the internet); it was run by hand on 2026-11-17: a real DOI with its real title resolved, the same DOI with an invented title was reported as registered under a different title, an invented DOI as non-existent, an arXiv URL as existing.70- **Docker Compose files** in `infra/` were not built or run.71- **Pinch-zoom and drag gestures** in the diagram viewer and the visual diagram editor: the underlying operations are unit-tested and the editor opens in the browser suite, but multi-touch gestures were not driven.72