spb/autoscholar

1# Test report — AutoScholar.ai23Updated 2026-10-30. Every line below was actually executed; results are copied, not assumed. Live-run sections are appended as runs complete.45## Automated suites (laptop, 2026-10-30)67| Suite | Command | Result |8|---|---|---|9| Lint/format | `pnpm lint` (Biome 2.6) | clean (2 infos) |10| Typecheck | `pnpm -r typecheck`, `apps/web tsc --noEmit` | 0 errors |11| Unit (22 tests) | `pnpm vitest run --project unit` | 22 passed: storage write-once/digest/traversal; auth Argon2id/CSRF/limiter/lockout; instruction validation; artifact contract/promotion/human-review rules; BibTeX/cite parsing; reported-value tracing; **policy hook** (sudo/ssh/launchctl/pm2/brew/LAN curl/credential paths/dangerouslyDisableSandbox denied; research commands allowed; writes confined incl. symlink escape and frozen paths; reviewer write scope; SSRF targets for WebFetch; sensitive reads; unknown tools) |12| Integration (12 tests, PostgreSQL `autoscholar_test`) | `AUTOSCHOLAR_INTEGRATION=1 pnpm vitest run --project integration` | 12 passed: bootstrap admin123 → forced rotation → reuse rejected → old session revoked; lockout after 5 failures; upload hashing/versioning/dedup/rejection; 3 concurrent launches with one idempotency key → one run; two workers cannot lease the same run; stale fencing tokens rejected; cooperative cancel; worker crash → lease expiry → retry resumes from checkpoint with higher fencing token → attempt limit → failed → admin retry; heartbeat revokes foreign leases; no dispatch to workers without sandbox; disabled worker rejected; simulated run → `quality_failed` with `worker.real` hard failure, never publishable, events redacted; scheduler runtime cap idempotent; concurrent publication → single `AS-WP-2026-0001`, idempotent re-publish, private `input/CLAUDE.md` and `code/.env` absent from the public bundle |13| E2E (25 tests, Playwright + axe) | `pnpm test:e2e` | 25 passed (public pages a11y wcag2a/aa on 11 pages; responsive screenshots 390/820/1440; dark mode; keyboard skip link; search/filters/pagination/empty state; paper page + PDF route 200 with digest header; traversal/unlisted 404/400; citation routes; JSON-LD; no private fields in public API; feeds/sitemap/robots/security headers; 14 GET + 15 POST admin routes 401 signed-out; wrong password 401; bootstrap rotation flow; CSRF 403; XSS sanitized; oversize 413; platform-targeting and `.exe` uploads 400; idempotent launch 201→200; live timeline; sign-out) |1415## Production checks (2026-10-30)1617| Check | Result |18|---|---|19| Control plane deployed via `mld deploy autoscholar --node M3U96b` | pm2 `autoscholar`, `autoscholar-scheduler` online; local health 200; tunnel route `autoscholar.ai → M3U96b:8470` created on BHS64 |20| Public HTTPS | `www.autoscholar.ai` A record added by the founder in the afternoon → canonical host switched to www, tunnel route re-pointed (`www.autoscholar.ai → M3U96b:8470`, apex redirect prepared). Verified over the public gateway: 8 public routes 200; HSTS/CSP/nosniff/X-Frame headers present; `/api/internal/*` through the gateway → 403; admin login over HTTPS (Secure cookie) → session 200, admin page 200, mutation without CSRF 403, logout 200; staging sample absent (`total: 0`, 404 on its page). Apex `autoscholar.ai` still has no A record (redirect inactive). |21| Admin bootstrap in production | login `admin123` → `mustChangePassword: true`; privileged call before rotation → 403; rotation accepted; `admin123` → 401 afterwards; new password login ok (`mustChangePassword: false`). Executed from M1M32 over the LAN with an explicit cookie header. |22| Worker install (M4M64c) | user `asworker` (uid 502, non-admin, hidden), LaunchDaemon running, `/etc/autoscholar/worker.env` root:asworker 0640, pf anchor loaded (LAN/Tailscale/CGNAT blocked for uid 502 except 192.168.2.85:8470 and DNS), tectonic bundle cached, enrolled (`compute_workers`: online, sandboxAvailable=true, Tectonic 0.17.0) |23| Worker self-test (`node dist/main.js selftest`, as asworker) | `ok: true` — CLAUDE.md passphrase recalled and written to `results/passphrase.txt` (instruction loading proven); denials recorded by enforcement: `Write: write outside the run workspace`, `Bash: LAN/loopback network access`, `ToolSearch` (then allowed); Seatbelt blocked `cp … /tmp/passphrase-backup.txt` (`Operation not permitted`); no write outside the workspace observed; LaTeX class compiled (2 pages); cost USD 0.13. With the production policy prompt the model declines the same steps on its own; enforcement was verified with the policy prompt removed. |24| Backup + restore | `scripts/backup.sh` → pg_dump + rsync to `M3U96a:~/backups/autoscholar` (dumps listed offsite); `scripts/restore-test.sh` restored into `autoscholar_restore`, counted rows, verified artifact digests, dropped the DB |25| Fault found and fixed during the first live run | second upload of a changed artifact within one attempt collided with the write-once store → HTTP 500 → runner reported a failed attempt → scheduler retried from the preflight checkpoint on attempt 2 (recovery path worked as designed). Fix: per-version storage keys; redeployed with `scripts/deploy-control-plane.sh` (≈5 s gap) |2627## Production smoke, load and accessibility (2026-10-30, over the LAN)2829- All 15 public routes return 200 (home, research, topics, series, about, methodology, infrastructure, transparency, 3 legal pages, feed.xml, sitemap.xml, robots.txt) and an unknown paper returns 404; `robots.txt` disallows `/admin` and `/api`.30- All 12 admin pages return 200 with a valid session; signed-out admin pages redirect (307) to login; admin APIs return 401.31- Load (20 concurrent clients, 15 s, mixed public pages/APIs from M1M32): 14 683 requests, 979 req/s, p50 17 ms, p95 39 ms, p99 47 ms, max 236 ms; the only errors were HTTP 429 from the public rate limiter (600/min per address) — confirmed by a second 5 s burst returning 100 % 429.32- Accessibility: axe-core wcag2a/aa on 11 public pages in the e2e suite (0 violations); keyboard skip link and reduced-motion verified.33- Dependency audit: `pnpm audit --prod` → 0 known vulnerabilities after upgrading pdfjs-dist to 6.4 (GHSA-hq66-cqwq-w95j).3435## Live research runs3637Recorded below as they complete (run IDs, stages, cost, QA outcome, publication).38