SPB Git
spboucher.ai

Repository Public MIT

spbzyquo-cloud-web

Commit

phase7: verification report — 169/169 models streaming live, 12/12 providers non-streaming+cancel, vision/citations/reasoning, security audit, cross-browser 15/15

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Simon-Pierre Boucher committed · Jul 31, 2026 parent cecb24f
2 changed files +104 −1 Browse files
2 changed files
Unified Split
modified docs/PLAN.md +15 −1
@@ -147,6 +147,20 @@ up to four independently streaming columns with promote-to-thread; math and
147 diagrams render (lazily); reading aloud, dictation, auto-trim, and conversation 147 diagrams render (lazily); reading aloud, dictation, auto-trim, and conversation
148 tools round out the spec. Sweep clean, `npm run check` green. 148 tools round out the spec. Sweep clean, `npm run check` green.
149 149
150−## Phase 7 — Verification — pending 150+## Phase 7 — Verification
151+
152+- [x] Provider matrix with real keys from a real browser: **169/169 tested models PASS streaming** (sonar-deep-research skipped by design); 12/12 non-streaming; 12/12 Stop/abort; vision on all 9 vision-capable providers; Perplexity citations (19); reasoning deltas on all thinking models → `docs/VERIFICATION.md`
153+- [x] Persistence: Phase 3 gate 13/13 (reload + browser restart + export/import + vault)
154+- [x] Security/privacy: network audit = 12 provider origins + localhost only; CSP blocks foreign origins; first-run notice; no dev harness in prod build; keys never committed
155+- [x] Cross-browser/responsive: 15/15 on Chromium + WebKit + Firefox, desktop + phone, light default
156+
157+**Phase 7 summary.** The whole catalog was exercised live from a browser
158+context using the user's keys: every one of the 169 tested models streamed
159+token-by-token through the ported provider layer, with non-streaming,
160+cancellation, vision, reasoning, and citations verified per provider, and the
161+network audit confirming not a single byte leaves the page except toward the
162+chosen providers. Three first-pass failures were all test-harness artifacts
163+(tiny token budgets on reasoning models, an abort timer slower than Cerebras)
164+and retested green.
151 165
152 ## Phase 8 — Build & deployment — pending 166 ## Phase 8 — Build & deployment — pending
added docs/VERIFICATION.md +89 −0
@@ -0,0 +1,89 @@
1+<!--
2+ VERIFICATION.md
3+ Zyquo Cloud Web
4+
5+ Author: Simon-Pierre Boucher
6+ Mail: contact@spboucher.ai
7+-->
8+
9+# Phase 7 — Verification Report (real keys, real browser)
10+
11+Run 2026-07-31 against the dev build with the user's real API keys, from an
12+actual Chromium page driving the app's own provider layer
13+(`scripts/verify-matrix.mjs`, `scripts/verify-browsers.mjs`,
14+`scripts/gate-phase2.mjs`, `scripts/gate-phase3.mjs`). Raw per-model JSON:
15+`/tmp/zyquo-verify.json` (not committed — contains timings only, no keys).
16+
17+## 1. Provider / model matrix (streaming, token-by-token)
18+
19+**169 of 170 catalog models verified streaming live — 169 PASS.**
20+1 model deliberately skipped: `sonar-deep-research` (long-running, per-search
21+billing; its sibling `sonar-reasoning-pro` covers the code path).
22+
23+| Provider | Models | Streaming | Non-streaming | Stop/abort | Vision | Reasoning deltas | Citations | CORS |
24+|---|---|---|---|---|---|---|---|---|
25+| OpenAI | 27 | 27/27 ✅ | ✅ | ✅ | ✅ gpt-5-nano | n/a (API omits) | — | direct ✅ |
26+| Anthropic | 11 | 11/11 ✅ | ✅ | ✅ | ✅ haiku-4.5 | n/a (thinking off by default) | — | direct + header ✅ |
27+| xAI | 5 | 5/5 ✅ | ✅ | ✅ | ✅ | ✅ (grok-4.x) | — | direct ✅ |
28+| Mistral | 10 | 10/10 ✅ | ✅ | ✅ | ✅ | ✅ (magistral array-content quirk) | — | direct ✅ |
29+| Gemini | 14 | 14/14 ✅ | ✅ | ✅ | ✅ | n/a | — | direct ✅ |
30+| Qwen/DashScope | 32 | 32/32 ✅ | ✅ | ✅ | ✅ | ✅ (enable_thinking) | — | direct ✅ |
31+| DeepSeek | 2 | 2/2 ✅ | ✅ | ✅ | (no vision models) | ✅ | — | direct ✅ |
32+| Kimi | 12 | 12/12 ✅ | ✅ | ✅ | ✅ | ✅ (K-series) | — | direct ✅ |
33+| Perplexity | 4 | 3/3 ✅ + 1 skip | ✅ | ✅ | (no vision models) | ✅ | ✅ 19 citations | direct ✅ |
34+| Together | 16 | 16/16 ✅ | ✅ ¹ | ✅ | (no vision models) | ✅ | — | direct ✅ |
35+| DeepInfra | 34 | 34/34 ✅ | ✅ | ✅ | ✅ | ✅ | — | direct ✅ |
36+| Cerebras | 3 | 3/3 ✅ | ✅ ² | ✅ ² | ✅ gemma-4-31b | ✅ | — | direct ✅ |
37+
38+¹ Together's first non-streaming probe used `Qwen/Qwen3.5-9B` (a
39+`requiresStreaming` model whose tiny budget went entirely to reasoning);
40+retested green on `Llama-3.3-70B-Instruct-Turbo`.
41+² Cerebras first probes used a 16-token budget on a reasoning model (empty
42+text) and an abort timer slower than Cerebras's ~3000 tok/s (essay finished
43+first). Retested green with a 512-token budget and a 120 ms abort:
44+`Generation stopped.`
45+
46+Notable observations recorded:
47+- Reasoning content arrives via `reasoning_content`/`reasoning` deltas and,
48+ for Mistral's magistral, via the array-content thinking chunks — all
49+ normalized by the ported parser (verified: 103–1291 reasoning chars on
50+ thinking models).
51+- Legacy models (gpt-3.5-turbo, moonshot-v1 family, o1…) all still served.
52+- Perplexity `sonar` returned 19 numbered citations, merged from
53+ `citations` + `search_results`.
54+
55+## 2. Persistence
56+
57+Phase 3 gate (`scripts/gate-phase3.mjs`), real Chromium with a persistent
58+profile: **13/13 PASS** — keys/settings/conversations survive reload **and a
59+full browser restart**; export→clear→import restores exactly; passphrase
60+vault encrypts at rest (AES-GCM), rejects wrong passphrase, unlocks, disables.
61+
62+## 3. Security / privacy sanity
63+
64+- **Network audit:** every origin contacted during the full matrix run:
65+ `localhost` + exactly the 12 provider API origins. Nothing else — no
66+ analytics, no third parties.
67+- **CSP:** `fetch('https://example.com')` from the page → blocked by
68+ `connect-src`; provider origins reachable. No inline/third-party scripts.
69+- **First-run notice** present and honest (localStorage, BYOK, passphrase
70+ option).
71+- **Prod build:** dev harness (`window.__zyquo`) absent; keys never logged;
72+ keys file `.keys.local.json` gitignored (verified never committed).
73+
74+## 4. Responsive / cross-browser
75+
76+`scripts/verify-browsers.mjs`: **15/15 PASS** across **Chromium, WebKit
77+(Safari engine), Firefox** — app loads, **light theme is the default** (bg
78+`rgb(250,251,253)`), real streaming completion renders in every engine, no
79+horizontal overflow at 1240×800 or 390×844, drawer navigation on phone.
80+Visual review at both widths in light + dark: `docs/PLAN.md` Phase 4.4.
81+
82+## 5. Phase gates recap
83+
84+- Phase 2 gate: token-by-token streaming in a real browser (26 incremental
85+ snapshots) + cancellation — PASS.
86+- Phase 3 gate: 13/13 persistence checks — PASS.
87+- Phase 7 matrix: 169/169 tested models streaming, 12/12 providers
88+ non-streaming + cancel, vision on all 9 providers that have vision models,
89+ citations, CORS all-direct — PASS.
90
Menu

Theme
↑↓navigate ↵open escclose /⌘Kanywhere