| |
1 | +<!-- |
| |
2 | + AI-BROWSER-RESEARCH.md |
| |
3 | + Zyquo Atlas |
| |
4 | + Author: Simon-Pierre Boucher |
| |
5 | + Mail: contact@spboucher.ai |
| |
6 | +--> |
| |
7 | + |
| |
8 | +# Zyquo Atlas — AI Browser Research (Phase 0.A) |
| |
9 | + |
| |
10 | +Compiled 2026-07-30 from several intensive web-research sessions across the current |
| |
11 | +AI-browser landscape, page-extraction techniques, AI-UX patterns, streaming engineering, |
| |
12 | +and browser-AI security. This document is the **design contract for every AI feature** in |
| |
13 | +Phases 3 and 6 — each feature below traces to a pattern documented here. Sources are cited |
| |
14 | +inline; where a claim rested only on low-quality SEO content it is marked *[unverified]*. |
| |
15 | + |
| |
16 | +--- |
| |
17 | + |
| |
18 | +## 0. Category state (July 2026) and what it means for Atlas |
| |
19 | + |
| |
20 | +Three facts shape the design: |
| |
21 | + |
| |
22 | +1. **The AI-sidebar-only browser is not a business.** Arc went to maintenance mode (May |
| |
23 | + 2025; The Browser Company acquired by Atlassian, Oct 2025). OpenAI is **shutting down |
| |
24 | + ChatGPT Atlas on 2026-08-09**, concluding "the browser is a feature, not the |
| |
25 | + destination." SigmaOS — a native SwiftUI+WebKit browser, *exactly Atlas's stack* — is |
| |
26 | + effectively abandoned. → **The browser must be excellent as a browser first** (Atlas's |
| |
27 | + Phase 2 gate is correct); AI is a multiplier, not the product. |
| |
28 | + (Engadget, TechCrunch 2026-07-09, r/macapps.) |
| |
29 | +2. **Indirect prompt injection is the unsolved, category-defining security problem.** Brave |
| |
30 | + demonstrated exploits in Comet (twice), Opera Neon, and Fellou; a plain "summarize this |
| |
31 | + page" click exfiltrated a user's email + OTP in Comet. On-device models do **not** fix |
| |
32 | + it. → **Atlas is read-oriented, not agentic** — a defensible, safer position — but even |
| |
33 | + read-only AI is exploitable (§7). (brave.com/blog: comet-prompt-injection, |
| |
34 | + unseeable-prompt-injections, prompt-injection-flaw-opera-neon, indirect-prompt-injection.) |
| |
35 | +3. **Agentic is where vendors went; grounded reading is where users get value.** Every |
| |
36 | + agent-mode review reports impressive demos, unreliable multi-step execution, and heavy |
| |
37 | + fallback to the reliable features (summarize, chat-with-page, compare tabs). → **Atlas |
| |
38 | + invests in reliable reading tools + massive customization + BYO-key model choice**, the |
| |
39 | + three gaps left by the failed/paywalled incumbents. |
| |
40 | + |
| |
41 | +**Naming note:** OpenAI's retiring product was also "Atlas." Zyquo Atlas docs disambiguate |
| |
42 | +as *Zyquo Atlas* throughout (never "Atlas" alone in user-facing text). |
| |
43 | + |
| |
44 | +--- |
| |
45 | + |
| |
46 | +## 1. Landscape — functional patterns per product |
| |
47 | + |
| |
48 | +### Arc / Arc Max (The Browser Company) — "AI as small, keystroke-invoked utilities" |
| |
49 | +The most important reference: AI as invisible utilities, each one interaction with a |
| |
50 | +shortcut, output where you already are. |
| |
51 | +- **Ask on Page:** hold **⌘F** — the find bar becomes an ask box; answer inline. Overloads |
| |
52 | + an existing muscle-memory shortcut ("find" and "ask about this page" are the same intent |
| |
53 | + at different fidelities). **Adopt.** |
| |
54 | +- **5-Second Previews:** hover a link (Shift+hover off search pages) → floating popover |
| |
55 | + summarizing the *destination* page. |
| |
56 | +- **Tidy Tab Titles / Tidy Downloads / Tidy Tabs:** zero-UI AI applied to chores; users |
| |
57 | + don't perceive it as "AI." Tidy Tabs only touches ephemeral tabs, never pinned. |
| |
58 | +- **Failure evidence to engineer against:** "Ask on Page told me it encountered an error 3 |
| |
59 | + of 4 times"; "5-second previews don't give much beyond the page title." → **Extraction |
| |
60 | + fragility is the #1 cause of perceived AI failure.** (Arc Max help; HN 37753840.) |
| |
61 | + |
| |
62 | +### Dia (Browser Company/Atlassian) — "the URL bar is the chat box" |
| |
63 | +- **Intent-detecting address bar:** classifies input as navigate / search / ask, "mostly |
| |
64 | + based on length and syntax." Documented failure: "sometimes I'm trying to google |
| |
65 | + something and get thrown into an AI chat against my intention." Keyboard overrides: |
| |
66 | + Ctrl+⌘+Return forces chat, Shift+⌘+Return forces search. |
| |
67 | +- **Skills:** saved prompts invoked by **`/shortcut`** in chat, created via a |
| |
68 | + Natural-Language Skill Builder, shared in a public gallery. One power user built 50+ |
| |
69 | + (`/summary`, `/research`, `/newsletter`…). **Strongest signal in the landscape that |
| |
70 | + user-authored slash-prompts are the killer feature — and it's cheap.** |
| |
71 | +- **Chat-with-all-tabs** via `@tabname` / `@history` mentions (mentioned tabs render as a |
| |
72 | + stacked pile); **Quick Insert** pushes AI output into the current field. |
| |
73 | +- **Regressions users resent:** no fuzzy ⌘T command bar (replaced by chat), no Spaces, no |
| |
74 | + Boosts. "Power users miss the keyboard launcher more than the sidebar." → **Ship the |
| |
75 | + palette AND the ask, not one replacing the other.** (miskakyto.fi; SupaSidebar; TechCrunch.) |
| |
76 | + |
| |
77 | +### Perplexity Comet — the agentic sidebar, most-tested, most-exploited |
| |
78 | +- **Assistant sidebar** (⌘A); **one-click page summary** (icon next to Assistant); |
| |
79 | + **@tab** cross-tab context; agentic tasks with **blue frames on pages it touches** and a |
| |
80 | + visible step list. |
| |
81 | +- **Failure evidence:** `@tab` targeting "inconsistent or a bit buggy" (→ use a picker with |
| |
82 | + chips, not fuzzy text); the agent "only analyzed the snippet, not the full page, so it |
| |
83 | + missed the deeper answer" and a reviewer asked for "**a visible list of the pages it |
| |
84 | + reviewed**" (→ Atlas's citation requirement); long agent chains "often fail halfway." |
| |
85 | +- **Security:** the canonical exploit — "summarize this page" fed page content to the LLM |
| |
86 | + **without distinguishing user instructions from untrusted content**; hidden Reddit-comment |
| |
87 | + instructions exfiltrated email + OTP with no further input. (cybernews; brave.com/blog.) |
| |
88 | + |
| |
89 | +### Brave Leo — the privacy-first, BYO-model reference (closest sibling to Atlas) |
| |
90 | +Study hardest; its stance maps onto Atlas's constraints almost exactly. |
| |
91 | +- Sidebar + full-page + address-bar; **no account, no login**; reads pages, **PDFs, Google |
| |
92 | + Docs/Sheets, YouTube transcripts, images**. |
| |
93 | +- **Skills (best-documented `/`-command design):** type `/` → **New** → fields for |
| |
94 | + **model + shortcut + prompt** (per-skill model binding); or **"Save as skill"** from a |
| |
95 | + good answer's `…` menu (creation-from-success). Built-ins `/summarize` `/explain` |
| |
96 | + `/improve`. **Adopt this creation flow verbatim.** |
| |
97 | +- **Model catalog + BYOM:** free & premium model tiers, "Automatic" routing default, |
| |
98 | + default at `brave://settings/leo-ai`, and **connect local models via Ollama or your own |
| |
99 | + API endpoint** — "rare among browser-native AI tools." |
| |
100 | +- **Privacy gold standard:** no chats/context/logs in the cloud, no training, account-free; |
| |
101 | + reverse proxy strips IP. |
| |
102 | +- **Agentic (opt-in, off by default):** isolated profile; a **second alignment-checker |
| |
103 | + model firewalled from page content**; all agent work in a visible tab; memory writes |
| |
104 | + shown and undoable; **explicitly rejects per-site permission prompts** ("low-signal |
| |
105 | + repeated prompts train users to ignore warnings"). → **Don't build a nag-wall.** |
| |
106 | + (Brave help; brave.com/blog/ai-browsing, leo-skills.) |
| |
107 | + |
| |
108 | +### Microsoft Edge Copilot Mode — "whole-browser AI toggle" + granular consent |
| |
109 | +- **One toggle** switches the whole browser into/out of Copilot Mode (visible chrome change). |
| |
110 | +- **Unified new-tab box** (chat+search+navigate); **multi-tab context** with permission; |
| |
111 | + **Actions** (agentic, opt-in, US-only); **Journeys** (past browsing auto-grouped into |
| |
112 | + topical "storylines" as new-tab cards — reviewers' most-liked feature). |
| |
113 | +- **Best-documented consent model:** Edge **classifies each query** as needing page context |
| |
114 | + or not; non-contextual queries send only query + URL + title + history; contextual queries |
| |
115 | + prompt for **active-tab** access. → **Adopt the query-classifier privacy architecture.** |
| |
116 | + (Edge blog 2025-07-28, 2025-10-23; Microsoft Q&A.) |
| |
117 | + |
| |
118 | +### Opera (Aria → Opera AI) & Neon |
| |
119 | +- **Aria command line ⌘/** (ask without leaving the page); explicit **Page-Context mode**; |
| |
120 | + NL tab commands. |
| |
121 | +- **The Aria→Opera AI rebuild is instructive:** moved access from left sidebar to a |
| |
122 | + top-right button, from overlay-on-content to a **side panel**; made it **context-aware by |
| |
123 | + default** with a per-chat revoke toggle; **retired the Command Line and Compose mode** |
| |
124 | + because "a context-aware chat subsumes them." → **Don't build Compose as a separate |
| |
125 | + surface.** Also ships **answer→image snapshot to clipboard** and **AI wallpaper |
| |
126 | + generation** (relevant to Atlas theming). Neon ($20/mo) is agentic (Neon Do, composable |
| |
127 | + Cards) — and was prompt-injection-exploited via an `opacity:0` element. |
| |
128 | + (blogs.opera.com migration post; brave.com/blog/prompt-injection-flaw-opera-neon.) |
| |
129 | + |
| |
130 | +### SigmaOS — same stack (SwiftUI+WebKit), a cautionary tale |
| |
131 | +- Native SwiftUI on WebKit; Airis gave **structured summaries + key points + follow-up Q&A** |
| |
132 | + and early model choice (GPT-4o/Perplexity/Claude). |
| |
133 | +- **Why it died for users:** removed traditional bookmarks and **broke redirect-based SSO** |
| |
134 | + for every institutional workflow ("bookmarks are deferred execution triggers for auth |
| |
135 | + flows"); limited extensions; no security updates. → **Atlas must-dos:** keep plain |
| |
136 | + bookmarks that store+navigate a URL verbatim; **Phase-7-test redirect SSO/OAuth on |
| |
137 | + WKWebView** (Okta/SAML/Google/Entra/library proxies); natively cover top extension jobs |
| |
138 | + (ad/tracker block, autofill, reader, per-site CSS); rely on WKWebView inheriting Safari's |
| |
139 | + security cadence. (sigmaos.com; Product Hunt reviews; r/macapps.) |
| |
140 | + |
| |
141 | +### Chrome — Gemini in Chrome + built-in on-device APIs |
| |
142 | +- **Best multi-tab UX:** current tab shared by default (toggleable); **add up to 10 more via |
| |
143 | + a picker or `@`-mention with fuzzy search**; checkmarks in the picker; **a tab whose |
| |
144 | + content is in use shows a "glowing underline" on the tab itself** — the best "content |
| |
145 | + leaving device" indicator in the industry. **Adopt the on-the-tab indicator.** |
| |
146 | +- **Ask-AI-about-history** ("the site with the walnut desk"); **Skills** (save+reuse |
| |
147 | + multi-tab prompt workflows); **select-from-screen** prompting (compare two selected |
| |
148 | + products); menu-bar/global invoke with a configurable shortcut; **auto-browse** with a |
| |
149 | + well-specified confirm/take-over safety taxonomy. |
| |
150 | +- **Built-in on-device APIs (Gemini Nano):** Translator/Language-Detector/**Summarizer** |
| |
151 | + stable; Writer/Rewriter/Proofreader/Prompt in trial. → The **task taxonomy** Google |
| |
152 | + settled on — *summarize, translate, detect-language, write, rewrite, proofread* — is the |
| |
153 | + right set of primitive `AIActions` for a browser. A local tier (Apple Foundation Models / |
| |
154 | + MLX / Ollama) is a legitimate *future* option for hover-summaries. (Google blog; |
| |
155 | + developer.chrome.com/docs/ai; Chrome at I/O 2026.) |
| |
156 | + |
| |
157 | +### ChatGPT Atlas (retiring 2026-08-09) — most feature-complete; best primary docs |
| |
158 | +- **Two writing surfaces, sharply distinguished** (adopt the framing verbatim): *"The |
| |
159 | + sidebar is for bigger questions and page-level context. Inline editing is for precise |
| |
160 | + small improvements to the text you're actively typing."* Inline: highlight in a field → |
| |
161 | + click the logo → ask → **replace in place** (Insert fallback). |
| |
162 | +- **Selection implicitly joins chat context** ("highlight a paragraph and Atlas includes it |
| |
163 | + automatically"); **Saved Prompts** recalled via `@` / bookmarks bar; **semantic ⌘F** |
| |
164 | + ("no exact match → find similar"); **per-site visibility toggle in the address bar** |
| |
165 | + ("when off, ChatGPT can't view page content"); **Agent Mode capability sandbox** (no code |
| |
166 | + execution, no filesystem, no passwords/autofill, pages not added to history, logged-out |
| |
167 | + mode, isolated clipboard). |
| |
168 | +- **Criticisms to design against:** the "anti-web browser" critique (AI content substituted |
| |
169 | + for real links — "zero links to Taylor Swift's actual website") → **AI answers must be |
| |
170 | + additive to prominent, plural, clickable real sources**; "tainted memories" CSRF made |
| |
171 | + memory a persistent injection sink → **memory writes visible + undoable**; keyboard/ |
| |
172 | + customization poverty → **precisely the gap Atlas fills.** (OpenAI release notes; Wikipedia.) |
| |
173 | + |
| |
174 | +### Others |
| |
175 | +Claude in Chrome/Cowork (extension; two permission modes + site-level grants); Fellou |
| |
176 | +(**plan-preview-before-execute** — best agentic-transparency idea); Zen/Orion/Vivaldi/Brave |
| |
177 | +as Arc-refugee destinations chosen for **customization, not AI** (validates Atlas pillar #1). |
| |
178 | +Market share: no provider is a safe sole bet (ChatGPT ~53%, Gemini ~27%, Claude ~8% of |
| |
179 | +gen-AI traffic, Jun 2026) — the strongest argument for **multi-provider/BYO-key**. |
| |
180 | + |
| |
181 | +--- |
| |
182 | + |
| |
183 | +## 2. Page content extraction for LLMs (the technical heart of `Content/`) |
| |
184 | + |
| |
185 | +Extraction failure *is* AI failure in the user's mind (Arc evidence, §1). Budget |
| |
186 | +disproportionately here. |
| |
187 | + |
| |
188 | +### Readability & the extractor landscape |
| |
189 | +- **Mozilla Readability.js** (Apache-2.0, the Firefox Reader engine): `_grabArticle()` |
| |
190 | + scores nodes (1 base + 1/comma + ≤3 for length + ±25 class/id weight), propagates to |
| |
191 | + ancestors, scales by `(1 − linkDensity)`, takes top `nbTopCandidates=5`, appends |
| |
192 | + qualifying siblings, and retries with relaxed flags until ≥ `charThreshold=500` chars. |
| |
193 | + Returns `{title, content(HTML), textContent, excerpt, byline, siteName, lang, |
| |
194 | + publishedTime}`. **`parse()` mutates the DOM — always `document.cloneNode(true)` first** |
| |
195 | + in a live WKWebView. `isProbablyReaderable(document)` is the cheap **article-vs-app gate**. |
| |
196 | + Does NOT sanitize output — treat page-derived strings as untrusted (injection surface). |
| |
197 | +- **Defuddle** (MIT, kepano): browser-first, more forgiving than Readability, standardizes |
| |
198 | + code/math/footnotes, and **emits Markdown directly** (`markdown: true`) — skipping a |
| |
199 | + separate Turndown step. Other refs: trafilatura (Python, server-only), Postlight |
| |
200 | + (unmaintained), dom-distiller (uses render signals). |
| |
201 | +- **2026 benchmark reality:** extractors converge on articles (F1 ≈ 0.93) but diverge on |
| |
202 | + structured/app pages (F1 0.41–0.84). **The article path is solved; invest in the fallback.** |
| |
203 | +- **Decision:** primary extractor = **Defuddle** (MIT, browser-first, native Markdown); |
| |
204 | + keep **Readability** as A/B fallback and `isProbablyReaderable` as the gate. Both run on a |
| |
205 | + clone so the live page is untouched. (github.com/mozilla/readability, kepano/defuddle; |
| |
206 | + arxiv 2605.21097.) |
| |
207 | + |
| |
208 | +### Fallback ladder when extraction fails (Gmail, dashboards, SPAs) |
| |
209 | +1. `isProbablyReaderable` gate; post-hoc check (extracted < `charThreshold` or < ~20% of |
| |
210 | + `body.innerText`) → treat as failed. |
| |
211 | +2. **`document.body.innerText`** — layout-aware "what the user can see" (excludes |
| |
212 | + `display:none`, inserts render-accurate breaks). Annotate `extractionQuality: .rawText`. |
| |
213 | +3. **Structured visible-text TreeWalker** → pseudo-Markdown (h1–h6→`#`, li→`-`, table→pipes, |
| |
214 | + a→links), skipping `display:none`/zero-rect nodes. |
| |
215 | +4. **Accessibility-tree-style walk** for app pages (role + accessible name + state). Raw HTML |
| |
216 | + dumps are an anti-pattern. |
| |
217 | +5. Last resort: title + URL + meta description, with the UI saying "couldn't read this page." |
| |
218 | + |
| |
219 | +### Why Markdown, and structure preservation |
| |
220 | +Markdown cuts tokens ~60–80% vs HTML; headings survive as `##` lines (chunk boundaries + |
| |
221 | +citation anchors), lists/tables keep semantics, links keep `[text](url)` for "answer with |
| |
222 | +sources." `PageContext.markdown` is canonical; keep `plainText` for token estimates and a |
| |
223 | +heading index `[(headingPath, charRange)]` for chunking + citations. (Turndown with |
| |
224 | +`headingStyle:'atx'`, `codeBlockStyle:'fenced'` if converting Readability HTML.) |
| |
225 | + |
| |
226 | +### WKWebView mechanics |
| |
227 | +- **Isolate everything in a named `WKContentWorld`** (`ZyquoAtlasContent`): the extractor |
| |
228 | + reads the DOM/selection/computed-styles (shared across worlds) but never collides with or |
| |
229 | + is observed by page JS. Message handlers registered in that world can't be spoofed by the |
| |
230 | + page. **Exception:** the `history.pushState` hook (SPA detection) must run in `.page` |
| |
231 | + because it wraps the page's own binding. |
| |
232 | +- **Pull:** `callAsyncJavaScript` (named args auto-serialize; body may `await`; resolves |
| |
233 | + Promises) — but **`evaluateJavaScript` crashes on an `undefined` result**, so every |
| |
234 | + injected entry point ends with a value and is wrapped `try/catch → {ok, ...}`. Cap |
| |
235 | + extraction in JS (~300–600 KB), send one JSON object; never stream the DOM across the |
| |
236 | + bridge. |
| |
237 | +- **Push:** world-scoped `WKScriptMessageHandler` for selection-changed, SPA-navigation, and |
| |
238 | + a MutationObserver `contentVersion` counter. |
| |
239 | +- **Selection:** `window.getSelection()`; `getRangeAt(0).getBoundingClientRect()` positions |
| |
240 | + the floating toolbar (account for `webView.pageZoom`); send ±N chars of surrounding text + |
| |
241 | + nearest heading for grounding; debounce `selectionchange` ~200 ms; post `{text, rect, |
| |
242 | + isEditable}` (isEditable drives Rewrite-vs-Explain). Cross-origin iframes need per-frame |
| |
243 | + observers (`forMainFrameOnly:false`) aggregated natively. |
| |
244 | +- **SPA navigation** fires no `WKNavigationDelegate` callbacks — detect via **KVO on |
| |
245 | + `webView.url` + a `pushState/replaceState/popstate` hook + MutationObserver**, all three; |
| |
246 | + each invalidates the `PageContext` cache and cancels in-flight AI for the old context. |
| |
247 | +- **App-page data paths:** special-case YouTube (transcript), PDF (PDFKit text), Google Docs |
| |
248 | + (full-document script/selection) rather than DOM scraping (Chrome reads the Workspace API |
| |
249 | + for shared Docs — same lesson). |
| |
250 | + |
| |
251 | +--- |
| |
252 | + |
| |
253 | +## 3. Chunking, context & long pages |
| |
254 | + |
| |
255 | +- **2026 long-context calculus:** frontier windows are 200K–2.5M tokens, but effective |
| |
256 | + context is ~50–65% of advertised (context rot), and filling big windows is slow + costly. |
| |
257 | + **A web page is almost never "long" in 2026 terms** (a long article ≈ 5–15K tokens; brutal |
| |
258 | + pages rarely pass 100K after boilerplate stripping). |
| |
259 | +- **Default = STUFF the whole page** for any Cloud model with ≥128K context. `Summarizer` |
| |
260 | + budget rule: `if estTokens(page) < 0.6 × model.contextWindow − reservedOutput → stuff; |
| |
261 | + else → map-reduce`. Map-reduce survives only for (i) small/cheap models chosen for |
| |
262 | + hover-summaries, (ii) pathological pages, (iii) **multi-tab compare** (N pages × 15K). |
| |
263 | + Map runs chunks concurrently; **stream only the reduce phase** to the UI (map partials |
| |
264 | + shown as progress "Reading section 3/9…"). **Refine is never used** (sequential → bad |
| |
265 | + streaming UX; page content isn't narrative enough to need it). |
| |
266 | +- **Chunker** over `PageContext.markdown`: split on heading boundaries (carry `headingPath`); |
| |
267 | + large sections → ~600-token Q&A chunks with ~50-token overlap; tiny sections merge; token |
| |
268 | + estimate `chars/4`. Each `Chunk {id, headingPath, text, charRange, firstWords}` — |
| |
269 | + `charRange`/`firstWords` power citation + highlight. Summarize chunks are coarser (2–4K, |
| |
270 | + no overlap). |
| |
271 | +- **Chat-with-page retrieval:** **stuff by default**; when over budget or multi-tab, a tiny |
| |
272 | + **Swift BM25** (k1=1.2, b=0.75, ~80 LOC) scores chunks against the question + recent turns, |
| |
273 | + taking top-k within budget + always the lead section. **No local embeddings in v1** (the |
| |
274 | + benchmark deltas don't justify the complexity for single-page scope) — leave a |
| |
275 | + `RelevanceSelector` protocol hook. This matches what Brave Leo ships (stuff/truncate, no |
| |
276 | + per-page vector DB). (firecrawl chunking benchmarks; Brave Leo deepwiki.) |
| |
277 | + |
| |
278 | +--- |
| |
279 | + |
| |
280 | +## 4. AI surfaces & UX patterns (Phase 3/6 feature contract) |
| |
281 | + |
| |
282 | +Every Phase 3/6 AI surface, with the pattern it traces to: |
| |
283 | + |
| |
284 | +1. **Smart omnibox (`OmniIntent`)** — graduated ladder, not a mode switch: URL → navigate |
| |
285 | + (always, immediately); fuzzy palette (tabs/history/bookmarks/actions); web search; **ask** |
| |
286 | + (streamed answer + prominent plural sources, in an anchored overlay, never navigating |
| |
287 | + away). Show the resolved mode inline before commit; bias to navigate/search; force-ask via |
| |
288 | + modifier (Dia keys); user-settable default. **Ship the palette AND the ask** (Dia's lesson). |
| |
289 | +2. **⌘F as the AI front door** — exact → fuzzy → **semantic** → **ask-about-page** (Arc |
| |
290 | + Ask-on-Page + OpenAI semantic find, combined; no one has shipped the full ladder). |
| |
291 | +3. **Chat-with-page sidebar** (right, ~360pt, collapsible; ⌘⇧A) — per-tab `PageContext` |
| |
292 | + context and per-tab history; model chip (all Cloud models) + Automatic/sticky/default; |
| |
293 | + quick-action row (Summarize, Key points, Translate, Ask selection); **numbered `[§n]` |
| |
294 | + citations that scroll-to-highlight** the source section on click. |
| |
295 | +4. **Selection floating toolbar** — on text selection: Explain / Summarize / Translate / |
| |
296 | + Rewrite / Ask; short results in an anchored popover, long results hand off to the sidebar |
| |
297 | + with the selection as context (Edge "transition to panel"); selection also implicitly |
| |
298 | + joins chat context (OpenAI). Support multiple/element selections later (Chrome I/O 2026). |
| |
299 | +5. **Inline writing assist in web text fields** — distinct from the sidebar (OpenAI framing): |
| |
300 | + highlight in a `<textarea>`/`contenteditable` → quiet logo affordance → improve / rewrite |
| |
301 | + / expand / shorten / fix-grammar / translate → **replace in place** via |
| |
302 | + `execCommand('insertText')` (preserves undo + triggers the site's React/framework input |
| |
303 | + pipeline) with a native-setter+`input`-event fallback; overlay UI in a `body`-appended |
| |
304 | + container positioned via `Range.getClientRects()` (Grammarly technique — never inject |
| |
305 | + markup into the editable content). Make the affordance **quiet and dismissible** (Edge |
| |
306 | + auto-popup backlash). |
| |
307 | +6. **Summarize / TL;DR** — one keystroke; key-points-first with progressive disclosure |
| |
308 | + (Airis structured summaries); long-page map-reduce with streaming progress. |
| |
309 | +7. **Translate page / selection** to a chosen language. |
| |
310 | +8. **Multi-tab reasoning** — gather several tabs' `PageContext`; **explicit picker with |
| |
311 | + visible chips** (not Comet's flaky `@tab1`); compare/summarize across them. |
| |
312 | +9. **Skills (`/shortcut`)** — ship in Phase 3, not later (industry consensus: Dia/Leo/Chrome/ |
| |
313 | + Neon/Comet all shipped it). Per-skill **model + shortcut + prompt** binding (Leo); |
| |
314 | + **create-from-a-good-answer** and a natural-language builder; built-ins (`/summarize` |
| |
315 | + `/explain` `/keypoints` `/translate` `/improve` `/tldr` `/compare-tabs`); JSON |
| |
316 | + import/export (doubles as the gallery story with no server). |
| |
317 | +10. **Opt-in auto-actions** — auto-summary on opening long articles; **link hover-summaries** |
| |
318 | + (Arc 5-Second Previews); always user-toggleable, per-action model (fast/cheap). |
| |
319 | +11. **Zero-UI AI on chores** — tidy tab titles on pin, tidy download filenames, suggest a |
| |
320 | + bookmark folder, auto-name tab groups (Arc/Opera); preview-then-apply for any AI tab |
| |
321 | + mutation. |
| |
322 | +12. **Ask-AI-about-my-history** — retrieval over the local full-text history index; send only |
| |
323 | + candidate titles/snippets, never ship history to a model (Chrome pattern, done privately). |
| |
324 | +13. **Global Quick Ask** — ⌥Space + menu-bar extra, user-configurable shortcut (Chrome |
| |
325 | + menu-bar Gemini). |
| |
326 | + |
| |
327 | +Keyboard census adopted: ⌘⇧A AI sidebar (Atlas/Comet), ⌥Space Quick Ask, ⌘F ask-ladder |
| |
328 | +(Arc), Dia-style omnibox force-modifiers. |
| |
329 | + |
| |
330 | +--- |
| |
331 | + |
| |
332 | +## 5. Streaming & responsiveness (60fps while a page loads AND AI streams) |
| |
333 | + |
| |
334 | +- **Coalesce SSE deltas** into a buffer, flush to `@Observable`/`@Published` state on a |
| |
335 | + timer **~33–100 ms** (30Hz buttery ↔ 100ms flicker-free) — never re-render per token. |
| |
336 | +- **Incremental Markdown safely:** the parse tree is unstable mid-stream (two backticks look |
| |
337 | + like inline code until the third arrives). Parse the buffer into **blocks**, render |
| |
338 | + completed blocks as **immutable views with stable hierarchical IDs**, render only the |
| |
339 | + trailing partial block as plain text; **only the last block ever mutates**, so SwiftUI |
| |
340 | + re-lays-out just the tail. Parse off the main actor, publish on `@MainActor`. Reference: |
| |
341 | + microsoft/SwiftStreamingMarkdown (feed full-source-so-far, not deltas; built-in inline |
| |
342 | + citations) — port/imitate. |
| |
343 | +- **Per-tab cancellation:** each tab's AI context owns at most one in-flight |
| |
344 | + `Task<Void, Error>`. Cancel on (a) `didStartProvisionalNavigation`, (b) SPA locationchange, |
| |
345 | + (c) tab close, (d) Stop, (e) a superseding request. Cancelling the Task cancels the |
| |
346 | + underlying `URLSession.bytes` stream automatically (Swift cooperative cancellation) — this |
| |
347 | + is exactly Zyquo Cloud's `AsyncThrowingStream` + `continuation.onTermination { task.cancel() }` |
| |
348 | + pattern (see `PROVIDER-REUSE.md` §5), so Atlas's cancel-on-navigation hooks straight in. |
| |
349 | +- Keep all AI networking/parsing/chunking off the main thread; WKWebView renders in its own |
| |
350 | + processes, so the only jank risk is *our* main actor. |
| |
351 | + |
| |
352 | +--- |
| |
353 | + |
| |
354 | +## 6. Multi-tab / multi-page reasoning |
| |
355 | + |
| |
356 | +- Global panel that **tracks the active tab**, with per-tab conversation state underneath |
| |
357 | + (dominant shipped pattern: Comet/Leo/Gemini). Add other tabs via an **explicit picker with |
| |
358 | + chips** + `@`-mention fuzzy search (Chrome), never freetext matching (Comet's failure). |
| |
359 | +- "Summarize/compare these N tabs" gathers each tab's `PageContext`; N×~15K tokens usually |
| |
360 | + exceeds a small model → **map-reduce across tabs** (§3): map = per-tab summary (concurrent), |
| |
361 | + reduce = the cross-tab comparison (streamed). Cite per claim which tab/section it came from. |
| |
362 | + |
| |
363 | +--- |
| |
364 | + |
| |
365 | +## 7. Safety, privacy & correctness (a design constraint from the first AI line) |
| |
366 | + |
| |
367 | +**Consent & indicators** |
| |
368 | +- **First-run opt-in** (Brave Leo) — AI off until confirmed. |
| |
369 | +- **On-the-tab "content in use" indicator** (Chrome's glowing underline) as Atlas's canonical |
| |
370 | + "content leaving device" signal, plus a chrome-level in-flight indicator, plus a per-site |
| |
371 | + **"invisible to AI" toggle in the omnibox** (OpenAI), plus a one-screen "what left this |
| |
372 | + device today" log. |
| |
373 | +- **Query classifier** (Edge): most AI interactions ship only query + URL + title + local |
| |
374 | + history; page content leaves **only when the action needs it**, always to the user's chosen |
| |
375 | + provider via their own key — **never to Zyquo** (this is the project privacy rule). |
| |
376 | +- **No nag-wall** (Brave): one clear global consent + a per-site *denylist*, prompts reserved |
| |
377 | + for genuinely consequential moments; consent scoped to the surface, never inherited from an |
| |
378 | + unrelated setting (OpenAI's mistake). |
| |
379 | +- **Never default to collecting content data** (Dia's mistake — 30-day undeleteable |
| |
380 | + retention). With BYO keys, Zyquo sees nothing. |
| |
381 | + |
| |
382 | +**Prompt injection (architectural invariants, even though Atlas is read-only)** |
| |
383 | +Both the Comet and Neon exploits were triggered by *nothing more than clicking "summarize |
| |
384 | +this page."* Therefore: |
| |
385 | +- **Page content is always untrusted data, never instructions.** Wrap extracted content in |
| |
386 | + explicit delimiters + datamarking (Microsoft Spotlighting), label it untrusted in the |
| |
387 | + system prompt, and treat model output derived from it as potentially unsafe (Brave |
| |
388 | + mitigation #1). |
| |
389 | +- **Extract visible rendered text only** — strip `display:none`, `opacity:0`, |
| |
390 | + `visibility:hidden`, zero-size/off-screen, `aria-hidden`, HTML comments, `<template>`, |
| |
391 | + low-contrast/white-on-white text (Neon's exploit rode raw-HTML hidden text). |
| |
392 | +- **The AI layer has zero tool access:** no navigation, no fetch, no filesystem, no shell, no |
| |
393 | + key-vault reads, no cookies/passwords/autofill, no `zyquo://` internal or non-HTTPS pages, |
| |
394 | + no cross-tab reach that isn't explicitly user-selected — so even a fully successful |
| |
395 | + injection can only produce a bad *summary*, never an action (a documented capability |
| |
396 | + sandbox, per OpenAI's Agent-Mode boundary list). |
| |
397 | +- **On-device models don't fix injection** (Brave, Jun 2026) — a local tier is a |
| |
398 | + cost/latency optimization, not a security control. |
| |
399 | + |
| |
400 | +**Grounding & citation hygiene** |
| |
401 | +A citation makes a bad answer look trustworthy — the worst failure (CJR ~60% attribution |
| |
402 | +error; EBU ~half of news answers flawed). Countermeasures with UX teeth: |
| |
403 | +- **`[§n]` chunk-ID citations** in the prompt (constrain IDs to supplied chunks, so a |
| |
404 | + hallucinated `[§7]` is detectable and droppable); optionally upgrade to Anthropic's native |
| |
405 | + Citations blocks when the provider is Anthropic. |
| |
406 | +- **Quote-and-locate:** require a short verbatim quote per citation; verify by substring |
| |
407 | + against the chunk (whitespace-fuzzy); drop unverifiable citations. |
| |
408 | +- **Click → scroll-and-highlight in the live page** via an injected `locateQuote()` using |
| |
409 | + `window.find`/Range + the CSS Custom Highlight API (styled with the theme accent), with a |
| |
410 | + `#:~:text=` Text-Fragment fallback for citations pointing at other tabs (WebKit supports |
| |
411 | + text fragments since Safari 16.1). `headingPath` chips are the zero-trust baseline even |
| |
412 | + when quote verification fails. |
| |
413 | +- Persistent lightweight "AI can be wrong — verify important info" line (industry standard); |
| |
414 | + **AI answers additive to prominent, plural, clickable real sources** (avoid OpenAI's |
| |
415 | + "anti-web browser" trap). |
| |
416 | + |
| |
417 | +--- |
| |
418 | + |
| |
419 | +## 8. Phase 7 additions derived from this research (beyond the CLAUDE.md spec) |
| |
420 | + |
| |
421 | +1. **Redirect-based SSO/OAuth on WKWebView** — Okta, SAML, Google, Microsoft Entra, library |
| |
422 | + proxies (the bug that killed SigmaOS for institutional users). |
| |
423 | +2. **Prompt-injection resistance suite** — pages with white-on-white text, `opacity:0` divs, |
| |
424 | + HTML comments, `aria-hidden` blocks, low-contrast text in images, and a spoiler-tagged |
| |
425 | + Reddit-style comment; assert Atlas **summarizes them and reports the embedded instructions |
| |
426 | + rather than following or silently ingesting them.** |
| |
427 | +3. **Extraction-quality suite** — news article, docs page, blog, JS-heavy app, very long |
| |
428 | + article, page with a user selection, PDF, YouTube; assert clean main-content + correct |
| |
429 | + selection + honest failure messaging + chunk-count transparency. |
| |
430 | + |
| |
431 | +--- |
| |
432 | + |
| |
433 | +## 9. Traceability — planned feature → pattern source |
| |
434 | + |
| |
435 | +| Atlas feature (Phase 3/6) | Pattern / source | |
| |
436 | +|---|---| |
| |
437 | +| Smart omnibox (navigate/search/ask, sources) | Dia intent bar (+overrides), Comet answers-with-sources, §4.1 | |
| |
438 | +| ⌘F ask-ladder (exact→fuzzy→semantic→ask) | Arc Ask-on-Page ⌘F + OpenAI semantic find, §4.2 | |
| |
439 | +| Chat-with-page sidebar + `[§n]` citations | Comet/Leo sidebar; Anthropic Citations; scroll-to-highlight, §4.3/§7 | |
| |
440 | +| Selection floating toolbar | Edge floating toolbar, OpenAI selection-in-context, §4.4 | |
| |
441 | +| Inline writing assist (replace-in-place) | OpenAI cursor-chat, Grammarly overlay+execCommand, §4.5 | |
| |
442 | +| Summarize / TL;DR (map-reduce, streaming) | Airis structured summaries; stuff-first calculus, §3/§4.6 | |
| |
443 | +| Multi-tab compare (picker+chips) | Chrome tab picker; Comet compare (fixing @tab), §4.8/§6 | |
| |
444 | +| Skills (`/shortcut`, per-skill model) | Dia/Leo/Chrome Skills, Leo creation flow, §4.9 | |
| |
445 | +| Auto-summary / hover previews (opt-in) | Arc 5-Second Previews, Edge auto, §4.10 | |
| |
446 | +| Zero-UI tidy (titles/downloads/groups) | Arc Tidy*, Opera NL tab commands, §4.11 | |
| |
447 | +| Ask-AI-about-history (local) | Chrome history search, §4.12 | |
| |
448 | +| Global Quick Ask (⌥Space + menu bar) | Chrome menu-bar Gemini, §4.13 | |
| |
449 | +| Content extraction pipeline | Defuddle/Readability + fallback ladder, §2 | |
| |
450 | +| Streaming at 60fps, cancel-on-nav | SwiftStreamingMarkdown, Swift cancellation, §5 | |
| |
451 | +| Privacy indicators + query classifier | Chrome tab-glow, Edge classifier, Brave consent, §7 | |
| |
452 | +| Injection-safe prompt hygiene | Brave 4 mitigations, MS Spotlighting, §7 | |
| |
453 | +| Import/export everything (trust) | Arc-death lock-in lesson, §1 | |