|
1 |
+<!-- |
|
2 |
+ AI-BROWSER-RESEARCH.md |
|
3 |
+ Zyquo Atlas |
|
4 |
+ Author: Simon-Pierre Boucher |
|
5 |
+ Mail: contact@spboucher.ai |
|
6 |
+--> |
|
7 |
+ |
|
8 |
+# Zyquo Atlas — AI Browser Research (Phase 0.A) |
|
9 |
+ |
|
10 |
+Compiled 2026-07-30 from several intensive web-research sessions across the current |
|
11 |
+AI-browser landscape, page-extraction techniques, AI-UX patterns, streaming engineering, |
|
12 |
+and browser-AI security. This document is the **design contract for every AI feature** in |
|
13 |
+Phases 3 and 6 — each feature below traces to a pattern documented here. Sources are cited |
|
14 |
+inline; where a claim rested only on low-quality SEO content it is marked *[unverified]*. |
|
15 |
+ |
|
16 |
+--- |
|
17 |
+ |
|
18 |
+## 0. Category state (July 2026) and what it means for Atlas |
|
19 |
+ |
|
20 |
+Three facts shape the design: |
|
21 |
+ |
|
22 |
+1. **The AI-sidebar-only browser is not a business.** Arc went to maintenance mode (May |
|
23 |
+ 2025; The Browser Company acquired by Atlassian, Oct 2025). OpenAI is **shutting down |
|
24 |
+ ChatGPT Atlas on 2026-08-09**, concluding "the browser is a feature, not the |
|
25 |
+ destination." SigmaOS — a native SwiftUI+WebKit browser, *exactly Atlas's stack* — is |
|
26 |
+ effectively abandoned. → **The browser must be excellent as a browser first** (Atlas's |
|
27 |
+ Phase 2 gate is correct); AI is a multiplier, not the product. |
|
28 |
+ (Engadget, TechCrunch 2026-07-09, r/macapps.) |
|
29 |
+2. **Indirect prompt injection is the unsolved, category-defining security problem.** Brave |
|
30 |
+ demonstrated exploits in Comet (twice), Opera Neon, and Fellou; a plain "summarize this |
|
31 |
+ page" click exfiltrated a user's email + OTP in Comet. On-device models do **not** fix |
|
32 |
+ it. → **Atlas is read-oriented, not agentic** — a defensible, safer position — but even |
|
33 |
+ read-only AI is exploitable (§7). (brave.com/blog: comet-prompt-injection, |
|
34 |
+ unseeable-prompt-injections, prompt-injection-flaw-opera-neon, indirect-prompt-injection.) |
|
35 |
+3. **Agentic is where vendors went; grounded reading is where users get value.** Every |
|
36 |
+ agent-mode review reports impressive demos, unreliable multi-step execution, and heavy |
|
37 |
+ fallback to the reliable features (summarize, chat-with-page, compare tabs). → **Atlas |
|
38 |
+ invests in reliable reading tools + massive customization + BYO-key model choice**, the |
|
39 |
+ three gaps left by the failed/paywalled incumbents. |
|
40 |
+ |
|
41 |
+**Naming note:** OpenAI's retiring product was also "Atlas." Zyquo Atlas docs disambiguate |
|
42 |
+as *Zyquo Atlas* throughout (never "Atlas" alone in user-facing text). |
|
43 |
+ |
|
44 |
+--- |
|
45 |
+ |
|
46 |
+## 1. Landscape — functional patterns per product |
|
47 |
+ |
|
48 |
+### Arc / Arc Max (The Browser Company) — "AI as small, keystroke-invoked utilities" |
|
49 |
+The most important reference: AI as invisible utilities, each one interaction with a |
|
50 |
+shortcut, output where you already are. |
|
51 |
+- **Ask on Page:** hold **⌘F** — the find bar becomes an ask box; answer inline. Overloads |
|
52 |
+ an existing muscle-memory shortcut ("find" and "ask about this page" are the same intent |
|
53 |
+ at different fidelities). **Adopt.** |
|
54 |
+- **5-Second Previews:** hover a link (Shift+hover off search pages) → floating popover |
|
55 |
+ summarizing the *destination* page. |
|
56 |
+- **Tidy Tab Titles / Tidy Downloads / Tidy Tabs:** zero-UI AI applied to chores; users |
|
57 |
+ don't perceive it as "AI." Tidy Tabs only touches ephemeral tabs, never pinned. |
|
58 |
+- **Failure evidence to engineer against:** "Ask on Page told me it encountered an error 3 |
|
59 |
+ of 4 times"; "5-second previews don't give much beyond the page title." → **Extraction |
|
60 |
+ fragility is the #1 cause of perceived AI failure.** (Arc Max help; HN 37753840.) |
|
61 |
+ |
|
62 |
+### Dia (Browser Company/Atlassian) — "the URL bar is the chat box" |
|
63 |
+- **Intent-detecting address bar:** classifies input as navigate / search / ask, "mostly |
|
64 |
+ based on length and syntax." Documented failure: "sometimes I'm trying to google |
|
65 |
+ something and get thrown into an AI chat against my intention." Keyboard overrides: |
|
66 |
+ Ctrl+⌘+Return forces chat, Shift+⌘+Return forces search. |
|
67 |
+- **Skills:** saved prompts invoked by **`/shortcut`** in chat, created via a |
|
68 |
+ Natural-Language Skill Builder, shared in a public gallery. One power user built 50+ |
|
69 |
+ (`/summary`, `/research`, `/newsletter`…). **Strongest signal in the landscape that |
|
70 |
+ user-authored slash-prompts are the killer feature — and it's cheap.** |
|
71 |
+- **Chat-with-all-tabs** via `@tabname` / `@history` mentions (mentioned tabs render as a |
|
72 |
+ stacked pile); **Quick Insert** pushes AI output into the current field. |
|
73 |
+- **Regressions users resent:** no fuzzy ⌘T command bar (replaced by chat), no Spaces, no |
|
74 |
+ Boosts. "Power users miss the keyboard launcher more than the sidebar." → **Ship the |
|
75 |
+ palette AND the ask, not one replacing the other.** (miskakyto.fi; SupaSidebar; TechCrunch.) |
|
76 |
+ |
|
77 |
+### Perplexity Comet — the agentic sidebar, most-tested, most-exploited |
|
78 |
+- **Assistant sidebar** (⌘A); **one-click page summary** (icon next to Assistant); |
|
79 |
+ **@tab** cross-tab context; agentic tasks with **blue frames on pages it touches** and a |
|
80 |
+ visible step list. |
|
81 |
+- **Failure evidence:** `@tab` targeting "inconsistent or a bit buggy" (→ use a picker with |
|
82 |
+ chips, not fuzzy text); the agent "only analyzed the snippet, not the full page, so it |
|
83 |
+ missed the deeper answer" and a reviewer asked for "**a visible list of the pages it |
|
84 |
+ reviewed**" (→ Atlas's citation requirement); long agent chains "often fail halfway." |
|
85 |
+- **Security:** the canonical exploit — "summarize this page" fed page content to the LLM |
|
86 |
+ **without distinguishing user instructions from untrusted content**; hidden Reddit-comment |
|
87 |
+ instructions exfiltrated email + OTP with no further input. (cybernews; brave.com/blog.) |
|
88 |
+ |
|
89 |
+### Brave Leo — the privacy-first, BYO-model reference (closest sibling to Atlas) |
|
90 |
+Study hardest; its stance maps onto Atlas's constraints almost exactly. |
|
91 |
+- Sidebar + full-page + address-bar; **no account, no login**; reads pages, **PDFs, Google |
|
92 |
+ Docs/Sheets, YouTube transcripts, images**. |
|
93 |
+- **Skills (best-documented `/`-command design):** type `/` → **New** → fields for |
|
94 |
+ **model + shortcut + prompt** (per-skill model binding); or **"Save as skill"** from a |
|
95 |
+ good answer's `…` menu (creation-from-success). Built-ins `/summarize` `/explain` |
|
96 |
+ `/improve`. **Adopt this creation flow verbatim.** |
|
97 |
+- **Model catalog + BYOM:** free & premium model tiers, "Automatic" routing default, |
|
98 |
+ default at `brave://settings/leo-ai`, and **connect local models via Ollama or your own |
|
99 |
+ API endpoint** — "rare among browser-native AI tools." |
|
100 |
+- **Privacy gold standard:** no chats/context/logs in the cloud, no training, account-free; |
|
101 |
+ reverse proxy strips IP. |
|
102 |
+- **Agentic (opt-in, off by default):** isolated profile; a **second alignment-checker |
|
103 |
+ model firewalled from page content**; all agent work in a visible tab; memory writes |
|
104 |
+ shown and undoable; **explicitly rejects per-site permission prompts** ("low-signal |
|
105 |
+ repeated prompts train users to ignore warnings"). → **Don't build a nag-wall.** |
|
106 |
+ (Brave help; brave.com/blog/ai-browsing, leo-skills.) |
|
107 |
+ |
|
108 |
+### Microsoft Edge Copilot Mode — "whole-browser AI toggle" + granular consent |
|
109 |
+- **One toggle** switches the whole browser into/out of Copilot Mode (visible chrome change). |
|
110 |
+- **Unified new-tab box** (chat+search+navigate); **multi-tab context** with permission; |
|
111 |
+ **Actions** (agentic, opt-in, US-only); **Journeys** (past browsing auto-grouped into |
|
112 |
+ topical "storylines" as new-tab cards — reviewers' most-liked feature). |
|
113 |
+- **Best-documented consent model:** Edge **classifies each query** as needing page context |
|
114 |
+ or not; non-contextual queries send only query + URL + title + history; contextual queries |
|
115 |
+ prompt for **active-tab** access. → **Adopt the query-classifier privacy architecture.** |
|
116 |
+ (Edge blog 2025-07-28, 2025-10-23; Microsoft Q&A.) |
|
117 |
+ |
|
118 |
+### Opera (Aria → Opera AI) & Neon |
|
119 |
+- **Aria command line ⌘/** (ask without leaving the page); explicit **Page-Context mode**; |
|
120 |
+ NL tab commands. |
|
121 |
+- **The Aria→Opera AI rebuild is instructive:** moved access from left sidebar to a |
|
122 |
+ top-right button, from overlay-on-content to a **side panel**; made it **context-aware by |
|
123 |
+ default** with a per-chat revoke toggle; **retired the Command Line and Compose mode** |
|
124 |
+ because "a context-aware chat subsumes them." → **Don't build Compose as a separate |
|
125 |
+ surface.** Also ships **answer→image snapshot to clipboard** and **AI wallpaper |
|
126 |
+ generation** (relevant to Atlas theming). Neon ($20/mo) is agentic (Neon Do, composable |
|
127 |
+ Cards) — and was prompt-injection-exploited via an `opacity:0` element. |
|
128 |
+ (blogs.opera.com migration post; brave.com/blog/prompt-injection-flaw-opera-neon.) |
|
129 |
+ |
|
130 |
+### SigmaOS — same stack (SwiftUI+WebKit), a cautionary tale |
|
131 |
+- Native SwiftUI on WebKit; Airis gave **structured summaries + key points + follow-up Q&A** |
|
132 |
+ and early model choice (GPT-4o/Perplexity/Claude). |
|
133 |
+- **Why it died for users:** removed traditional bookmarks and **broke redirect-based SSO** |
|
134 |
+ for every institutional workflow ("bookmarks are deferred execution triggers for auth |
|
135 |
+ flows"); limited extensions; no security updates. → **Atlas must-dos:** keep plain |
|
136 |
+ bookmarks that store+navigate a URL verbatim; **Phase-7-test redirect SSO/OAuth on |
|
137 |
+ WKWebView** (Okta/SAML/Google/Entra/library proxies); natively cover top extension jobs |
|
138 |
+ (ad/tracker block, autofill, reader, per-site CSS); rely on WKWebView inheriting Safari's |
|
139 |
+ security cadence. (sigmaos.com; Product Hunt reviews; r/macapps.) |
|
140 |
+ |
|
141 |
+### Chrome — Gemini in Chrome + built-in on-device APIs |
|
142 |
+- **Best multi-tab UX:** current tab shared by default (toggleable); **add up to 10 more via |
|
143 |
+ a picker or `@`-mention with fuzzy search**; checkmarks in the picker; **a tab whose |
|
144 |
+ content is in use shows a "glowing underline" on the tab itself** — the best "content |
|
145 |
+ leaving device" indicator in the industry. **Adopt the on-the-tab indicator.** |
|
146 |
+- **Ask-AI-about-history** ("the site with the walnut desk"); **Skills** (save+reuse |
|
147 |
+ multi-tab prompt workflows); **select-from-screen** prompting (compare two selected |
|
148 |
+ products); menu-bar/global invoke with a configurable shortcut; **auto-browse** with a |
|
149 |
+ well-specified confirm/take-over safety taxonomy. |
|
150 |
+- **Built-in on-device APIs (Gemini Nano):** Translator/Language-Detector/**Summarizer** |
|
151 |
+ stable; Writer/Rewriter/Proofreader/Prompt in trial. → The **task taxonomy** Google |
|
152 |
+ settled on — *summarize, translate, detect-language, write, rewrite, proofread* — is the |
|
153 |
+ right set of primitive `AIActions` for a browser. A local tier (Apple Foundation Models / |
|
154 |
+ MLX / Ollama) is a legitimate *future* option for hover-summaries. (Google blog; |
|
155 |
+ developer.chrome.com/docs/ai; Chrome at I/O 2026.) |
|
156 |
+ |
|
157 |
+### ChatGPT Atlas (retiring 2026-08-09) — most feature-complete; best primary docs |
|
158 |
+- **Two writing surfaces, sharply distinguished** (adopt the framing verbatim): *"The |
|
159 |
+ sidebar is for bigger questions and page-level context. Inline editing is for precise |
|
160 |
+ small improvements to the text you're actively typing."* Inline: highlight in a field → |
|
161 |
+ click the logo → ask → **replace in place** (Insert fallback). |
|
162 |
+- **Selection implicitly joins chat context** ("highlight a paragraph and Atlas includes it |
|
163 |
+ automatically"); **Saved Prompts** recalled via `@` / bookmarks bar; **semantic ⌘F** |
|
164 |
+ ("no exact match → find similar"); **per-site visibility toggle in the address bar** |
|
165 |
+ ("when off, ChatGPT can't view page content"); **Agent Mode capability sandbox** (no code |
|
166 |
+ execution, no filesystem, no passwords/autofill, pages not added to history, logged-out |
|
167 |
+ mode, isolated clipboard). |
|
168 |
+- **Criticisms to design against:** the "anti-web browser" critique (AI content substituted |
|
169 |
+ for real links — "zero links to Taylor Swift's actual website") → **AI answers must be |
|
170 |
+ additive to prominent, plural, clickable real sources**; "tainted memories" CSRF made |
|
171 |
+ memory a persistent injection sink → **memory writes visible + undoable**; keyboard/ |
|
172 |
+ customization poverty → **precisely the gap Atlas fills.** (OpenAI release notes; Wikipedia.) |
|
173 |
+ |
|
174 |
+### Others |
|
175 |
+Claude in Chrome/Cowork (extension; two permission modes + site-level grants); Fellou |
|
176 |
+(**plan-preview-before-execute** — best agentic-transparency idea); Zen/Orion/Vivaldi/Brave |
|
177 |
+as Arc-refugee destinations chosen for **customization, not AI** (validates Atlas pillar #1). |
|
178 |
+Market share: no provider is a safe sole bet (ChatGPT ~53%, Gemini ~27%, Claude ~8% of |
|
179 |
+gen-AI traffic, Jun 2026) — the strongest argument for **multi-provider/BYO-key**. |
|
180 |
+ |
|
181 |
+--- |
|
182 |
+ |
|
183 |
+## 2. Page content extraction for LLMs (the technical heart of `Content/`) |
|
184 |
+ |
|
185 |
+Extraction failure *is* AI failure in the user's mind (Arc evidence, §1). Budget |
|
186 |
+disproportionately here. |
|
187 |
+ |
|
188 |
+### Readability & the extractor landscape |
|
189 |
+- **Mozilla Readability.js** (Apache-2.0, the Firefox Reader engine): `_grabArticle()` |
|
190 |
+ scores nodes (1 base + 1/comma + ≤3 for length + ±25 class/id weight), propagates to |
|
191 |
+ ancestors, scales by `(1 − linkDensity)`, takes top `nbTopCandidates=5`, appends |
|
192 |
+ qualifying siblings, and retries with relaxed flags until ≥ `charThreshold=500` chars. |
|
193 |
+ Returns `{title, content(HTML), textContent, excerpt, byline, siteName, lang, |
|
194 |
+ publishedTime}`. **`parse()` mutates the DOM — always `document.cloneNode(true)` first** |
|
195 |
+ in a live WKWebView. `isProbablyReaderable(document)` is the cheap **article-vs-app gate**. |
|
196 |
+ Does NOT sanitize output — treat page-derived strings as untrusted (injection surface). |
|
197 |
+- **Defuddle** (MIT, kepano): browser-first, more forgiving than Readability, standardizes |
|
198 |
+ code/math/footnotes, and **emits Markdown directly** (`markdown: true`) — skipping a |
|
199 |
+ separate Turndown step. Other refs: trafilatura (Python, server-only), Postlight |
|
200 |
+ (unmaintained), dom-distiller (uses render signals). |
|
201 |
+- **2026 benchmark reality:** extractors converge on articles (F1 ≈ 0.93) but diverge on |
|
202 |
+ structured/app pages (F1 0.41–0.84). **The article path is solved; invest in the fallback.** |
|
203 |
+- **Decision:** primary extractor = **Defuddle** (MIT, browser-first, native Markdown); |
|
204 |
+ keep **Readability** as A/B fallback and `isProbablyReaderable` as the gate. Both run on a |
|
205 |
+ clone so the live page is untouched. (github.com/mozilla/readability, kepano/defuddle; |
|
206 |
+ arxiv 2605.21097.) |
|
207 |
+ |
|
208 |
+### Fallback ladder when extraction fails (Gmail, dashboards, SPAs) |
|
209 |
+1. `isProbablyReaderable` gate; post-hoc check (extracted < `charThreshold` or < ~20% of |
|
210 |
+ `body.innerText`) → treat as failed. |
|
211 |
+2. **`document.body.innerText`** — layout-aware "what the user can see" (excludes |
|
212 |
+ `display:none`, inserts render-accurate breaks). Annotate `extractionQuality: .rawText`. |
|
213 |
+3. **Structured visible-text TreeWalker** → pseudo-Markdown (h1–h6→`#`, li→`-`, table→pipes, |
|
214 |
+ a→links), skipping `display:none`/zero-rect nodes. |
|
215 |
+4. **Accessibility-tree-style walk** for app pages (role + accessible name + state). Raw HTML |
|
216 |
+ dumps are an anti-pattern. |
|
217 |
+5. Last resort: title + URL + meta description, with the UI saying "couldn't read this page." |
|
218 |
+ |
|
219 |
+### Why Markdown, and structure preservation |
|
220 |
+Markdown cuts tokens ~60–80% vs HTML; headings survive as `##` lines (chunk boundaries + |
|
221 |
+citation anchors), lists/tables keep semantics, links keep `[text](url)` for "answer with |
|
222 |
+sources." `PageContext.markdown` is canonical; keep `plainText` for token estimates and a |
|
223 |
+heading index `[(headingPath, charRange)]` for chunking + citations. (Turndown with |
|
224 |
+`headingStyle:'atx'`, `codeBlockStyle:'fenced'` if converting Readability HTML.) |
|
225 |
+ |
|
226 |
+### WKWebView mechanics |
|
227 |
+- **Isolate everything in a named `WKContentWorld`** (`ZyquoAtlasContent`): the extractor |
|
228 |
+ reads the DOM/selection/computed-styles (shared across worlds) but never collides with or |
|
229 |
+ is observed by page JS. Message handlers registered in that world can't be spoofed by the |
|
230 |
+ page. **Exception:** the `history.pushState` hook (SPA detection) must run in `.page` |
|
231 |
+ because it wraps the page's own binding. |
|
232 |
+- **Pull:** `callAsyncJavaScript` (named args auto-serialize; body may `await`; resolves |
|
233 |
+ Promises) — but **`evaluateJavaScript` crashes on an `undefined` result**, so every |
|
234 |
+ injected entry point ends with a value and is wrapped `try/catch → {ok, ...}`. Cap |
|
235 |
+ extraction in JS (~300–600 KB), send one JSON object; never stream the DOM across the |
|
236 |
+ bridge. |
|
237 |
+- **Push:** world-scoped `WKScriptMessageHandler` for selection-changed, SPA-navigation, and |
|
238 |
+ a MutationObserver `contentVersion` counter. |
|
239 |
+- **Selection:** `window.getSelection()`; `getRangeAt(0).getBoundingClientRect()` positions |
|
240 |
+ the floating toolbar (account for `webView.pageZoom`); send ±N chars of surrounding text + |
|
241 |
+ nearest heading for grounding; debounce `selectionchange` ~200 ms; post `{text, rect, |
|
242 |
+ isEditable}` (isEditable drives Rewrite-vs-Explain). Cross-origin iframes need per-frame |
|
243 |
+ observers (`forMainFrameOnly:false`) aggregated natively. |
|
244 |
+- **SPA navigation** fires no `WKNavigationDelegate` callbacks — detect via **KVO on |
|
245 |
+ `webView.url` + a `pushState/replaceState/popstate` hook + MutationObserver**, all three; |
|
246 |
+ each invalidates the `PageContext` cache and cancels in-flight AI for the old context. |
|
247 |
+- **App-page data paths:** special-case YouTube (transcript), PDF (PDFKit text), Google Docs |
|
248 |
+ (full-document script/selection) rather than DOM scraping (Chrome reads the Workspace API |
|
249 |
+ for shared Docs — same lesson). |
|
250 |
+ |
|
251 |
+--- |
|
252 |
+ |
|
253 |
+## 3. Chunking, context & long pages |
|
254 |
+ |
|
255 |
+- **2026 long-context calculus:** frontier windows are 200K–2.5M tokens, but effective |
|
256 |
+ context is ~50–65% of advertised (context rot), and filling big windows is slow + costly. |
|
257 |
+ **A web page is almost never "long" in 2026 terms** (a long article ≈ 5–15K tokens; brutal |
|
258 |
+ pages rarely pass 100K after boilerplate stripping). |
|
259 |
+- **Default = STUFF the whole page** for any Cloud model with ≥128K context. `Summarizer` |
|
260 |
+ budget rule: `if estTokens(page) < 0.6 × model.contextWindow − reservedOutput → stuff; |
|
261 |
+ else → map-reduce`. Map-reduce survives only for (i) small/cheap models chosen for |
|
262 |
+ hover-summaries, (ii) pathological pages, (iii) **multi-tab compare** (N pages × 15K). |
|
263 |
+ Map runs chunks concurrently; **stream only the reduce phase** to the UI (map partials |
|
264 |
+ shown as progress "Reading section 3/9…"). **Refine is never used** (sequential → bad |
|
265 |
+ streaming UX; page content isn't narrative enough to need it). |
|
266 |
+- **Chunker** over `PageContext.markdown`: split on heading boundaries (carry `headingPath`); |
|
267 |
+ large sections → ~600-token Q&A chunks with ~50-token overlap; tiny sections merge; token |
|
268 |
+ estimate `chars/4`. Each `Chunk {id, headingPath, text, charRange, firstWords}` — |
|
269 |
+ `charRange`/`firstWords` power citation + highlight. Summarize chunks are coarser (2–4K, |
|
270 |
+ no overlap). |
|
271 |
+- **Chat-with-page retrieval:** **stuff by default**; when over budget or multi-tab, a tiny |
|
272 |
+ **Swift BM25** (k1=1.2, b=0.75, ~80 LOC) scores chunks against the question + recent turns, |
|
273 |
+ taking top-k within budget + always the lead section. **No local embeddings in v1** (the |
|
274 |
+ benchmark deltas don't justify the complexity for single-page scope) — leave a |
|
275 |
+ `RelevanceSelector` protocol hook. This matches what Brave Leo ships (stuff/truncate, no |
|
276 |
+ per-page vector DB). (firecrawl chunking benchmarks; Brave Leo deepwiki.) |
|
277 |
+ |
|
278 |
+--- |
|
279 |
+ |
|
280 |
+## 4. AI surfaces & UX patterns (Phase 3/6 feature contract) |
|
281 |
+ |
|
282 |
+Every Phase 3/6 AI surface, with the pattern it traces to: |
|
283 |
+ |
|
284 |
+1. **Smart omnibox (`OmniIntent`)** — graduated ladder, not a mode switch: URL → navigate |
|
285 |
+ (always, immediately); fuzzy palette (tabs/history/bookmarks/actions); web search; **ask** |
|
286 |
+ (streamed answer + prominent plural sources, in an anchored overlay, never navigating |
|
287 |
+ away). Show the resolved mode inline before commit; bias to navigate/search; force-ask via |
|
288 |
+ modifier (Dia keys); user-settable default. **Ship the palette AND the ask** (Dia's lesson). |
|
289 |
+2. **⌘F as the AI front door** — exact → fuzzy → **semantic** → **ask-about-page** (Arc |
|
290 |
+ Ask-on-Page + OpenAI semantic find, combined; no one has shipped the full ladder). |
|
291 |
+3. **Chat-with-page sidebar** (right, ~360pt, collapsible; ⌘⇧A) — per-tab `PageContext` |
|
292 |
+ context and per-tab history; model chip (all Cloud models) + Automatic/sticky/default; |
|
293 |
+ quick-action row (Summarize, Key points, Translate, Ask selection); **numbered `[§n]` |
|
294 |
+ citations that scroll-to-highlight** the source section on click. |
|
295 |
+4. **Selection floating toolbar** — on text selection: Explain / Summarize / Translate / |
|
296 |
+ Rewrite / Ask; short results in an anchored popover, long results hand off to the sidebar |
|
297 |
+ with the selection as context (Edge "transition to panel"); selection also implicitly |
|
298 |
+ joins chat context (OpenAI). Support multiple/element selections later (Chrome I/O 2026). |
|
299 |
+5. **Inline writing assist in web text fields** — distinct from the sidebar (OpenAI framing): |
|
300 |
+ highlight in a `<textarea>`/`contenteditable` → quiet logo affordance → improve / rewrite |
|
301 |
+ / expand / shorten / fix-grammar / translate → **replace in place** via |
|
302 |
+ `execCommand('insertText')` (preserves undo + triggers the site's React/framework input |
|
303 |
+ pipeline) with a native-setter+`input`-event fallback; overlay UI in a `body`-appended |
|
304 |
+ container positioned via `Range.getClientRects()` (Grammarly technique — never inject |
|
305 |
+ markup into the editable content). Make the affordance **quiet and dismissible** (Edge |
|
306 |
+ auto-popup backlash). |
|
307 |
+6. **Summarize / TL;DR** — one keystroke; key-points-first with progressive disclosure |
|
308 |
+ (Airis structured summaries); long-page map-reduce with streaming progress. |
|
309 |
+7. **Translate page / selection** to a chosen language. |
|
310 |
+8. **Multi-tab reasoning** — gather several tabs' `PageContext`; **explicit picker with |
|
311 |
+ visible chips** (not Comet's flaky `@tab1`); compare/summarize across them. |
|
312 |
+9. **Skills (`/shortcut`)** — ship in Phase 3, not later (industry consensus: Dia/Leo/Chrome/ |
|
313 |
+ Neon/Comet all shipped it). Per-skill **model + shortcut + prompt** binding (Leo); |
|
314 |
+ **create-from-a-good-answer** and a natural-language builder; built-ins (`/summarize` |
|
315 |
+ `/explain` `/keypoints` `/translate` `/improve` `/tldr` `/compare-tabs`); JSON |
|
316 |
+ import/export (doubles as the gallery story with no server). |
|
317 |
+10. **Opt-in auto-actions** — auto-summary on opening long articles; **link hover-summaries** |
|
318 |
+ (Arc 5-Second Previews); always user-toggleable, per-action model (fast/cheap). |
|
319 |
+11. **Zero-UI AI on chores** — tidy tab titles on pin, tidy download filenames, suggest a |
|
320 |
+ bookmark folder, auto-name tab groups (Arc/Opera); preview-then-apply for any AI tab |
|
321 |
+ mutation. |
|
322 |
+12. **Ask-AI-about-my-history** — retrieval over the local full-text history index; send only |
|
323 |
+ candidate titles/snippets, never ship history to a model (Chrome pattern, done privately). |
|
324 |
+13. **Global Quick Ask** — ⌥Space + menu-bar extra, user-configurable shortcut (Chrome |
|
325 |
+ menu-bar Gemini). |
|
326 |
+ |
|
327 |
+Keyboard census adopted: ⌘⇧A AI sidebar (Atlas/Comet), ⌥Space Quick Ask, ⌘F ask-ladder |
|
328 |
+(Arc), Dia-style omnibox force-modifiers. |
|
329 |
+ |
|
330 |
+--- |
|
331 |
+ |
|
332 |
+## 5. Streaming & responsiveness (60fps while a page loads AND AI streams) |
|
333 |
+ |
|
334 |
+- **Coalesce SSE deltas** into a buffer, flush to `@Observable`/`@Published` state on a |
|
335 |
+ timer **~33–100 ms** (30Hz buttery ↔ 100ms flicker-free) — never re-render per token. |
|
336 |
+- **Incremental Markdown safely:** the parse tree is unstable mid-stream (two backticks look |
|
337 |
+ like inline code until the third arrives). Parse the buffer into **blocks**, render |
|
338 |
+ completed blocks as **immutable views with stable hierarchical IDs**, render only the |
|
339 |
+ trailing partial block as plain text; **only the last block ever mutates**, so SwiftUI |
|
340 |
+ re-lays-out just the tail. Parse off the main actor, publish on `@MainActor`. Reference: |
|
341 |
+ microsoft/SwiftStreamingMarkdown (feed full-source-so-far, not deltas; built-in inline |
|
342 |
+ citations) — port/imitate. |
|
343 |
+- **Per-tab cancellation:** each tab's AI context owns at most one in-flight |
|
344 |
+ `Task<Void, Error>`. Cancel on (a) `didStartProvisionalNavigation`, (b) SPA locationchange, |
|
345 |
+ (c) tab close, (d) Stop, (e) a superseding request. Cancelling the Task cancels the |
|
346 |
+ underlying `URLSession.bytes` stream automatically (Swift cooperative cancellation) — this |
|
347 |
+ is exactly Zyquo Cloud's `AsyncThrowingStream` + `continuation.onTermination { task.cancel() }` |
|
348 |
+ pattern (see `PROVIDER-REUSE.md` §5), so Atlas's cancel-on-navigation hooks straight in. |
|
349 |
+- Keep all AI networking/parsing/chunking off the main thread; WKWebView renders in its own |
|
350 |
+ processes, so the only jank risk is *our* main actor. |
|
351 |
+ |
|
352 |
+--- |
|
353 |
+ |
|
354 |
+## 6. Multi-tab / multi-page reasoning |
|
355 |
+ |
|
356 |
+- Global panel that **tracks the active tab**, with per-tab conversation state underneath |
|
357 |
+ (dominant shipped pattern: Comet/Leo/Gemini). Add other tabs via an **explicit picker with |
|
358 |
+ chips** + `@`-mention fuzzy search (Chrome), never freetext matching (Comet's failure). |
|
359 |
+- "Summarize/compare these N tabs" gathers each tab's `PageContext`; N×~15K tokens usually |
|
360 |
+ exceeds a small model → **map-reduce across tabs** (§3): map = per-tab summary (concurrent), |
|
361 |
+ reduce = the cross-tab comparison (streamed). Cite per claim which tab/section it came from. |
|
362 |
+ |
|
363 |
+--- |
|
364 |
+ |
|
365 |
+## 7. Safety, privacy & correctness (a design constraint from the first AI line) |
|
366 |
+ |
|
367 |
+**Consent & indicators** |
|
368 |
+- **First-run opt-in** (Brave Leo) — AI off until confirmed. |
|
369 |
+- **On-the-tab "content in use" indicator** (Chrome's glowing underline) as Atlas's canonical |
|
370 |
+ "content leaving device" signal, plus a chrome-level in-flight indicator, plus a per-site |
|
371 |
+ **"invisible to AI" toggle in the omnibox** (OpenAI), plus a one-screen "what left this |
|
372 |
+ device today" log. |
|
373 |
+- **Query classifier** (Edge): most AI interactions ship only query + URL + title + local |
|
374 |
+ history; page content leaves **only when the action needs it**, always to the user's chosen |
|
375 |
+ provider via their own key — **never to Zyquo** (this is the project privacy rule). |
|
376 |
+- **No nag-wall** (Brave): one clear global consent + a per-site *denylist*, prompts reserved |
|
377 |
+ for genuinely consequential moments; consent scoped to the surface, never inherited from an |
|
378 |
+ unrelated setting (OpenAI's mistake). |
|
379 |
+- **Never default to collecting content data** (Dia's mistake — 30-day undeleteable |
|
380 |
+ retention). With BYO keys, Zyquo sees nothing. |
|
381 |
+ |
|
382 |
+**Prompt injection (architectural invariants, even though Atlas is read-only)** |
|
383 |
+Both the Comet and Neon exploits were triggered by *nothing more than clicking "summarize |
|
384 |
+this page."* Therefore: |
|
385 |
+- **Page content is always untrusted data, never instructions.** Wrap extracted content in |
|
386 |
+ explicit delimiters + datamarking (Microsoft Spotlighting), label it untrusted in the |
|
387 |
+ system prompt, and treat model output derived from it as potentially unsafe (Brave |
|
388 |
+ mitigation #1). |
|
389 |
+- **Extract visible rendered text only** — strip `display:none`, `opacity:0`, |
|
390 |
+ `visibility:hidden`, zero-size/off-screen, `aria-hidden`, HTML comments, `<template>`, |
|
391 |
+ low-contrast/white-on-white text (Neon's exploit rode raw-HTML hidden text). |
|
392 |
+- **The AI layer has zero tool access:** no navigation, no fetch, no filesystem, no shell, no |
|
393 |
+ key-vault reads, no cookies/passwords/autofill, no `zyquo://` internal or non-HTTPS pages, |
|
394 |
+ no cross-tab reach that isn't explicitly user-selected — so even a fully successful |
|
395 |
+ injection can only produce a bad *summary*, never an action (a documented capability |
|
396 |
+ sandbox, per OpenAI's Agent-Mode boundary list). |
|
397 |
+- **On-device models don't fix injection** (Brave, Jun 2026) — a local tier is a |
|
398 |
+ cost/latency optimization, not a security control. |
|
399 |
+ |
|
400 |
+**Grounding & citation hygiene** |
|
401 |
+A citation makes a bad answer look trustworthy — the worst failure (CJR ~60% attribution |
|
402 |
+error; EBU ~half of news answers flawed). Countermeasures with UX teeth: |
|
403 |
+- **`[§n]` chunk-ID citations** in the prompt (constrain IDs to supplied chunks, so a |
|
404 |
+ hallucinated `[§7]` is detectable and droppable); optionally upgrade to Anthropic's native |
|
405 |
+ Citations blocks when the provider is Anthropic. |
|
406 |
+- **Quote-and-locate:** require a short verbatim quote per citation; verify by substring |
|
407 |
+ against the chunk (whitespace-fuzzy); drop unverifiable citations. |
|
408 |
+- **Click → scroll-and-highlight in the live page** via an injected `locateQuote()` using |
|
409 |
+ `window.find`/Range + the CSS Custom Highlight API (styled with the theme accent), with a |
|
410 |
+ `#:~:text=` Text-Fragment fallback for citations pointing at other tabs (WebKit supports |
|
411 |
+ text fragments since Safari 16.1). `headingPath` chips are the zero-trust baseline even |
|
412 |
+ when quote verification fails. |
|
413 |
+- Persistent lightweight "AI can be wrong — verify important info" line (industry standard); |
|
414 |
+ **AI answers additive to prominent, plural, clickable real sources** (avoid OpenAI's |
|
415 |
+ "anti-web browser" trap). |
|
416 |
+ |
|
417 |
+--- |
|
418 |
+ |
|
419 |
+## 8. Phase 7 additions derived from this research (beyond the CLAUDE.md spec) |
|
420 |
+ |
|
421 |
+1. **Redirect-based SSO/OAuth on WKWebView** — Okta, SAML, Google, Microsoft Entra, library |
|
422 |
+ proxies (the bug that killed SigmaOS for institutional users). |
|
423 |
+2. **Prompt-injection resistance suite** — pages with white-on-white text, `opacity:0` divs, |
|
424 |
+ HTML comments, `aria-hidden` blocks, low-contrast text in images, and a spoiler-tagged |
|
425 |
+ Reddit-style comment; assert Atlas **summarizes them and reports the embedded instructions |
|
426 |
+ rather than following or silently ingesting them.** |
|
427 |
+3. **Extraction-quality suite** — news article, docs page, blog, JS-heavy app, very long |
|
428 |
+ article, page with a user selection, PDF, YouTube; assert clean main-content + correct |
|
429 |
+ selection + honest failure messaging + chunk-count transparency. |
|
430 |
+ |
|
431 |
+--- |
|
432 |
+ |
|
433 |
+## 9. Traceability — planned feature → pattern source |
|
434 |
+ |
|
435 |
+| Atlas feature (Phase 3/6) | Pattern / source | |
|
436 |
+|---|---| |
|
437 |
+| Smart omnibox (navigate/search/ask, sources) | Dia intent bar (+overrides), Comet answers-with-sources, §4.1 | |
|
438 |
+| ⌘F ask-ladder (exact→fuzzy→semantic→ask) | Arc Ask-on-Page ⌘F + OpenAI semantic find, §4.2 | |
|
439 |
+| Chat-with-page sidebar + `[§n]` citations | Comet/Leo sidebar; Anthropic Citations; scroll-to-highlight, §4.3/§7 | |
|
440 |
+| Selection floating toolbar | Edge floating toolbar, OpenAI selection-in-context, §4.4 | |
|
441 |
+| Inline writing assist (replace-in-place) | OpenAI cursor-chat, Grammarly overlay+execCommand, §4.5 | |
|
442 |
+| Summarize / TL;DR (map-reduce, streaming) | Airis structured summaries; stuff-first calculus, §3/§4.6 | |
|
443 |
+| Multi-tab compare (picker+chips) | Chrome tab picker; Comet compare (fixing @tab), §4.8/§6 | |
|
444 |
+| Skills (`/shortcut`, per-skill model) | Dia/Leo/Chrome Skills, Leo creation flow, §4.9 | |
|
445 |
+| Auto-summary / hover previews (opt-in) | Arc 5-Second Previews, Edge auto, §4.10 | |
|
446 |
+| Zero-UI tidy (titles/downloads/groups) | Arc Tidy*, Opera NL tab commands, §4.11 | |
|
447 |
+| Ask-AI-about-history (local) | Chrome history search, §4.12 | |
|
448 |
+| Global Quick Ask (⌥Space + menu bar) | Chrome menu-bar Gemini, §4.13 | |
|
449 |
+| Content extraction pipeline | Defuddle/Readability + fallback ladder, §2 | |
|
450 |
+| Streaming at 60fps, cancel-on-nav | SwiftStreamingMarkdown, Swift cancellation, §5 | |
|
451 |
+| Privacy indicators + query classifier | Chrome tab-glow, Edge classifier, Brave consent, §7 | |
|
452 |
+| Injection-safe prompt hygiene | Brave 4 mitigations, MS Spotlighting, §7 | |
|
453 |
+| Import/export everything (trust) | Arc-death lock-in lesson, §1 | |