feat: SPB Git 2.0 — scoped search palette, split diffs, insights, Research Mode, opt-in AI Q&A, compression, a11y/QA tooling, docs
Phases 3–7 of docs/SPBGIT-2-PLAN.md: /api/v1/search with files/commits/refs,
⌘K scopes + recent queries, unified/split diffs with a page line budget,
/:repo/insights (calendar, contributors, hot paths, dependency manifests),
/:repo/research (opt-in, static analysis only), src/ai (BM25 index per blob,
grounded Q&A with verified citations, Anthropic SDK / OpenAI-compatible /
test-only fake, daily budget, 404 when disabled), @fastify/compress (br q4),
AA-compliant shiki themes, scripts/qa/{screens,axe}.py, security regression
tests, CHANGELOG, DEPLOY, BENCHMARKS, AGENT-WORKSPACE-DESIGN, CLAUDE.md/README
refresh. 86 tests, 0 overflow at 320–1920 (Chromium+WebKit), 0 axe violations.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
50 changed files +4,048 −419
added
CHANGELOG.md
+93 −0
@@ -0,0 +1,93 @@ | ||
| 1 | +<!-- | |
| 2 | + ───────────────────────────────────────────── | |
| 3 | + SPB Git — Personal Git Platform | |
| 4 | + ───────────────────────────────────────────── | |
| 5 | + Author : Simon-Pierre Boucher | |
| 6 | + Contact : contact@spboucher.ai | |
| 7 | + File : CHANGELOG.md | |
| 8 | + Purpose : Release notes | |
| 9 | + License : MIT © Simon-Pierre Boucher | |
| 10 | + ───────────────────────────────────────────── | |
| 11 | +--> | |
| 12 | +# Changelog | |
| 13 | + | |
| 14 | +## 2.0.0 — 2026-11-01 · "The Intelligent Git Forge" | |
| 15 | + | |
| 16 | +Plan and per-phase log: `docs/SPBGIT-2-PLAN.md`, `docs/SPBGIT-2-PROGRESS.md`. | |
| 17 | +Deployment and rollback: `docs/DEPLOY.md`. Benchmarks: `docs/BENCHMARKS.md`. | |
| 18 | + | |
| 19 | +### Reliability (production incident fixes) | |
| 20 | +- `/raw/*` streams blobs instead of buffering up to 64 MB; blob pages read | |
| 21 | + only size + an 8 KB binary probe for PDFs, images, binaries and large files. | |
| 22 | + Fixes the memory-restart loop (2 861 PM2 restarts, RSS up to 2.1 GB). | |
| 23 | +- Git subprocesses are bounded by a 12-slot semaphore (`SPBGIT_GIT_CONCURRENCY`). | |
| 24 | +- Overview, calendar and search caches are validated against live ref stamps, | |
| 25 | + so a push missed by the post-receive hook can no longer leave stale totals | |
| 26 | + ("all-time" 3 332 < "52 weeks" 3 346 is fixed; both come from the same pass). | |
| 27 | +- One compact log line per completed request; assets logged only on error. | |
| 28 | +- Prepared diffs, highlighted blobs, rendered Markdown blobs and blame are | |
| 29 | + cached per commit; brotli/gzip compression in-process. | |
| 30 | + | |
| 31 | +### Design 2.0 | |
| 32 | +- Three first-class themes — Light, Dark (charcoal/slate), Midnight (deep | |
| 33 | + navy) — following the OS by default, overridable and persisted. | |
| 34 | +- Semantic design tokens, 4/8 px spacing, tabular numerals, Bricolage display | |
| 35 | + type; mobile-first AppShell with a compact sticky header, navigation sheet, | |
| 36 | + bottom-sheet dialogs on phones, 44 px touch targets, safe-area insets, no | |
| 37 | + horizontal overflow at 320–1920 px (Chromium + WebKit). | |
| 38 | +- Component library (Nunjucks macros): icons, chips, stat strip, tabs, | |
| 39 | + breadcrumbs, ref picker, empty/error/skeleton states, copy button, toast, | |
| 40 | + confirmation dialog. | |
| 41 | +- Home: compact hero, stat strip, pinned, searchable catalog (deep-linkable | |
| 42 | + filters), activity, stack ring, calendar with metric definitions. | |
| 43 | +- Repository: compact stat strip, tabs (Code · Commits · Branches · Releases · | |
| 44 | + Insights · Research), clone sheet with selectable URLs, lazy expandable file | |
| 45 | + tree (desktop sidebar / phone sheet), README with table of contents. | |
| 46 | +- Source browser: sticky toolbar, wrap (default on phones), copy, raw, blame, | |
| 47 | + history, permalink, line ranges, markdown source/rendered toggle. | |
| 48 | +- Commits: timeline grouped by day; commit page with changed-file list, | |
| 49 | + unified **or split** diffs, wrap toggle, large diffs deferred (`?all=1`). | |
| 50 | +- Search: ⌘K palette with scopes (repos, files, commits, branches), recent | |
| 51 | + queries, keyboard + touch navigation; `/search` page groups everything; | |
| 52 | + `/api/v1/search` is permission-trivial (every repository is public) and | |
| 53 | + index status is exposed. | |
| 54 | +- `/about/metrics` documents every statistic; `/api/v1/stats` returns | |
| 55 | + 52-week totals and streaks with a link to the definitions. | |
| 56 | +- `/:repo/insights`: 52-week calendar, contributors, hottest paths, declared | |
| 57 | + dependencies parsed from manifests (npm, PyPI, Cargo, Go, Swift PM). | |
| 58 | + | |
| 59 | +### Research Mode (opt-in) | |
| 60 | +- `/:repo/research` (tab when `research: true` on the repository; preview | |
| 61 | + with a banner when LaTeX/BibTeX/figure signals are detected): manuscript PDF | |
| 62 | + viewer, LaTeX main documents with parsed title/authors/environment counts, | |
| 63 | + BibTeX, figure gallery, tables, reproducibility checklist, experiment | |
| 64 | + outputs, notebooks, tagged versions with per-version PDF/zip links. Nothing | |
| 65 | + is compiled or executed. | |
| 66 | + | |
| 67 | +### AI groundwork (opt-in, off by default) | |
| 68 | +- Provider-agnostic service (`src/ai/`): Anthropic (official SDK, server-side | |
| 69 | + key, server-side refusal fallbacks), OpenAI-compatible local servers, and a | |
| 70 | + test-only fake. Disabled providers are absent — nothing is simulated. | |
| 71 | +- Grounded repository Q&A (`POST /api/v1/repos/:name/ask`) over a BM25 chunk | |
| 72 | + index cached per blob (incremental across commits); answers must cite | |
| 73 | + `path#Lx-Ly`, citations are verified against the retrieved excerpts and | |
| 74 | + unverifiable ones are removed; untrusted content is wrapped as data. | |
| 75 | +- Commit summaries (`POST /api/v1/repos/:name/commits/:sha/summary`, cached). | |
| 76 | +- Daily token budget, per-route rate limit, concurrency cap, `/api/v1/ai/status`. | |
| 77 | + | |
| 78 | +### Agent Workspace | |
| 79 | +- Design only: `docs/AGENT-WORKSPACE-DESIGN.md` (scoped job tokens, | |
| 80 | + sandboxed runner, approval gates, audit log). Not implemented by design. | |
| 81 | + | |
| 82 | +### Tooling | |
| 83 | +- `scripts/qa/screens.py` (overflow + touch targets + screenshots across | |
| 84 | + widths/themes/engines) and `scripts/qa/axe.py` (WCAG 2.x A/AA audit), | |
| 85 | + both on Playwright for Python. | |
| 86 | +- Tests: 59 → 86 (semaphore, split rows, research/insights parsers, scoped | |
| 87 | + search, tree API, streamed raw, missed-hook coherence, AI with fake provider, | |
| 88 | + security regressions). | |
| 89 | + | |
| 90 | +### Removed | |
| 91 | +- Instrument Serif font, constellation canvas, typewriter terminal, count-up | |
| 92 | + numerals, mouse spotlight, scroll reveal (decorative motion that cost CPU on | |
| 93 | + phones). `.stat`/`.rstat` tiles replaced by the compact stat strip. | |
modified
CLAUDE.md
+69 −18
@@ -12,8 +12,8 @@ | ||
| 12 | 12 | | **Product name** | SPB Git | |
| 13 | 13 | | **Owner / sole user** | Simon-Pierre Boucher | |
| 14 | 14 | | **Contact** | [contact@spboucher.ai](mailto:contact@spboucher.ai) | |
| 15 | −| **Public domain** | `https://git.spboucher.ai` (ngrok custom domain) | | |
| 16 | −| **Deployment host** | Node **m3u96a** | | |
| 15 | +| **Public domain** | `https://git.spboucher.ai` (MacLustr Tunnel — WireGuard + Caddy on BHS64; ngrok retired 2026-09-10) | | |
| 16 | +| **Deployment host** | **M1M32** (MacLustr gateway) — `~/srv/spbgit/app`, PM2 `spbgit-server`, `mld` manifest pinned; see `docs/DEPLOY.md` | | |
| 17 | 17 | | **Runtime** | Node.js ≥ 20, ESM only | |
| 18 | 18 | | **Visibility model** | **ALL repositories are PUBLIC (read-only)**. Write access = owner only. No sign-up, no visitor accounts, ever. | |
| 19 | 19 | |
@@ -175,18 +175,55 @@ On every push, the hook POSTs to `http://127.0.0.1:7420/internal/hooks/post-rece | ||
| 175 | 175 | |
| 176 | 176 | ## 4. Web UI — "It must look sharp" (Design Spec) |
| 177 | 177 | |
| 178 | −### 4.1 Design system | |
| 179 | − | |
| 180 | −- **Aesthetic (v4 "Monolith", 2026-09-08):** cinematic developer-forge. Obsidian canvas with a drifting gradient mesh + faint grid + film grain (`.bg` stack), glass surfaces (`--surface-grad` + `backdrop-filter`), electric accents (indigo `#86a4ff`, cyan `#46d6ff`, mint `#3ff0bd`, violet `#c996ff`) and an **ember** counterpoint (`#ff8a3d → #ffc857`) reserved for activity/heat (streaks, heatmap top level, latest release). Dark by default, light theme toggle (persisted in `localStorage`, respects `prefers-color-scheme` on first visit). Design history: v1 flat → v2 gradients → v3 "Nightforge" aurora → v4 "Monolith". | |
| 181 | −- **Design tokens** (CSS custom properties in `assets/css/tokens.css`): `--bg #04060b`, `--surface #0b0f18`, translucent `--border`, `--text #e6e9f2`, `--muted #8f97ad`, `--accent`, `--accent-2`, `--accent-3`, `--cyan`, `--ember`, `--ember-2`, gradients (`--gradient-brand/hot/ember/text`), heat scale `--heat-0…4` (cold → ember), radii `8/12/18/26px`, depth `--shadow-1/2/card/pop`, `--ring`, `--glass`, `--nav-bg`. | |
| 182 | −- **Typography:** `Inter` (UI) + `JetBrains Mono` (code) + **`Bricolage Grotesque` 700/800** (display: hero name, section titles, stat numerals, repo titles) + **`Instrument Serif` italic** (accent: hero tagline, error message). All self-hosted woff2 in `assets/fonts/` via `scripts/fetch-fonts.mjs` — no third-party font CDNs. Bricolage 800 woff also feeds satori for OG cards. | |
| 183 | −- **Motion:** cross-document view transitions (`@view-transition { navigation: auto }`), scroll reveal (`.reveal` + IntersectionObserver), count-up numerals (`[data-count]`), mouse spotlight on any `.spot` card, hero constellation canvas (`#hero-canvas`, drifting git-graph nodes, paused when hidden) and terminal typewriter (`#hero-terminal`). Everything honours `prefers-reduced-motion`. | |
| 184 | −- **Layout:** max-width `1240px`, sticky glass top nav: monogram + "**SPB Git**" wordmark + `forge` tag, primary links (Repositories · Activity · API), a search trigger that opens the **⌘K palette** (`/` or `⌘K` anywhere; repos + commands + "Search …"), theme toggle, link to `spboucher.ai`. | |
| 185 | −- **Data widgets:** every repo card carries a 26-week commit **sparkline** (`overview.spark`, computed server-side in `lib/overview.mjs`, cached per repo); the home hero shows 5 stat tiles (repos, commits, languages, commits/52 weeks, day streak); the sidebar has a live activity feed and a **Stack ring** (`languageStack`, equal weight per repo); the heatmap card reports streak / longest streak / active days / busiest day (`stats/activity.mjs → streaks()`); commit history is grouped by day (`routes.mjs → groupByDay`); README pages get a sticky **table of contents** rail (client-side, ≥3 headings, ≥1100px). | |
| 186 | −- **Assets are cache-busted** with `?v={{ assetVersion }}` (package.json version) — bump the version on every design release. | |
| 187 | −- ⚠️ Cards with sticky headers (`.blob-card`, `.diff-file`, `.readme-section`) use `overflow: clip`, never `overflow: hidden` — `hidden` turns the card into the sticky element's scrollport and pushes the header down by its `top` offset. | |
| 188 | −- Fully responsive down to 360px. Code views scroll horizontally, never break the page. | |
| 189 | −- Footer on every page: 4-column grid (about + clone snippet · Explore · Machine · Owner) and a bottom bar `© Simon-Pierre Boucher · Powered by SPB Git v<version>` with a live-status dot. | |
| 178 | +### 4.1 Design system (v2.0 "Intelligent Git Forge", 2026-11-01) | |
| 179 | + | |
| 180 | +- **Themes:** three first-class themes in `assets/css/tokens.css` — `light` | |
| 181 | + (soft blue-white), `dark` (charcoal/slate), `midnight` (deep navy, electric | |
| 182 | + blue, restrained turquoise — successor of v4 "Monolith"). `theme-boot.js` | |
| 183 | + resolves `auto|light|dark|midnight` from `localStorage['spbgit-theme']` | |
| 184 | + (absent = follow the OS) before first paint; `app.js` syncs the header | |
| 185 | + button, the menu sheet and the ⌘K "Switch theme" command. The brand (S mark, | |
| 186 | + blue→teal gradient, static grid/mesh motif) is constant across themes; | |
| 187 | + gradients are reserved for the hero title, primary buttons and tab | |
| 188 | + underlines. | |
| 189 | +- **Tokens:** semantic only — `--bg`, `--surface-0..3`, `--fg/--fg-muted/--fg-faint`, | |
| 190 | + `--border(-strong)`, `--accent(-hover/-soft)`, `--accent-2`, `--ember`, | |
| 191 | + status `--ok/--warn/--danger/--info` (+`-soft`), diff colours, `--heat-0..4`, | |
| 192 | + spacing `--space-1..10` (4/8 px), radii, shadows, `--ring`, motion | |
| 193 | + `--dur*`/`--ease*`, `--touch: 44px`, `--header-h`. Legacy v4 aliases | |
| 194 | + (`--text`, `--muted`, `--surface`…) remain for `markdown.css`. Every theme is | |
| 195 | + checked with `scripts/qa/axe.py` (WCAG AA). | |
| 196 | +- **Typography:** Inter (UI) · JetBrains Mono (code) · Bricolage Grotesque | |
| 197 | + 700/800 (display: hero, section titles, stat values, repo title). Tabular | |
| 198 | + numerals on every statistic. Fonts self-hosted (`scripts/fetch-fonts.mjs`), | |
| 199 | + three preloads. Instrument Serif was removed in 2.0. | |
| 200 | +- **Layout / AppShell (`layout.njk`):** compact sticky header (56 px desktop, | |
| 201 | + 52 px phone, safe-area aware) with brand, primary links, search trigger | |
| 202 | + (⌘K / `/`), theme button, menu button on phones; `<dialog>`-based | |
| 203 | + navigation sheet, theme popover, ⌘K palette, clone panel and file-tree sheet | |
| 204 | + (bottom sheets < 768 px, modals above); 4-column footer. Max width 1240 px, | |
| 205 | + gutters 24/16 px. | |
| 206 | +- **Mobile-first contract:** base styles are the phone layout; enhancements at | |
| 207 | + 560/768/1024/1280 px. Rules: every grid track is `minmax(0, 1fr)`, scroll | |
| 208 | + containers (`.stat-strip`, `.tabs-track`, `.code-view`, `.diff-table-wrap`, | |
| 209 | + `.heatmap-scroll`, `.blob-actions`) are bounded and keyboard-focusable, | |
| 210 | + essential controls are ≥ 44 px on coarse pointers, no hover-only actions, | |
| 211 | + `100dvh`, reduced-motion respected (mesh drift only ≥ 768 px). Verified by | |
| 212 | + `python3 -I scripts/qa/screens.py <url> <out>` — **run it before any CSS | |
| 213 | + change ships**; it must report `OK — 0 failing`. | |
| 214 | +- **Component library (`views/components/ui.njk`):** `icon`, `chip`, | |
| 215 | + `statStrip`, `tabs`, `breadcrumbs`, `refSelect`, `emptyState`, `errorState`, | |
| 216 | + `skeleton`, `copyButton`; client primitives in `app.js`: `spbToast`, | |
| 217 | + `spbConfirm`, dialogs, clipboard, file tree, code/diff helpers, ⌘K. | |
| 218 | +- **Data widgets:** repo cards keep the 26-week sparkline; stat strips replace | |
| 219 | + the v4 tiles; the heatmap card and `/about/metrics` state every definition | |
| 220 | + (commit totals are coherent by construction — one `rev-list` pass per repo, | |
| 221 | + caches validated against ref stamps). | |
| 222 | +- **Assets are cache-busted** with `?v={{ assetVersion }}` (package.json | |
| 223 | + version) — bump the version on every release. | |
| 224 | +- ⚠️ Cards with sticky headers (`.blob-card`, `.diff-file`, `.readme-section`) | |
| 225 | + use `overflow: clip`, never `overflow: hidden`. | |
| 226 | +- Code views scroll horizontally inside their card, never the page. | |
| 190 | 227 | |
| 191 | 228 | ### 4.2 Pages |
| 192 | 229 | |
@@ -221,7 +258,13 @@ On every push, the hook POSTs to `http://127.0.0.1:7420/internal/hooks/post-rece | ||
| 221 | 258 | |
| 222 | 259 | **`/:repo/branches`, `/:repo/tags`** — lists with ahead/behind counts vs default branch (branches) and archive download links (tags → releases-lite). |
| 223 | 260 | |
| 224 | −**`/search?q=`** — global search across repo names, descriptions, topics, and README content (build a tiny inverted index in cache; refresh on push). Grouped results. | |
| 261 | +**`/search?q=`** — global search across repo names, descriptions, topics, README content, **file paths, commit messages and refs** (`lib/search.mjs`, index validated against ref stamps; `/api/v1/search?q&scope&repo&limit` powers the ⌘K palette). Grouped results. | |
| 262 | + | |
| 263 | +**`/:repo/insights`** — 52-week calendar, contributors, hottest paths, declared dependencies parsed from manifests (`insights/insights.mjs`). | |
| 264 | + | |
| 265 | +**`/:repo/research`** — opt-in Research Mode (`research: true` via `PATCH /api/v1/repos/:name`; preview when LaTeX/BibTeX/figure signals are detected): manuscript PDF, LaTeX main documents, figures, tables, reproducibility checklist, outputs, versions (`research/profile.mjs`; never compiles or executes). | |
| 266 | + | |
| 267 | +**`/about/metrics`** — definitions of every statistic. | |
| 225 | 268 | |
| 226 | 269 | **Error pages** — custom 404/500 with the design system (404: "This ref does not exist in any timeline."). |
| 227 | 270 | |
@@ -309,12 +352,18 @@ Public reads, token-gated writes (`Authorization: Bearer <PAT>`). | ||
| 309 | 352 | - `GET /api/v1/repos` · `GET /api/v1/repos/:name` · `GET /api/v1/repos/:name/languages` · `GET /api/v1/repos/:name/commits?ref&page` |
| 310 | 353 | - `POST /api/v1/repos` · `PATCH /api/v1/repos/:name` · `DELETE /api/v1/repos/:name` (auth) |
| 311 | 354 | - `POST /api/v1/tokens` · `GET /api/v1/tokens` · `DELETE /api/v1/tokens/:id` (auth; creation of the *first* token happens server-side via `deploy/setup` script printing a bootstrap token once) |
| 312 | −- `GET /api/v1/stats` — totals for the home hero. | |
| 355 | +- `GET /api/v1/stats` — totals for the home hero (`commits`, `commitsLast52Weeks`, streaks, `definitions` link). | |
| 356 | +- `GET /api/v1/search?q&scope=all|repos|files|commits|branches|readmes&repo&limit` · `GET /api/v1/repos/:name/tree?ref&path` (lazy file tree). | |
| 357 | +- **AI (opt-in, 404 unless `SPBGIT_AI_PROVIDER` is set):** `GET /api/v1/ai/status` · `POST /api/v1/repos/:name/ask {question, ref}` · `POST /api/v1/repos/:name/commits/:sha/summary` — 12 req/min/IP, daily token budget, grounded answers with verified `path#Lx-Ly` citations (`src/ai/`). Providers: `anthropic` (official SDK), `openai-compatible` (local MLX/llama.cpp), `fake` (test only). Never simulate a provider. | |
| 313 | 358 | - Rate limit public endpoints (200 req/min/IP, `@fastify/rate-limit`), sane JSON error envelope `{ "error": { "code", "message" } }`. |
| 314 | 359 | |
| 315 | 360 | --- |
| 316 | 361 | |
| 317 | −## 8. Deployment — node m3u96a + ngrok + `git.spboucher.ai` | |
| 362 | +## 8. Deployment — M1M32 + MacLustr Tunnel + `git.spboucher.ai` | |
| 363 | + | |
| 364 | +> **Current (2026-11-01):** production runs on **M1M32** from `~/srv/spbgit/app` (a git clone of this repository), managed by PM2 through the `mld` manifest `~/dispatch/apps/spbgit.json`, exposed by the MacLustr Tunnel (Caddy on BHS64 → `wg1` → M1M32:7420). **Deploy/rollback procedure: `docs/DEPLOY.md`.** The ngrok material below is historical; `deploy/ngrok.yml` and the tunnel app in `deploy/ecosystem.config.cjs` are unused. | |
| 365 | + | |
| 366 | +### 8.0 Historical — node m3u96a + ngrok | |
| 318 | 367 | |
| 319 | 368 | ### 8.1 `deploy/setup-m3u96a.sh` (idempotent) |
| 320 | 369 | |
@@ -349,7 +398,8 @@ endpoints: | ||
| 349 | 398 | |
| 350 | 399 | - `deploy/backup.sh` — nightly cron: `tar.zst` of `/srv/git` + `data/` into `/srv/spbgit/backups/`, keep 14, log result. |
| 351 | 400 | - `GET /healthz` (uptime, repo count, cache size) for monitoring; pm2 healthcheck hits it. |
| 352 | −- Structured logs (pino), request logging with latency, log rotation. | |
| 401 | +- Structured logs (pino): one compact line per completed request (`method,url,status,ms,ip`), assets only on error; install `pm2-logrotate` (see `docs/DEPLOY.md`). | |
| 402 | +- Reliability rules (2.0): `/raw` streams (`Repos.blobStream`), views probe blobs with `Repos.blobInfo` before reading, git subprocesses are capped by `gitSemaphore` (`SPBGIT_GIT_CONCURRENCY`, default 12), caches that depend on refs carry a `refStamp` and are rebuilt when it changes — never trust the hook alone. | |
| 353 | 403 | |
| 354 | 404 | --- |
| 355 | 405 | |
@@ -381,6 +431,7 @@ endpoints: | ||
| 381 | 431 | - [ ] Every generated file in the codebase carries the **Simon-Pierre Boucher / contact@spboucher.ai** header (`npm run check:headers` passes). |
| 382 | 432 | - [ ] OG image cards, sitemap, Atom feed, custom 404 — all present. |
| 383 | 433 | - [ ] Lighthouse ≥ 95 / 95 on home and a repo page. |
| 434 | +- [ ] `python3 -I scripts/qa/screens.py` and `scripts/qa/axe.py` report 0 failures (mobile contract + WCAG AA) — see `docs/SPBGIT-2-PROGRESS.md`. | |
| 384 | 435 | |
| 385 | 436 | --- |
| 386 | 437 | |
modified
README.md
+11 −6
@@ -18,7 +18,8 @@ | ||
| 18 | 18 | [](https://nodejs.org) |
| 19 | 19 | [](LICENSE) |
| 20 | 20 | [](https://fastify.dev) |
| 21 | −[](https://git.spboucher.ai) | |
| 21 | +[](https://git.spboucher.ai) | |
| 22 | +[](CHANGELOG.md) | |
| 22 | 23 | |
| 23 | 24 | Self-hosted software forge · every repo public & clonable · owner-only writes |
| 24 | 25 | |
@@ -33,7 +34,7 @@ Self-hosted software forge · every repo public & clonable · owner-only writes | ||
| 33 | 34 | SPB Git is a complete, self-hosted GitHub-equivalent built for one person: |
| 34 | 35 | |
| 35 | 36 | - **Git hosting** — Smart HTTP v2 (streamed `upload-pack`/`receive-pack`), anonymous clone/fetch, PAT-authenticated push, post-receive hooks, zip/tar.gz snapshots. |
| 36 | −- **Showcase web UI** (design v4 « Monolith » : mesh + grid background, display typography Bricolage Grotesque / Instrument Serif, sparklines per repo, stat tiles, Stack ring, streaks, ⌘K commands, README table of contents, view transitions) — SSR (Nunjucks), dark/light themes, pinned repos, activity feed, contribution heatmap, GitHub-grade README rendering (badges, mermaid, task lists, relative images), shiki-highlighted file browsing with line anchors, commits, diffs, blame, branches, tags, global search, OG cards, Atom feed, sitemap. | |
| 37 | +- **Showcase web UI 2.0** — mobile-first, three themes (Light · Dark · Midnight), SSR (Nunjucks) with a small vanilla-JS layer: compact stat strips, pinned repos, searchable catalog, activity feed, contribution calendar with documented metrics (`/about/metrics`), GitHub-grade README rendering (badges, mermaid, task lists, relative images), lazy file tree, shiki-highlighted blobs with line anchors, commits, unified/split diffs, blame, branches, releases, per-repo insights, opt-in Research Mode, ⌘K search over repos/files/commits/branches, OG cards, Atom feed, sitemap. See `CHANGELOG.md` and `docs/`. | |
| 37 | 38 | - **`spbgit` CLI** — the terminal command center: `init`, `token`, `list`, `create --push`, `clone --all`, `status`, `commit`, `push`, `pull`, `sync`, `open`, `info`, `rm`, `doctor`, `release`. |
| 38 | 39 | - **Releases** — GitHub-style binary assets (dmg, pkg, zip…) attached to tags, streamed to disk with sha256 sidecars, public downloads at `/releases/<repo>/<tag>/<file>`: |
| 39 | 40 | ```bash |
@@ -41,7 +42,8 @@ SPB Git is a complete, self-hosted GitHub-equivalent built for one person: | ||
| 41 | 42 | spbgit release upload myapp v1.0.0 MyApp.dmg # → https://git.spboucher.ai/releases/myapp/v1.0.0/MyApp.dmg |
| 42 | 43 | spbgit release list myapp |
| 43 | 44 | ``` |
| 44 | −- **JSON API** — public reads, token-gated writes at `/api/v1/*`. | |
| 45 | +- **JSON API** — public reads, token-gated writes at `/api/v1/*`; `/api/v1/search`, `/api/v1/repos/:name/tree`. | |
| 46 | +- **AI (opt-in)** — grounded repository Q&A and commit summaries with verified citations, provider-agnostic (`SPBGIT_AI_PROVIDER=anthropic|openai-compatible`), disabled by default. | |
| 45 | 47 | |
| 46 | 48 | **Filesystem is the database.** Bare repos under the git root are the source of truth; `meta.json` holds descriptions/topics/pins; everything expensive is cached per `<repo>@<sha>` and busted on push. |
| 47 | 49 | |
@@ -67,10 +69,11 @@ spbgit init # paste server URL + token | ||
| 67 | 69 | spbgit create hello --push # first repo, live immediately |
| 68 | 70 | ``` |
| 69 | 71 | |
| 70 | −## Deployment (node m3u96a) | |
| 72 | +## Deployment (M1M32, MacLustr gateway) | |
| 73 | + | |
| 74 | +Production is the clone at `M1M32:~/srv/spbgit/app` under PM2 (`mld` manifest). **Procedure, verification and rollback: `docs/DEPLOY.md`.** The historical bootstrap below targets a fresh machine: | |
| 71 | 75 | |
| 72 | 76 | ```bash |
| 73 | −# on m3u96a, from the app directory | |
| 74 | 77 | bash deploy/setup-m3u96a.sh |
| 75 | 78 | ``` |
| 76 | 79 | |
@@ -92,7 +95,9 @@ A systemd unit is provided as the documented alternative: `deploy/spbgit.service | ||
| 92 | 95 | | `npm run lint` | eslint | |
| 93 | 96 | | `npm run check:headers` | fail if any file lacks the author header | |
| 94 | 97 | | `npm run inject:headers` | add missing headers in bulk | |
| 95 | −| `npm run fetch:fonts` | (re)download self-hosted Inter / JetBrains Mono | | |
| 98 | +| `npm run fetch:fonts` | (re)download self-hosted Inter / JetBrains Mono / Bricolage | | |
| 99 | +| `python3 -I scripts/qa/screens.py URL OUT` | responsive QA: overflow + touch targets + screenshots (Playwright for Python) | | |
| 100 | +| `python3 -I scripts/qa/axe.py URL` | WCAG 2.x A/AA audit with axe-core | | |
| 96 | 101 | |
| 97 | 102 | ## Security model |
| 98 | 103 | |
added
docs/AGENT-WORKSPACE-DESIGN.md
+125 −0
@@ -0,0 +1,125 @@ | ||
| 1 | +<!-- | |
| 2 | + ───────────────────────────────────────────── | |
| 3 | + SPB Git — Personal Git Platform | |
| 4 | + ───────────────────────────────────────────── | |
| 5 | + Author : Simon-Pierre Boucher | |
| 6 | + Contact : contact@spboucher.ai | |
| 7 | + File : docs/AGENT-WORKSPACE-DESIGN.md | |
| 8 | + Purpose : Phase 6 — design (not implementation) of the security-gated Agent Workspace | |
| 9 | + License : MIT © Simon-Pierre Boucher | |
| 10 | + ───────────────────────────────────────────── | |
| 11 | +--> | |
| 12 | +# Agent Workspace — design document (Phase 6, not shipped) | |
| 13 | + | |
| 14 | +**Status: design only.** Per the SPB Git 2.0 directive, agentic code changes | |
| 15 | +ship only once isolation, scoped permissions, approval gates and auditability | |
| 16 | +can be demonstrated. None of that exists on the M1M32 gateway today (the forge | |
| 17 | +runs as the owner's user with full access to every bare repository), so this | |
| 18 | +phase deliberately delivers a design and the preconditions, not a feature. | |
| 19 | + | |
| 20 | +## 1. Goal | |
| 21 | + | |
| 22 | +Let the owner ask an agent, from an issue-like request, to analyse a | |
| 23 | +repository, propose a code change with tests, and open a reviewable change — | |
| 24 | +without ever giving the agent a shell on the gateway or write access to | |
| 25 | +`~/srv/git`. | |
| 26 | + | |
| 27 | +## 2. Non-goals (explicit) | |
| 28 | + | |
| 29 | +- No automatic merge, no automatic deploy, no push to a default branch. | |
| 30 | +- No agent execution on the gateway host (M1M32) — ever. The gateway holds | |
| 31 | + every repository, the PAT store and the tunnel key. | |
| 32 | +- No access to private data: SPB Git has none (every repository is public), but | |
| 33 | + `data/tokens.json`, `data/ai-usage.json`, `.env` and the tunnel configuration | |
| 34 | + must be unreachable from the sandbox. | |
| 35 | + | |
| 36 | +## 3. Architecture | |
| 37 | + | |
| 38 | +``` | |
| 39 | +owner ──(PAT, approval UI)──▶ SPB Git (M1M32) | |
| 40 | + │ 1. creates a job record (audit log) | |
| 41 | + │ 2. issues a *scoped, short-lived* job token | |
| 42 | + ▼ | |
| 43 | + Agent runner (separate node, e.g. m4mn) | |
| 44 | + ┌──────────────────────────────────────┐ | |
| 45 | + │ container / sandbox per job │ | |
| 46 | + │ - clone over HTTPS (anonymous) │ | |
| 47 | + │ - no network except git.spboucher.ai│ | |
| 48 | + │ and the configured model provider │ | |
| 49 | + │ - CPU/RAM/time limits, no sudo │ | |
| 50 | + │ - writes only inside the workspace │ | |
| 51 | + └───────────────┬──────────────────────┘ | |
| 52 | + │ 3. pushes to refs/agents/<job>/* | |
| 53 | + │ using the job token (branch-scoped) | |
| 54 | + ▼ | |
| 55 | + SPB Git: "Proposed change" page (diff, tests, logs) | |
| 56 | + │ 4. owner approves → fast-forward to | |
| 57 | + │ a normal branch or rejects | |
| 58 | +``` | |
| 59 | + | |
| 60 | +### 3.1 Scoped job tokens (precondition in SPB Git) | |
| 61 | + | |
| 62 | +Today a PAT grants every write. The workspace requires a new token class: | |
| 63 | + | |
| 64 | +| Field | Meaning | | |
| 65 | +|---|---| | |
| 66 | +| `scope.repo` | exactly one repository | | |
| 67 | +| `scope.refs` | glob, e.g. `refs/agents/<job-id>/*` — `receive-pack` rejects any other ref via a `pre-receive` hook that validates the token id passed in `GIT_AUTH_TOKEN_ID` | | |
| 68 | +| `expires` | ≤ 2 h | | |
| 69 | +| `capabilities` | `push:agent-ref` only — no API writes, no release uploads, no token management | | |
| 70 | + | |
| 71 | +Implementation sketch: `TokenStore.create(label, {scope})`, `smart-http` | |
| 72 | +exports the verified token id to the git environment, and a `pre-receive` | |
| 73 | +hook (installed next to `post-receive`) checks `refs` against `scope.refs`. | |
| 74 | + | |
| 75 | +### 3.2 Isolation (precondition on the runner node) | |
| 76 | + | |
| 77 | +- One job = one fresh container (Docker on a Mac mini, or an Apptainer/`sandbox-exec` | |
| 78 | + profile as used by UQO-Chat). Root filesystem read-only except `/work`. | |
| 79 | +- Egress allow-list: `git.spboucher.ai:443` and the AI provider host only. | |
| 80 | +- Resource limits: 2 CPU, 4 GB, 30 min wall clock, 2 GB disk; the runner kills | |
| 81 | + the container on breach and records the reason. | |
| 82 | +- No secrets in the container except the job token (environment variable, | |
| 83 | + revoked when the job ends). | |
| 84 | + | |
| 85 | +### 3.3 Approval gates | |
| 86 | + | |
| 87 | +| Action | Gate | | |
| 88 | +|---|---| | |
| 89 | +| Start a job | owner, authenticated with a full PAT, from the web UI or `spbgit agent run <repo> "<task>"` | | |
| 90 | +| Agent pushes `refs/agents/<job>/proposal` | automatic (scoped token) | | |
| 91 | +| Promote proposal to a real branch | owner approval in the UI (`POST /api/v1/repos/:name/agent-jobs/:id/approve`) | | |
| 92 | +| Merge / deploy | **out of scope**; the owner does it with normal tooling | | |
| 93 | +| Delete a job's refs | owner, with a confirmation dialog (`spbConfirm`) | | |
| 94 | + | |
| 95 | +### 3.4 Audit log | |
| 96 | + | |
| 97 | +Append-only `data/agent-jobs.jsonl`: job id, repository, requester token id, | |
| 98 | +task text, provider + model, container id, start/end, exit reason, token spend, | |
| 99 | +refs written, approval decision + approver token id. Exposed read-only at | |
| 100 | +`/api/v1/agent-jobs` (owner PAT) and on the proposal page. | |
| 101 | + | |
| 102 | +### 3.5 Prompt-injection posture | |
| 103 | + | |
| 104 | +The agent reads untrusted repository content. The same defenses as Phase 5 | |
| 105 | +apply (content wrapped as data, instructions in the system prompt only), plus: | |
| 106 | +the agent cannot read anything outside its workspace, cannot reach the network | |
| 107 | +except the two allowed hosts, and cannot write outside `refs/agents/<job>/*`. | |
| 108 | +A hostile repository can therefore waste a job's budget but cannot exfiltrate | |
| 109 | +or damage anything. | |
| 110 | + | |
| 111 | +## 4. Acceptance criteria before any implementation | |
| 112 | + | |
| 113 | +1. Scoped job tokens exist and are covered by e2e tests proving that a job | |
| 114 | + token cannot push to `refs/heads/*`, cannot call `/api/v1/repos` writes, and | |
| 115 | + expires. | |
| 116 | +2. A runner node is prepared (`mld prepare`), the sandbox profile denies all | |
| 117 | + egress except the allow-list, and a test job proves the file-system and | |
| 118 | + network limits. | |
| 119 | +3. The audit log and the proposal page exist and are covered by tests. | |
| 120 | +4. The owner has reviewed this document and approved the runner node choice. | |
| 121 | + | |
| 122 | +## 5. Estimated size | |
| 123 | + | |
| 124 | +Tokens + pre-receive hook: ~2 days. Runner (containers, limits, job protocol): | |
| 125 | +~3 days. UI (proposal page, approval): ~2 days. Not started. | |
added
docs/BENCHMARKS.md
+81 −0
@@ -0,0 +1,81 @@ | ||
| 1 | +<!-- | |
| 2 | + ───────────────────────────────────────────── | |
| 3 | + SPB Git — Personal Git Platform | |
| 4 | + ───────────────────────────────────────────── | |
| 5 | + Author : Simon-Pierre Boucher | |
| 6 | + Contact : contact@spboucher.ai | |
| 7 | + File : docs/BENCHMARKS.md | |
| 8 | + Purpose : SPB Git 2.0 — before/after measurements (production baseline, local A/B of 1.1.0 vs 2.0.0) | |
| 9 | + License : MIT © Simon-Pierre Boucher | |
| 10 | + ───────────────────────────────────────────── | |
| 11 | +--> | |
| 12 | +# Benchmarks — SPB Git 1.1.0 → 2.0.0 | |
| 13 | + | |
| 14 | +All numbers measured on 2026-11-01 with Playwright Chromium. "Mobile" = | |
| 15 | +375×812, CPU ×4 slowdown, 1.6 Mb/s / 150 ms RTT. "Desktop" = 1440×900, no | |
| 16 | +throttling. CLS was 0 on every page in both versions. | |
| 17 | + | |
| 18 | +## A. Production baseline (1.1.0 on M1M32 behind the tunnel, gzip at the edge) | |
| 19 | + | |
| 20 | +| Page | Mobile LCP | Mobile TTFB | Desktop LCP | HTML (gz) | CSS (gz) | JS (gz) | Fonts | | |
| 21 | +|---|---|---|---|---|---|---|---| | |
| 22 | +| `/` (157 repos) | 1 216 ms | 182 ms | 320 ms | 34.0 KB | 18.7 KB | 9.0 KB | 208.8 KB (5 files) | | |
| 23 | +| `/spbgit` | 276 ms | 144 ms | 236 ms | 9.7 KB | — | — | — | | |
| 24 | +| blob `src/server.mjs` | 552 ms | 424 ms | 328 ms | 13.2 KB | — | — | — | | |
| 25 | +| commit `810e402` (60 files) | 1 612 ms | **1 463 ms** | 1 336 ms | 120.6 KB | — | — | — | | |
| 26 | + | |
| 27 | +Server, warm: `/` 64 ms · repo 58 ms · `/api/v1/repos` 10 ms. PM2: 2 861 | |
| 28 | +memory restarts, RSS peaks 0.9–2.1 GB, 599 MB request log. | |
| 29 | + | |
| 30 | +## B. Local A/B — same machine, same 9-repo dev dataset, warm caches | |
| 31 | + | |
| 32 | +1.1.0 ran from a git worktree (`810e402`) on port 7422, 2.0.0 on 7421. 1.1.0 | |
| 33 | +has no in-process compression (production relies on the edge); 2.0.0 compresses | |
| 34 | +in-process (brotli q4). Column "wire" is what the browser transferred. | |
| 35 | + | |
| 36 | +| Page | Version | Mobile LCP | Mobile DCL | Desktop LCP | Wire HTML | Server avg (10 req) | | |
| 37 | +|---|---|---|---|---|---|---| | |
| 38 | +| `/` | 1.1.0 | 1 632 ms | 1 685 ms | 144 ms | 98.3 KB (raw) | 7 ms | | |
| 39 | +| `/` | **2.0.0** | **1 572 ms** | **882 ms** | **48 ms** | **12.6 KB (br)** | **3 ms** | | |
| 40 | +| `/spbgit` | 1.1.0 | 568 ms | 673 ms | 88 ms | 43.6 KB | 37 ms | | |
| 41 | +| `/spbgit` | **2.0.0** | **180 ms** | **221 ms** | **68 ms** | **12.2 KB** | 33 ms | | |
| 42 | +| blob `src/server.mjs` | 1.1.0 | 568 ms | 876 ms | 172 ms | 102.9 KB | 105 ms | | |
| 43 | +| blob `src/server.mjs` | **2.0.0** | **188 ms** | **241 ms** | 168 ms | **14.1 KB** | 116 ms | | |
| 44 | +| commit `810e402` | 1.1.0 | 1 132 ms | 15 267 ms | 868 ms | 3 004.9 KB | 679 ms | | |
| 45 | +| commit `810e402` | **2.0.0** | **220 ms** | **697 ms** | **168 ms** | **55.1 KB** | **44 ms** | | |
| 46 | +| commits list | 1.1.0 | 600 ms | 715 ms | 136 ms | 46.9 KB | 73 ms | | |
| 47 | +| commits list | **2.0.0** | **220 ms** | **216 ms** | 124 ms | **10.0 KB** | 77 ms | | |
| 48 | + | |
| 49 | +Static assets over the wire (brotli): tokens.css 3.2 KB, app.css 12.3 KB, | |
| 50 | +markdown.css 1.8 KB, app.js 10.3 KB, theme-boot.js 0.5 KB. Fonts 186 KB | |
| 51 | +(4 preloaded woff2 files; Instrument Serif dropped). | |
| 52 | + | |
| 53 | +Cold cache (first home request after wiping the cache): 1.1.0 304 ms, 2.0.0 | |
| 54 | +381 ms for 9 repositories (2.0 also computes research signals and ref | |
| 55 | +stamps). Production will take ~10 s once after deploy for 157 repositories. | |
| 56 | + | |
| 57 | +## C. Why the commit page changed so much | |
| 58 | + | |
| 59 | +1.1.0 highlighted every hunk on every request (679 ms server time) and shipped | |
| 60 | +all 60 files (3.0 MB raw). 2.0.0 caches the prepared diff per sha | |
| 61 | +(`commit-v3.json`), renders files until a 1 500-line page budget and defers | |
| 62 | +the rest behind "Load diff" / "Load all files" (`?all=1`, 1.8 MB raw / 70 KB | |
| 63 | +br), and compresses in-process. | |
| 64 | + | |
| 65 | +## D. Memory | |
| 66 | + | |
| 67 | +2.0.0 idle after the QA runs: RSS 313 MB, heap 89 MB (`/healthz`). The 1.1.0 | |
| 68 | +production process reached 750 MB+ within minutes of crawler traffic on | |
| 69 | +`/raw` and blob pages; 2.0.0 streams those (3 MB test binary: exact bytes, | |
| 70 | +`Content-Length` from `cat-file -s`, no buffer) and caps git at 12 processes. | |
| 71 | +Production confirmation of the restart count is part of the post-deploy | |
| 72 | +verification in `docs/DEPLOY.md`. | |
| 73 | + | |
| 74 | +## E. Budgets (75th percentile, representative mobile) | |
| 75 | + | |
| 76 | +Targets from the directive: LCP ≤ 2.5 s, INP ≤ 200 ms, CLS ≤ 0.1. Local | |
| 77 | +results meet LCP and CLS on every page; INP is not measurable without real | |
| 78 | +interaction traces — the interactive surfaces (palette, sheets, copy, tree) | |
| 79 | +run no layout-thrashing work and were manually exercised on the throttled | |
| 80 | +profile without visible lag. Re-measure on production after deployment with | |
| 81 | +the same `scripts/qa` recipes (see `docs/SPBGIT-2-PROGRESS.md` Phase 7). | |
added
docs/DEPLOY.md
+111 −0
@@ -0,0 +1,111 @@ | ||
| 1 | +<!-- | |
| 2 | + ───────────────────────────────────────────── | |
| 3 | + SPB Git — Personal Git Platform | |
| 4 | + ───────────────────────────────────────────── | |
| 5 | + Author : Simon-Pierre Boucher | |
| 6 | + Contact : contact@spboucher.ai | |
| 7 | + File : docs/DEPLOY.md | |
| 8 | + Purpose : SPB Git 2.0 — deployment on M1M32 (MacLustr gateway), rollback, verification | |
| 9 | + License : MIT © Simon-Pierre Boucher | |
| 10 | + ───────────────────────────────────────────── | |
| 11 | +--> | |
| 12 | +# Deploying SPB Git 2.0 to M1M32 | |
| 13 | + | |
| 14 | +Production is the git clone at `M1M32:~/srv/spbgit/app`, run by PM2 as | |
| 15 | +`spbgit-server` (manifest `M1M32:~/dispatch/apps/spbgit.json`, pinned to | |
| 16 | +M1M32, published through the MacLustr Tunnel as https://git.spboucher.ai). | |
| 17 | +`origin` of that clone is spbgit itself (`https://git.spboucher.ai/spbgit.git`, | |
| 18 | +bare repo `~/srv/git/spbgit.git` on the same machine). | |
| 19 | + | |
| 20 | +## 0. Pre-flight (laptop) | |
| 21 | + | |
| 22 | +```bash | |
| 23 | +cd ~/Desktop/Cluster/projets/spbgit | |
| 24 | +npm run lint && npm run check:headers && npm test # 86 tests | |
| 25 | +python3 -I scripts/qa/screens.py http://127.0.0.1:7421 /tmp/qa --no-shots # dev server on 7421 | |
| 26 | +git log --oneline -3 # 2.0.0 commits on main | |
| 27 | +git push origin main # → spbgit bare repo on M1M32 | |
| 28 | +``` | |
| 29 | + | |
| 30 | +## 1. Deploy (M1M32) | |
| 31 | + | |
| 32 | +```bash | |
| 33 | +ssh M1M32 | |
| 34 | +export PATH=/opt/homebrew/bin:$PATH | |
| 35 | +cd ~/srv/spbgit/app | |
| 36 | +git fetch origin && git status # must be clean, on main | |
| 37 | +cp -a ~/srv/spbgit/cache ~/srv/spbgit/cache.bak-$(date +%Y%m%d) # optional: 2 GB, speeds up rollback | |
| 38 | +git pull --ff-only origin main | |
| 39 | +npm ci --omit=dev --no-audit --no-fund # @anthropic-ai/sdk, @fastify/compress added in 2.0 | |
| 40 | +node scripts/fetch-fonts.mjs # woff2 are git-ignored | |
| 41 | +pm2 restart spbgit-server --update-env | |
| 42 | +pm2 logs spbgit-server --lines 20 --nostream | |
| 43 | +``` | |
| 44 | + | |
| 45 | +The first home request after the restart recomputes 157 overviews with the | |
| 46 | +new cache shape (`overview-v5.json`, `heatmap-v5.json`, `search-index-v2.json`); | |
| 47 | +expect ~10 s once, then ≤ 100 ms. Old `overview-v4.json`/`heatmap-v4.json` | |
| 48 | +files are simply ignored (delete them with | |
| 49 | +`find ~/srv/spbgit/cache -name 'overview-v4.json' -delete` when convenient). | |
| 50 | + | |
| 51 | +### Recommended at the same time (operational, reversible) | |
| 52 | + | |
| 53 | +- **Memory limit**: the 2.0 server idles around 250–350 MB and no longer | |
| 54 | + buffers blobs; keep `max_memory_restart: 750M` in the manifest but expect | |
| 55 | + zero memory restarts. Check after 24 h: `pm2 describe spbgit-server | grep restarts`. | |
| 56 | +- **Log rotation**: `~/.pm2/logs/spbgit-server-out.log` was 599 MB. | |
| 57 | + `pm2 install pm2-logrotate && pm2 set pm2-logrotate:max_size 50M && pm2 set pm2-logrotate:retain 7`, | |
| 58 | + then `pm2 flush spbgit-server` once. | |
| 59 | +- **AI (optional, off by default)**: add to the manifest env | |
| 60 | + `SPBGIT_AI_PROVIDER=anthropic`, `SPBGIT_AI_API_KEY=…` (server-side only), | |
| 61 | + optionally `SPBGIT_AI_MODEL` (default `claude-opus-5-5`), | |
| 62 | + `SPBGIT_AI_DAILY_BUDGET_TOKENS` (default 500 000). Or point at a local | |
| 63 | + OpenAI-compatible server: `SPBGIT_AI_PROVIDER=openai-compatible`, | |
| 64 | + `SPBGIT_AI_BASE_URL=http://<node>:<port>/v1`, `SPBGIT_AI_MODEL=…`. | |
| 65 | + Every repository is public, so no private code is involved, but enabling a | |
| 66 | + remote provider still sends repository excerpts and visitor questions to it — | |
| 67 | + decide explicitly. | |
| 68 | + | |
| 69 | +## 2. Verify | |
| 70 | + | |
| 71 | +```bash | |
| 72 | +curl -s https://git.spboucher.ai/healthz | jq . # version 2.0.0, features, git concurrency | |
| 73 | +curl -s -o /dev/null -w '%{http_code} %{time_total}\n' https://git.spboucher.ai/ | |
| 74 | +curl -s -o /dev/null -w '%{http_code}\n' https://git.spboucher.ai/spbgit/blob/main/src/server.mjs | |
| 75 | +curl -s "https://git.spboucher.ai/api/v1/search?q=server&scope=files&limit=3" | jq .files | |
| 76 | +curl -s https://git.spboucher.ai/api/v1/stats | jq '{commits, commitsLast52Weeks}' # commits ≥ commitsLast52Weeks | |
| 77 | +git ls-remote https://git.spboucher.ai/spbgit.git | head -2 # Smart HTTP advertisement | |
| 78 | +# from the laptop: a real clone + a push to a scratch repo with a PAT | |
| 79 | +``` | |
| 80 | + | |
| 81 | +Then the manual Safari/iPhone checklist in `docs/SPBGIT-2-PROGRESS.md` (Phase 7). | |
| 82 | + | |
| 83 | +## 3. Rollback | |
| 84 | + | |
| 85 | +```bash | |
| 86 | +ssh M1M32 | |
| 87 | +export PATH=/opt/homebrew/bin:$PATH | |
| 88 | +cd ~/srv/spbgit/app | |
| 89 | +git checkout 810e402 # last 1.1.0 commit (v4 "Monolith") | |
| 90 | +npm ci --omit=dev --no-audit --no-fund | |
| 91 | +pm2 restart spbgit-server --update-env | |
| 92 | +``` | |
| 93 | + | |
| 94 | +Data is untouched by 2.0 (bare repos, `meta.json`, `activity.jsonl`, | |
| 95 | +`clones.json`, `tokens.json`, releases). 2.0 only *adds* `data/ai-usage.json` | |
| 96 | +(when AI is enabled) and new cache file names; 1.1.0 ignores both. The 1.1.0 | |
| 97 | +caches (`overview-v4.json`, `heatmap-v4.json`, `search-index.json`) are left in | |
| 98 | +place by 2.0, so a rollback is warm immediately. Return to 2.0 with | |
| 99 | +`git checkout main`. | |
| 100 | + | |
| 101 | +## 4. What changed operationally | |
| 102 | + | |
| 103 | +| Item | 1.1.0 | 2.0.0 | | |
| 104 | +|---|---|---| | |
| 105 | +| `/raw/*` | whole blob buffered (≤ 64 MB) | streamed from `git cat-file`, `Content-Length` from `cat-file -s` | | |
| 106 | +| git subprocesses | unbounded | 12 concurrent (`SPBGIT_GIT_CONCURRENCY`) | | |
| 107 | +| Request logging | 2 lines per request (incl. assets) | 1 line per completed request, assets only on error | | |
| 108 | +| Compression | none in-process | brotli q4 / gzip 5 for text ≥ 1 KB (`@fastify/compress`) | | |
| 109 | +| Overview cache | per repo, busted by hook only | validated against ref stamps (HEAD, packed-refs, refs/heads, refs/tags mtimes) | | |
| 110 | +| New routes | — | `/about/metrics`, `/:repo/insights`, `/:repo/research`, `/api/v1/search`, `/api/v1/repos/:name/tree`, `/api/v1/ai/*` (404 unless enabled) | | |
| 111 | +| Dependencies | — | `@anthropic-ai/sdk`, `@fastify/compress`; dev: `axe-core` | | |
modified
docs/SPBGIT-2-PROGRESS.md
+107 −0
@@ -90,3 +90,110 @@ midnight + light; full run archived outside the repo). | ||
| 90 | 90 | (3 MB uncompressed for the 60-file v4 commit; ~120 KB over the gzip edge) — |
| 91 | 91 | per-file lazy loading is listed as remaining work; WebKit run and 200 % zoom |
| 92 | 92 | are done in Phase 7. |
| 93 | + | |
| 94 | +## Phase 3 — Developer workflows (2026-11-01) ✅ | |
| 95 | + | |
| 96 | +**Files changed**: `src/web/assets/js/app.js` (⌘K: scoped server search with | |
| 97 | +debounce, recent queries, keyboard/touch navigation, repo-context commands), | |
| 98 | +`src/lib/search.mjs` + `src/api/v1.mjs` (files/commits/branches, index status), | |
| 99 | +`src/web/views/commit.njk` + `src/web/routes.mjs` (unified/split, wrap, page | |
| 100 | +line budget with deferred files, cached prepared diff), `src/insights/insights.mjs` | |
| 101 | ++ `src/web/views/insights.njk` (per-repository analytics: 52-week calendar, | |
| 102 | +contributors, hottest paths, declared dependencies from manifests — | |
| 103 | +"dependency graph where data supports it" is delivered as a dependency list | |
| 104 | +with registry links; a true graph needs lockfile resolution and is listed as | |
| 105 | +remaining work), `src/web/views/metrics.njk` (`/about/metrics`). | |
| 106 | + | |
| 107 | +**Metric coherence**: `test/e2e/roundtrip.test.mjs` disables the hook, pushes | |
| 108 | +a commit and asserts `stats.commits` and `stats.commitsLast52Weeks` both move | |
| 109 | +by exactly one and that all-time ≥ 52-week. Root cause and definitions are | |
| 110 | +documented on `/about/metrics` and in the plan (§4.2). | |
| 111 | + | |
| 112 | +**Tests**: unit `routes-helpers`, `research-insights`; e2e search/tree/split. | |
| 113 | + | |
| 114 | +## Phase 4 — Research Mode (2026-11-01) ✅ | |
| 115 | + | |
| 116 | +**Files**: `src/research/profile.mjs` (pure classifier + LaTeX metadata | |
| 117 | +parser, no compilation), `src/web/views/research.njk`, route | |
| 118 | +`/:repo/research`, `research` flag on `PATCH /api/v1/repos/:name`, | |
| 119 | +auto-detection in `lib/overview.mjs` (`researchDetected`) with a banner on the | |
| 120 | +repository page. The page reads at most 256 KB of each LaTeX candidate and | |
| 121 | +never executes anything; PDFs are rendered by the browser from `/raw`. | |
| 122 | +Verified on `wp20_uqo` (31 PDFs, 2 main documents, 127 figures). | |
| 123 | + | |
| 124 | +## Phase 5 — AI groundwork (2026-11-01) ✅ (opt-in, disabled by default) | |
| 125 | + | |
| 126 | +**Files**: `src/ai/index.mjs` (chunking, per-blob cache, BM25), `src/ai/ask.mjs` | |
| 127 | +(service, budget, concurrency, prompt-injection wrapping, citation | |
| 128 | +verification), `src/ai/providers/{anthropic,openai-compatible,fake}.mjs`, | |
| 129 | +`src/config.mjs` (`SPBGIT_AI_*`), API routes (`/api/v1/ai/status`, | |
| 130 | +`POST /api/v1/repos/:name/ask`, `POST /api/v1/repos/:name/commits/:sha/summary`, | |
| 131 | +12 req/min/IP), ask card on the repository page and "Summarize" on commits | |
| 132 | +(rendered only when enabled). Anthropic provider uses the official SDK | |
| 133 | +(`claude-opus-5-5` default, server-side refusal fallbacks, `refusal` stop | |
| 134 | +reason surfaced as an error). **No provider is simulated**: without | |
| 135 | +configuration the routes 404 and the UI shows nothing; the `fake` provider is | |
| 136 | +refused outside `SPBGIT_ENV=test` (unit-tested). | |
| 137 | + | |
| 138 | +**Not done (needs credentials/infra)**: embeddings — retrieval is lexical | |
| 139 | +(BM25) by design so that nothing leaves the server until a question is asked. | |
| 140 | + | |
| 141 | +## Phase 6 — Agent Workspace (2026-11-01) ✅ design only | |
| 142 | + | |
| 143 | +`docs/AGENT-WORKSPACE-DESIGN.md`: scoped job tokens + pre-receive hook, | |
| 144 | +sandboxed runner on a separate node, approval gates, audit log, acceptance | |
| 145 | +criteria. No code shipped — the preconditions (token scopes, runner) do not | |
| 146 | +exist yet. | |
| 147 | + | |
| 148 | +## Phase 7 — Hardening & release (2026-11-01) ✅ (deployment pending approval) | |
| 149 | + | |
| 150 | +- **Cross-engine QA**: `scripts/qa/screens.py` — Chromium 13 pages × | |
| 151 | + 320/360/375/390/430/768/1024/1440/1920 × light/dark/midnight: 0 overflow, | |
| 152 | + 0 small target; WebKit 375/1440: 0; 200 % zoom: 0; landscape 812×375: 0. | |
| 153 | +- **Accessibility**: `scripts/qa/axe.py` (WCAG 2.x A/AA + best-practice) — | |
| 154 | + from 142 violation groups on the first 2.0 build to **0 on home, repo, | |
| 155 | + commits, branches, search, metrics, 404** after fixes (list semantics for | |
| 156 | + the stat strip, keyboard-focusable scroll regions, contrast tokens, | |
| 157 | + underlined inline links, `role="img"` on the pin). Code pages: shiki themes | |
| 158 | + switched to `github-light-high-contrast` / `github-dark-default` so every | |
| 159 | + syntax token meets AA, and gutters on changed lines use the full foreground. | |
| 160 | + **Final run (`docs/screenshots/v2/axe-report.json`): 0 violations on 9 pages × | |
| 161 | + 3 themes × 375/1440.** | |
| 162 | +- **Performance**: `docs/BENCHMARKS.md` (in-process brotli/gzip added; commit | |
| 163 | + page 3.0 MB → 55 KB wire, 679 ms → 44 ms server). | |
| 164 | +- **Security regression tests**: CSP + frame + nosniff headers, no inline | |
| 165 | + scripts, repo HTML served as text from `/raw` with a sandbox CSP, hostile | |
| 166 | + README (onerror/script/javascript:) neutralized while badges survive, | |
| 167 | + loopback-only hook with repo validation, traversal on blob/raw/tree/search. | |
| 168 | +- **Docs**: `CHANGELOG.md`, `docs/DEPLOY.md` (deploy + rollback), `README.md` | |
| 169 | + and `CLAUDE.md` updated for 2.0. | |
| 170 | + | |
| 171 | +### Manual Safari / iPhone checklist (to run on production after deploy) | |
| 172 | + | |
| 173 | +| Check | Expected | | |
| 174 | +|---|---| | |
| 175 | +| Open https://git.spboucher.ai on iPhone Safari, portrait | header 52 px, no horizontal scroll, stat strip swipes, theme follows iOS appearance | | |
| 176 | +| Switch Light/Dark/Midnight from the menu sheet, reload | choice persists; status bar colour matches | | |
| 177 | +| Repository → Clone | bottom sheet; URL selectable; Copy shows a toast; ZIP downloads | | |
| 178 | +| Repository → Files | tree sheet; folders expand lazily; current file highlighted | | |
| 179 | +| Blob | wrap on by default; line-number tap sets `#L` hash; Permalink copies a sha URL | | |
| 180 | +| Commit | deferred files load with "Load diff"; wrap toggle; split hidden on phone | | |
| 181 | +| ⌘K (search icon) | keyboard opens without zoom (16 px input), scopes scroll, results open on tap | | |
| 182 | +| Rotate to landscape | no overflow, header stays reachable | | |
| 183 | +| Dynamic Type at the largest accessibility size | no clipped controls | | |
| 184 | +| VoiceOver | skip link, landmarks, dialog labels, list semantics of the stat strip | | |
| 185 | + | |
| 186 | +## Remaining work (not done, in priority order) | |
| 187 | + | |
| 188 | +1. **Deploy to M1M32** (`docs/DEPLOY.md`) and re-run the production | |
| 189 | + measurements (LCP/INP on real iPhone, restart count after 24 h). | |
| 190 | +2. Per-file lazy loading of diffs over the API (today: page budget + `?all=1`). | |
| 191 | +3. True dependency graph (lockfile resolution, cross-repository edges). | |
| 192 | +4. Embedding-based retrieval behind the same provider abstraction (requires a | |
| 193 | + credentialed embeddings endpoint; BM25 stays the default). | |
| 194 | +5. Agent Workspace preconditions (scoped tokens + pre-receive hook). | |
| 195 | +6. Playwright tests in the Node test runner (today Python Playwright scripts | |
| 196 | + are the browser QA; vitest covers server/API). | |
| 197 | +7. `deploy/ecosystem.config.cjs` / `deploy/setup-m3u96a.sh` still describe | |
| 198 | + the pre-mld (ngrok) deployment; production is driven by the `mld` | |
| 199 | + manifest. Documentation debt only. | |
added
docs/screenshots/v2/ask-375-midnight-chromium.png
+0 −0
Binary file not shown.
added
docs/screenshots/v2/axe-report.json
+1 −0
@@ -0,0 +1 @@ | ||
| 1 | +[] | |
| \ No newline at end of file | ||
added
docs/screenshots/v2/blob-tree-1440-dark-chromium.png
+0 −0
Binary file not shown.
added
docs/screenshots/v2/clone-375-midnight-chromium.png
+0 −0
Binary file not shown.
added
docs/screenshots/v2/insights-1440-dark-chromium.png
+0 −0
Binary file not shown.
added
docs/screenshots/v2/menu-375-midnight-chromium.png
+0 −0
Binary file not shown.
added
docs/screenshots/v2/palette-375-midnight-chromium.png
+0 −0
Binary file not shown.
modified
docs/screenshots/v2/report.json
+1789 −330
Large diff (2,718 lines) is not loaded by default to keep the page fast.Load diff
added
docs/screenshots/v2/research-1440-dark-chromium.png
+0 −0
Binary file not shown.
added
docs/screenshots/v2/research-375-midnight-chromium.png
+0 −0
Binary file not shown.
added
docs/screenshots/v2/tree-375-midnight-chromium.png
+0 −0
Binary file not shown.
modified
package-lock.json
+142 −2
Large diff (223 lines) is not loaded by default to keep the page fast.Load diff
modified
package.json
+3 −0
Large diff (15 lines) is not loaded by default to keep the page fast.Load diff
added
scripts/qa/axe.py
+48 −0
Large diff (48 lines) is not loaded by default to keep the page fast.Load diff
added
src/ai/ask.mjs
+207 −0
Large diff (207 lines) is not loaded by default to keep the page fast.Load diff
added
src/ai/index.mjs
+118 −0
Large diff (118 lines) is not loaded by default to keep the page fast.Load diff
added
src/ai/providers/anthropic.mjs
+51 −0
Large diff (51 lines) is not loaded by default to keep the page fast.Load diff
added
src/ai/providers/fake.mjs
+31 −0
Large diff (31 lines) is not loaded by default to keep the page fast.Load diff
added
src/ai/providers/openai-compatible.mjs
+47 −0
Large diff (47 lines) is not loaded by default to keep the page fast.Load diff
modified
src/api/v1.mjs
+37 −0
Large diff (43 lines) is not loaded by default to keep the page fast.Load diff
added
src/insights/insights.mjs
+108 −0
Large diff (108 lines) is not loaded by default to keep the page fast.Load diff
modified
src/render/highlight.mjs
+2 −1
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added
src/research/profile.mjs
+155 −0
Large diff (155 lines) is not loaded by default to keep the page fast.Load diff
modified
src/server.mjs
+20 −1
Large diff (57 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/assets/css/app.css
+41 −16
Large diff (138 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/assets/css/tokens.css
+10 −10
Large diff (49 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/assets/js/app.js
+66 −1
Large diff (85 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/routes.mjs
+65 −10
Large diff (169 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/blame.njk
+2 −2
Large diff (15 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/blob.njk
+1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/commit.njk
+4 −1
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/components/ui.njk
+9 −13
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/error.njk
+1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/home.njk
+1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
added
src/web/views/insights.njk
+64 −0
Large diff (64 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/layout.njk
+2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/partials/repo-card.njk
+1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/partials/repo-header.njk
+2 −1
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
modified
src/web/views/repo.njk
+17 −1
Large diff (36 lines) is not loaded by default to keep the page fast.Load diff
added
src/web/views/research.njk
+134 −0
Large diff (134 lines) is not loaded by default to keep the page fast.Load diff
added
test/e2e/ai.test.mjs
+159 −0
Large diff (159 lines) is not loaded by default to keep the page fast.Load diff
modified
test/e2e/roundtrip.test.mjs
+38 −0
Large diff (44 lines) is not loaded by default to keep the page fast.Load diff
added
test/unit/research-insights.test.mjs
+75 −0
Large diff (76 lines) is not loaded by default to keep the page fast.Load diff