Repository Public MIT Selected
1<!--2 ─────────────────────────────────────────────3 SPB Git — Personal Git Platform4 ─────────────────────────────────────────────5 Author : Simon-Pierre Boucher6 Contact : contact@spboucher.ai7 File : docs/SPBGIT-2-PLAN.md8 Purpose : SPB Git 2.0 — architecture, feature inventory, risks, phased plan, acceptance criteria9 License : MIT © Simon-Pierre Boucher10 ─────────────────────────────────────────────11-->12# SPB Git 2.0 — The Intelligent Git Forge · Plan1314Written 2026-11-01 after Phase 0 (reconnaissance) on the real repository15(`spbgit` 1.1.0, commit `810e402`, deployed on M1M32 `~/srv/spbgit/app`,16public at https://git.spboucher.ai through the MacLustr Tunnel).1718## 1. Architecture as found1920| Layer | Implementation | Notes |21|---|---|---|22| Runtime | Node ≥ 20 (25.9 on M1M32), ESM, Fastify 5, pino | PM2 `spbgit-server`, port 7420, loopback; TLS at the BHS64 gateway |23| Git hosting | `src/git/smart-http.mjs` spawns `git upload-pack / receive-pack --stateless-rpc`, streamed, gzip bodies | Anonymous clone/fetch, PAT (argon2id) for push, HTTP Basic user `spb` |24| Repo model | `src/git/repo.mjs` shells out to `git` (refs, tree, blob, log, diff parser, blame, archive) | `maxBuffer` 64 MB per call, blobs fully buffered |25| Storage | bare repos `~/srv/git/*.git` (157), `data/meta.json` (desc/topics/pins), `data/activity.jsonl`, `data/clones.json`, `data/tokens.json`, `data/releases/` | filesystem is the database |26| Cache | `~/srv/spbgit/cache/repos/<repo>/<sha>/*`, `repos/<repo>/_repo/overview-v4.json`, `global/*` | post-receive hook POSTs to `/internal/hooks/post-receive` (loopback only) → busts repo + global |27| Rendering | Nunjucks SSR (`src/web/views`), hand-written CSS (`tokens.css`, `app.css`, `markdown.css`), vanilla JS (`app.js`, `theme-boot.js`), shiki dual-theme, markdown-it + sanitize-html, satori/resvg OG cards | No bundler, no framework, CSP `script-src 'self'` (no inline JS) |28| API | `/api/v1/*` public reads, Bearer PAT writes, `/healthz`, `/feed.atom`, `/sitemap.xml`, `/og/*.png`, `/raw/*`, `/archive/*`, `/releases/*` | JSON error envelope, rate limit 200/min/IP |29| CLI | `cli/spbgit.mjs` (commander) | untouched by this plan |30| Tests | vitest: 6 unit files + 1 e2e (real `git push` round trip, releases) — **59/59 green** at baseline; `eslint`, `check:headers` green | no browser tests yet |31| Deploy | `mld` manifest `M1M32:~/dispatch/apps/spbgit.json` (pinned to M1M32), PM2 `max_memory_restart 750M`, nightly `deploy/backup.sh` | ngrok removed 2026-09-10; `deploy/ecosystem.config.cjs` still documents a tunnel app (dead) |3233Visibility model (CLAUDE.md §0): **every repository is public, read-only**; the34only writes are owner PAT operations through the API/CLI. The web UI has no35login and no authenticated action by design — "authenticated actions on a phone"36therefore means: clone URLs, release downloads and the public API, all of which37must work from Safari/iOS. This plan does not add a web login.3839## 2. Feature inventory (existing — must keep working)4041Home (hero, stats, pinned bento, repo grid with sparklines + client filters,42activity feed, language ring, heatmap) · Repo overview (header, 6 stat tiles,43language bar, tabs, branch select, clone box, zip/tar.gz, file table with last44commit, README + TOC) · Tree · Blob (shiki, line anchors + ranges, wrap, copy,45raw, blame, history, permalink, markdown render/source, images, PDF inline,46binaries, notebooks as JSON) · Commits (grouped by day, stats, pagination,47path filter) · Commit (diff with per-hunk highlighting, file nav, rename/binary)48· Blame (age-coloured) · Branches (ahead/behind) · Tags/releases (assets with49sha256) · Search (names/desc/topics/README) · ⌘K palette (repos + commands) ·50OG images, favicon, sitemap, robots, Atom · 404/500 · Smart HTTP · API v1 ·51releases upload · hooks · CLI.5253## 3. Baseline measurements (2026-11-01, production, Playwright Chromium)5455Mobile = 375×812, CPU ×4, ~1.6 Mb/s. Desktop = 1440×900, no throttling.5657| Page | Mobile LCP | Mobile TTFB | Desktop LCP | HTML | CSS | JS | Fonts |58|---|---|---|---|---|---|---|---|59| `/` | 1 216 ms | 182 ms | 320 ms | 34 KB | 18.7 KB | 9 KB | 209 KB (5 files preloaded) |60| `/spbgit` | 276 ms | 144 ms | 236 ms | 9.7 KB | cached | cached | cached |61| blob `src/server.mjs` | 552 ms | 424 ms | 328 ms | 13 KB | | | |62| commit `810e402` | 1 612 ms | **1 463 ms** | 1 336 ms | **121 KB** | | | |6364CLS = 0 everywhere. Server timing on M1M32: `/` 64 ms warm, repo 58 ms,65`/api/v1/repos` 10 ms. 157 repos, cache 2.0 GB / 9 684 files.6667Responsive audit (`document.documentElement.scrollWidth` at 375 px, Chromium +68WebKit): **every page overflows horizontally** (399 px in production, 377–383 px69locally). Causes found by bisection: footer grid `1fr` track inheriting the70`min-content` of `pre.footer-cmd` (fix: `minmax(0,1fr)` / `min-width:0`), the71hero `canvas` positioned `inset:-24px` + `width:calc(100%+48px)`, and the72repo tab strip. Baseline screenshots: `docs/screenshots/baseline/`.7374## 4. Critical risks found in Phase 075761. **Memory restart loop (production incident).** PM2 restarted `spbgit-server`77 **2 861 times** (`exceeds --max-memory-restart`, RSS up to 2.1 GB vs 750 MB78 limit). Every restart drops in-flight pushes/clones and makes the post-receive79 hook POST fail (hook has a 15 s timeout and swallows errors). Request mix80 before a restart: crawlers on `/<repo>/blob/*` and `/raw/*` of large files81 (`phd_thesis` PDFs). `Repos.blob()` buffers whole blobs (up to 64 MB) even82 when the view only needs size/type, `/raw` buffers again; concurrency is83 unbounded. Fix in Phase 1 (reliability track): stream `/raw`, metadata-only84 blob probe, bounded git concurrency, cheaper `/healthz`.852. **Commit totals inconsistent** ("all-time" 3 332 < "52 weeks" 3 346). Root86 cause verified: `overview-v4.json` is keyed by repo, not by HEAD, and is only87 invalidated by the hook; 8 repos were pushed while the server was restarting88 (risk 1) so their cached `commitCount` is stale (e.g. `llm-api` 0 vs 5), while89 the heatmap is rebuilt from live `rev-list` whenever *any* push clears90 `global/`. Fix: validate overview cache against live HEAD (cheap ref stamp),91 derive both numbers from the same per-repo timestamp source, document metric92 definitions.933. **Unbounded logs.** pino logs every request at info; `~/.pm2/logs/spbgit-server-out.log`94 is 599 MB with no rotation. Fix: compact completion-only logging, exclude95 assets, document `pm2-logrotate`.964. **Commit page cost.** Per-hunk shiki highlighting on every request (1.2–1.5 s97 TTFB, 121 KB HTML). Fix: cache the prepared diff per sha, collapse large files.985. **Mobile layout.** Horizontal overflow on every page; touch targets < 44 px99 (tabs 36 px, `.btn-sm` 28 px, copy buttons); desktop-only hover reveal on100 heading anchors; `.hero-terminal` and constellation canvas run on phones;101 `100vh` without `dvh`; no safe-area insets; sticky header 64 px on phones.1026. **Themes.** Only dark/light; theme toggle is a 2-state button without an103 "OS" mode; no `color-scheme` meta per theme.1047. **Dead deploy config.** `deploy/ecosystem.config.cjs` + `deploy/ngrok.yml`105 still describe ngrok; CLAUDE.md names host `m3u96a`. Production is M1M32 via106 `mld` + MacLustr Tunnel. Documentation only — the running manifest is correct.1078. **Fonts.** 209 KB of preloaded fonts on first visit (5 files). Keep108 self-hosting but drop preloads for non-critical weights, subset usage.109110## 5. Design direction (decisions)111112- **Identity kept**: S mark, electric-blue→teal gradient, network motif (static113 SVG/CSS, animated only on desktop without `prefers-reduced-motion`).114- **Three themes**: `light` (soft blue-white `#f4f6fb`), `dark` (charcoal/slate115 `#15181f`), `midnight` (deep navy `#070b16`, electric blue, restrained116 turquoise — the successor of the v4 obsidian look). Default follows the OS117 (`auto` → light/dark); explicit override persisted in `localStorage`118 (`spbgit-theme` = `auto|light|dark|midnight`), applied before first paint by119 `theme-boot.js`, exposed in the header menu and the ⌘K palette.120- **Semantic tokens** in `tokens.css`: surfaces (`--surface-0..3`), foreground121 (`--fg`, `--fg-muted`, `--fg-faint`), borders, focus ring, status122 (`--ok/--warn/--danger/--info`), spacing scale 4/8 px (`--space-1..10`),123 radii, shadows, motion durations, syntax colours (shiki vars). WCAG AA checked124 per theme (contrast table in progress log).125- **Density**: Linear-like. Compact sticky header (56 px desktop, 52 px phone),126 compact `StatStrip` instead of five hero tiles and six repo tiles, 1-column127 phone layouts, bottom-sheet dialogs on phones (palette, menu, branch picker).128- **Component library** (Nunjucks macros in `src/web/views/components/` +129 matching CSS sections): AppShell, Header, MobileNavigation (sheet), Breadcrumbs,130 StatStrip, RepositoryCard, StatusChip, Tabs, SearchDialog, CodeViewer, FileTree,131 CommitTimeline, DiffViewer, EmptyState, LoadingSkeleton, ErrorState, Toast,132 ConfirmationDialog (JS, used by destructive CLI-less flows only if they ever133 appear; shipped as a primitive).134- **Typography**: Inter (UI) / JetBrains Mono (code) / Bricolage Grotesque135 (display, 700 + 800). Instrument Serif italic dropped (one less font).136 `font-variant-numeric: tabular-nums` on every statistic.137138## 6. Phased milestones139140| Phase | Scope | Acceptance |141|---|---|---|142| 0 Reconnaissance | this document, baseline numbers, screenshots, risks | done |143| 1 Design system + reliability P0 | tokens/themes, AppShell, header + mobile sheet, buttons/tabs/cards/chips, skeleton/empty/error, a11y foundations; streaming raw, blob probe, bounded concurrency, head-validated overview cache, compact logging, overflow test harness | tests green, no overflow at 320–1920, three themes, memory fix unit-tested |144| 2 Core screens | home, repo overview, catalog, source browser (tree + blob), clone panel, file tree sidebar on desktop | all old URLs 200, README/badges unchanged, screenshots at 9 widths |145| 3 Developer workflows | ⌘K with server-side search (repos/files/commits/branches), commits timeline, diff viewer (unified/split, wrap, cached), branch nav, analytics + metric definitions, keyboard/touch | search API tests, commit-count coherence test, Playwright mobile smoke |146| 4 Research Mode | opt-in `research` flag + auto-detection banner, `/:repo/research`: PDF, LaTeX sources, figures, tables, reproducibility checklist, outputs, version links | no code execution, no LaTeX compile, tests on fixtures |147| 5 AI groundwork | `src/ai/` provider-agnostic (Anthropic, OpenAI-compatible, local OpenAI-compatible e.g. MLX/llama.cpp), incremental file index, grounded Q&A with `path@sha#Lx-Ly` citations, prompt-injection wrapping, token/concurrency budgets, feature-flagged off by default | disabled providers are *absent*, never simulated; tests use an explicit fake provider under `SPBGIT_ENV=test` |148| 6 Agent Workspace | design document only (isolation, scopes, approvals, audit) | `docs/AGENT-WORKSPACE-DESIGN.md` |149| 7 Hardening & release | cross-engine QA (Chromium + WebKit) at 320/360/375/390/430/768/1024/1440/1920, landscape, 200 % zoom, axe audit, perf re-measure, security regression tests, changelog, deploy + rollback runbook | all acceptance criteria below |150151Deployment to M1M32 (`git pull` + `pm2 restart`) is **not** performed by the152implementer without explicit approval (directive: pause before production153changes).154155## 7. Acceptance criteria (from the directive, made checkable)1561571. `npm test` includes the Smart HTTP round trip (clone/push/fetch) and passes;158 every pre-2.0 URL pattern is covered by a route test (home, repo, tree, blob,159 raw, archive, commits, commit, blame, branches, tags/releases, search, feeds,160 OG, API).1612. Playwright run at 320–430 px and desktop, in light/dark/midnight, asserts162 `scrollWidth === clientWidth` on every core page and 44×44 px targets on163 primary controls.1643. Metric definitions documented (`docs/METRICS.md`), unit test proving165 all-time ≥ 52-week on the same source, no placeholder data in production166 templates (demo fixtures only under `dev/`).1674. Search/file browsing: path traversal + ref validation tests; search index168 built only from repos on disk (all public).1695. Markdown/diff/code: sanitizer tests (script/iframe/event handlers stripped,170 badges kept), diff whitespace preserved, keyboard-reachable actions.1716. Automated tests + a manual Safari/iPhone checklist recorded in the progress172 log.1737. AI/Research features behind flags; `/healthz` reports feature flags.1748. Deliverables: `CHANGELOG.md`, screenshots (`docs/screenshots/`), tests,175 `docs/BENCHMARKS.md` (before/after), `docs/DEPLOY.md` (deploy + rollback),176 remaining work list in `docs/SPBGIT-2-PROGRESS.md`.177