SPB Git

Repository Public MIT Selected

CHANGELOG.md 12.1 KB

# Changelog

# 3.1.2 — 2026-11-19

Fix: a crawler spread over thousands of addresses made the site unreachable (6 481 addresses in 26 minutes, one to three requests each, 89 % of them for pages pinned to a commit; median answer 7.6 s, restarts for memory).

  • Visitor pass (lib/crawl-guard.mjs): a signed timestamp in the spb_pass cookie (30 days, HttpOnly), handed out by every HTML page. Key in <data dir>/visitor-pass.key.
  • Historical pages asked with a pass keep the per-address budget (SPBGIT_HISTORICAL_RPM, 30), which also applies to the pass itself. Asked without a pass, they share one budget for all addresses, SPBGIT_HISTORICAL_ANON_RPM (20).
  • Beyond the shared budget a browser receives its pass and a page that reloads after two seconds; other clients receive 429 with the clone URL.

# 3.1.1 — 2026-11-17

Fix: snapshot archives filled the disk (226 GB of cache, 91 GB of it abandoned archive.*.tmp files).

  • git/archive.mjs: one git archive per archive whatever the number of concurrent requests, at most SPBGIT_ARCHIVE_CONCURRENCY (2) builds at a time, temporary file removed on failure and on shutdown.
  • Size ceiling SPBGIT_ARCHIVE_MAX_MB (2048, 0 = none): beyond it the build is stopped and the answer is 413 with the clone URL; the verdict is remembered per commit.
  • Cache janitor (Cache#pruneTransient): temporary files older than one hour and archives not served for SPBGIT_ARCHIVE_KEEP_HOURS (24) are removed, including under branch and tag tips.
  • /archive/*: SPBGIT_ARCHIVE_RPM (6) requests per minute per address.
  • deploy/prune-cache.sh (cron) stays as a second line of defence.

# 3.1.0 — 2026-11-17

Live metric badges for READMEs.

  • GET /badge/:repo/:metric.svg — SVG badges rendered by the forge from the cached repository overview: commits, last-commit (coloured by freshness), activity (26-week bar chart), language, languages, size, branches, tags, license, clones; site totals under /badge/_site/{repos,commits,languages}.svg. Options style=flat|flat-square|for-the-badge, label=, color=. Cached five minutes; an unknown repository or metric answers 404 with a red badge (src/render/badge.mjs).
  • Insights — a "Live metric badges" section previews every badge of the repository with its Markdown.
  • /about/api — badge reference. robots.txt disallows /badge/.

# 3.0.1 — 2026-11-10

Crawler containment (production was restarting 10–30 times per hour and the cache had reached 76 GB).

  • robots.txt — only the default branches are crawlable; pages pinned to a commit SHA, history, blame and raw files are disallowed (Crawl-delay: 5).
  • Historical pages (blob|tree|blame|commits|raw at a SHA, single commits) — a per-address budget of SPBGIT_HISTORICAL_RPM requests per minute (default 30), 429 with Retry-After beyond it, and X-Robots-Tag: noindex, nofollow on every such response (src/lib/crawl-guard.mjs).
  • Cache janitor — the cache of a commit that is no longer a branch or tag tip is dropped after SPBGIT_CACHE_STALE_HOURS (default 24); runs two minutes after start and every six hours, off the request path (Cache#pruneRepo).
  • Memory limit — max_memory_restart raised from 750 MB to 2 GB: the JavaScript heap stays under 350 MB but resident memory settles around 1.2–1.3 GB under crawl (git output buffers, highlighting), so the old limit restarted the server every one to two minutes.

# 3.0.0 — 2026-11-09 · "Monograph"

A visual redesign of the whole forge in the language of www.spboucher.ai. No route, API, git or cache behaviour was removed. System: docs/DESIGN.md.

# Design system

  • Paper, ink and one indigo accent in three themes: Light (the paper of spboucher.ai), Dark (its night edition), Midnight (true black, OLED). Tokens are oklch; hairline borders, 2 px ink rules, square corners, no gradients, no glow, shadows only on dialogs.
  • Type: Fraunces (names, titles, numerals), Inter (text), IBM Plex Mono (labels, metadata, code). Fonts are self-hosted and tracked in git (7 woff2, 200 KB; the two preloaded files weigh 116 KB), with metric-adjusted fallbacks so the swap does not shift the layout.
  • One icon set (Lucide, ISC) replaces Octicons and the text glyphs of the palette; the logo, favicon, touch icon and OG cards use the black "SB" monogram tile of the personal site.
  • House syntax themes "paper" and "ink" (shiki) replace the GitHub themes.

# Screens

  • Front page: masthead with a one-line statement and a framed clone specimen ("Fig. 1"), ruled figure row, numbered sections — selected repositories, 52 weeks of commits (calendar, language share, latest pushes), and the repository index as a typographic table with filters.
  • Repository: eyebrow (visibility, licence, research), serif title, description, mono topic tags, inline clone box, figures, language bar, mono tabs; file list with a latest-commit strip; README as a document sheet with an "On this page" outline. Sub-pages use a compact header. The research banner became a header link and a "Research · preview" tab.
  • Code: breadcrumb toolbar, file header (name, size, lines, language), line numbers behind a hairline, hanging indent in wrap mode, image / PDF / notebook / binary / too-large states; blame with an age scale.
  • History: day-grouped commit ledger; commit page with serif subject, quoted body, stats bar, unified/split segmented control, restyled diffs.
  • Branches, releases, insights, research, search: ledgers, ruled result groups, figure captions, checklists.
  • New: /about/api (reference of every public endpoint), linked from the header, footer, menu and palette. /about/metrics gained an outline.
  • Shell: mono navigation with an underline marker, search trigger, theme menu, bottom sheets on phones, ⌘K palette with scoped tabs and icons, colophon footer, navigation progress hairline, 404/500 page, print stylesheet.

# Details

  • Relative dates carry the exact instant in title everywhere.
  • Code no longer wraps by default on phones; it scrolls inside its block.
  • Heading anchors are hidden on touch screens.
  • Cache keys bumped where the markup changed (readme-v3, md-v3, hl-v3, commit-v4, insights-v2, heatmap-v6, OG/favicons v3).

# Operations (found while deploying)

  • /healthz no longer walks the cache synchronously: with 75 GB / millions of cached files in production the walk froze every request for about 100 s per call. The footprint is now computed in the background (Cache.footprintAsync, refreshed at most every 15 minutes); cache is null until the first walk completes and cacheComputedAt states its age.

# QA

  • 89 tests; scripts/qa/screens.py now also covers /about/api, insights and an empty search (16 pages × 9 widths × 3 themes) and more controls; scripts/qa/axe.py covers 15 pages × 2 widths × 3 themes.

# 2.0.0 — 2026-11-01 · "The Intelligent Git Forge"

Plan and per-phase log: docs/SPBGIT-2-PLAN.md, docs/SPBGIT-2-PROGRESS.md. Deployment and rollback: docs/DEPLOY.md. Benchmarks: docs/BENCHMARKS.md.

# Reliability (production incident fixes)

  • /raw/* streams blobs instead of buffering up to 64 MB; blob pages read only size + an 8 KB binary probe for PDFs, images, binaries and large files. Fixes the memory-restart loop (2 861 PM2 restarts, RSS up to 2.1 GB).
  • Git subprocesses are bounded by a 12-slot semaphore (SPBGIT_GIT_CONCURRENCY).
  • Overview, calendar and search caches are validated against live ref stamps, so a push missed by the post-receive hook can no longer leave stale totals ("all-time" 3 332 < "52 weeks" 3 346 is fixed; both come from the same pass).
  • One compact log line per completed request; assets logged only on error.
  • Prepared diffs, highlighted blobs, rendered Markdown blobs and blame are cached per commit; brotli/gzip compression in-process.

# Design 2.0

  • Three first-class themes — Light, Dark (charcoal/slate), Midnight (deep navy) — following the OS by default, overridable and persisted.
  • Semantic design tokens, 4/8 px spacing, tabular numerals, Bricolage display type; mobile-first AppShell with a compact sticky header, navigation sheet, bottom-sheet dialogs on phones, 44 px touch targets, safe-area insets, no horizontal overflow at 320–1920 px (Chromium + WebKit).
  • Component library (Nunjucks macros): icons, chips, stat strip, tabs, breadcrumbs, ref picker, empty/error/skeleton states, copy button, toast, confirmation dialog.
  • Home: compact hero, stat strip, pinned, searchable catalog (deep-linkable filters), activity, stack ring, calendar with metric definitions.
  • Repository: compact stat strip, tabs (Code · Commits · Branches · Releases · Insights · Research), clone sheet with selectable URLs, lazy expandable file tree (desktop sidebar / phone sheet), README with table of contents.
  • Source browser: sticky toolbar, wrap (default on phones), copy, raw, blame, history, permalink, line ranges, markdown source/rendered toggle.
  • Commits: timeline grouped by day; commit page with changed-file list, unified or split diffs, wrap toggle, large diffs deferred (?all=1).
  • Search: ⌘K palette with scopes (repos, files, commits, branches), recent queries, keyboard + touch navigation; /search page groups everything; /api/v1/search is permission-trivial (every repository is public) and index status is exposed.
  • /about/metrics documents every statistic; /api/v1/stats returns 52-week totals and streaks with a link to the definitions.
  • /:repo/insights: 52-week calendar, contributors, hottest paths, declared dependencies parsed from manifests (npm, PyPI, Cargo, Go, Swift PM).

# Research Mode (opt-in)

  • /:repo/research (tab when research: true on the repository; preview with a banner when LaTeX/BibTeX/figure signals are detected): manuscript PDF viewer, LaTeX main documents with parsed title/authors/environment counts, BibTeX, figure gallery, tables, reproducibility checklist, experiment outputs, notebooks, tagged versions with per-version PDF/zip links. Nothing is compiled or executed.

# AI groundwork (opt-in, off by default)

  • Provider-agnostic service (src/ai/): Anthropic (official SDK, server-side key, server-side refusal fallbacks), OpenAI-compatible local servers, and a test-only fake. Disabled providers are absent — nothing is simulated.
  • Grounded repository Q&A (POST /api/v1/repos/:name/ask) over a BM25 chunk index cached per blob (incremental across commits); answers must cite path#Lx-Ly, citations are verified against the retrieved excerpts and unverifiable ones are removed; untrusted content is wrapped as data.
  • Commit summaries (POST /api/v1/repos/:name/commits/:sha/summary, cached).
  • Daily token budget, per-route rate limit, concurrency cap, /api/v1/ai/status.

# Agent Workspace

  • Design only: docs/AGENT-WORKSPACE-DESIGN.md (scoped job tokens, sandboxed runner, approval gates, audit log). Not implemented by design.

# Tooling

  • scripts/qa/screens.py (overflow + touch targets + screenshots across widths/themes/engines) and scripts/qa/axe.py (WCAG 2.x A/AA audit), both on Playwright for Python.
  • Tests: 59 → 86 (semaphore, split rows, research/insights parsers, scoped search, tree API, streamed raw, missed-hook coherence, AI with fake provider, security regressions).

# Removed

  • Instrument Serif font, constellation canvas, typewriter terminal, count-up numerals, mouse spotlight, scroll reveal (decorative motion that cost CPU on phones). .stat/.rstat tiles replaced by the compact stat strip.
Files main
Menu

Theme