feat: skeleton, auth, storage core — config, sqlite schema, blob CAS, tree ops, chunked uploads, sessions, lockout, shares, search, preview backend
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
33 changed files +11,616 −0
added
.gitignore
+18 −0
@@ -0,0 +1,18 @@ | ||
| 1 | +# ───────────────────────────────────────────── | |
| 2 | +# SPB Drive — Personal Cloud Drive | |
| 3 | +# ───────────────────────────────────────────── | |
| 4 | +# Author : Simon-Pierre Boucher | |
| 5 | +# Contact : contact@spboucher.ai | |
| 6 | +# File : .gitignore | |
| 7 | +# Purpose : Git ignore rules (deps, runtime data, logs, secrets) | |
| 8 | +# License : MIT © Simon-Pierre Boucher | |
| 9 | +# ───────────────────────────────────────────── | |
| 10 | + | |
| 11 | +node_modules/ | |
| 12 | +data/ | |
| 13 | +*.log | |
| 14 | +.DS_Store | |
| 15 | +.env | |
| 16 | +coverage/ | |
| 17 | +dist/ | |
| 18 | +test/.tmp/ | |
added
CLAUDE.md
+369 −0
@@ -0,0 +1,369 @@ | ||
| 1 | +# CLAUDE.md — **SPB Drive** · Personal Cloud Drive for Simon-Pierre Boucher | |
| 2 | + | |
| 3 | +> **Read this entire document before writing a single line of code.** | |
| 4 | +> This file is the single source of truth. Every architectural decision, naming convention, UI detail, and non-negotiable rule lives here. When in doubt, re-read this file. When this file conflicts with your instinct, this file wins. | |
| 5 | + | |
| 6 | +--- | |
| 7 | + | |
| 8 | +## 0. Identity & Non-Negotiables | |
| 9 | + | |
| 10 | +| Key | Value | | |
| 11 | +|---|---| | |
| 12 | +| **Product name** | SPB Drive | | |
| 13 | +| **Owner / sole user** | Simon-Pierre Boucher | | |
| 14 | +| **Contact** | [contact@spboucher.ai](mailto:contact@spboucher.ai) | | |
| 15 | +| **Public domain** | `https://drive.spboucher.ai` (ngrok custom domain) | | |
| 16 | +| **Deployment host** | Node **m3u96b** | | |
| 17 | +| **Runtime** | Node.js ≥ 20, ESM only | | |
| 18 | +| **Access model** | **Private by default.** The entire drive is behind a password login. The ONLY public surfaces are explicitly created **share links**. | | |
| 19 | +| **Initial password** | `sn18brady` — seeded at first boot, **stored argon2-hashed, never in code, never in the repo, never logged**. Changeable from Settings. | | |
| 20 | + | |
| 21 | +### 0.1 THE GOLDEN RULE — Mandatory Author Header | |
| 22 | + | |
| 23 | +**Every single file you generate — source, config, script, style, test, migration — MUST begin with an author header.** No exceptions. A file without this header is a bug. | |
| 24 | + | |
| 25 | +Canonical template (adapt comment syntax per language): | |
| 26 | + | |
| 27 | +```js | |
| 28 | +/** | |
| 29 | + * ───────────────────────────────────────────── | |
| 30 | + * SPB Drive — Personal Cloud Drive | |
| 31 | + * ───────────────────────────────────────────── | |
| 32 | + * Author : Simon-Pierre Boucher | |
| 33 | + * Contact : contact@spboucher.ai | |
| 34 | + * File : <relative/path/filename.ext> | |
| 35 | + * Purpose : <one-line description> | |
| 36 | + * License : MIT © Simon-Pierre Boucher | |
| 37 | + * ───────────────────────────────────────────── | |
| 38 | + */ | |
| 39 | +``` | |
| 40 | + | |
| 41 | +| Language | Syntax | | |
| 42 | +|---|---| | |
| 43 | +| JS / TS / CSS / SCSS | `/** ... */` block | | |
| 44 | +| Python / Bash / YAML / Dockerfile / TOML | `#` lines | | |
| 45 | +| HTML / Nunjucks | `<!-- ... -->` / `{# ... #}` | | |
| 46 | +| SQL | `-- ` lines | | |
| 47 | + | |
| 48 | +Provide `scripts/inject-headers.mjs` and `scripts/check-headers.mjs`; `npm run check:headers` must fail CI if any tracked file lacks the header. | |
| 49 | + | |
| 50 | +### 0.2 Password / secret handling rules (absolute) | |
| 51 | + | |
| 52 | +- The literal string `sn18brady` must appear **nowhere** in the codebase, config files, tests, fixtures, or logs. It is provided once via the `SPBDRIVE_BOOTSTRAP_PASSWORD` env var (or interactive prompt) on first boot, hashed with **argon2id**, stored in `data/auth.json`, and the env var is then ignored forever. | |
| 53 | +- Password change flow in Settings (requires current password). Also a break-glass CLI on the server: `node scripts/reset-password.mjs` (interactive, local only). | |
| 54 | +- Session secret, share-link signing key: generated randomly at first boot into `data/keys.json` (chmod 600). | |
| 55 | + | |
| 56 | +--- | |
| 57 | + | |
| 58 | +## 1. Product Vision | |
| 59 | + | |
| 60 | +SPB Drive is a **self-hosted Google Drive / Dropbox replacement** for one person. Three purposes: | |
| 61 | + | |
| 62 | +1. **Vault**: all of Simon-Pierre's files, organized in folders, uploadable from any browser, safe on m3u96b. | |
| 63 | +2. **Universal previewer**: click any file → beautiful in-browser preview. Images, video, audio, PDF, Office docs, code, Markdown, CSV, archives, fonts — *everything imaginable* previews without downloading. | |
| 64 | +3. **Sharing machine**: any file or folder → one click → clean public URL (`https://drive.spboucher.ai/s/<token>`) with optional expiry, password, and download limits. Recipients need no account. | |
| 65 | + | |
| 66 | +Quality bar: *"If a stranger receives a share link, the preview page should look like a polished commercial product."* | |
| 67 | + | |
| 68 | +--- | |
| 69 | + | |
| 70 | +## 2. High-Level Architecture | |
| 71 | + | |
| 72 | +``` | |
| 73 | +┌──────────────────────────── node m3u96b ─────────────────────────────┐ | |
| 74 | +│ │ | |
| 75 | +│ ┌─────────────┐ ┌────────────────────────────────────────────────┐ │ | |
| 76 | +│ │ ngrok │ │ SPB Drive Server (Node 20) │ │ | |
| 77 | +│ │ tunnel │──▶│ Fastify app :7430 │ │ | |
| 78 | +│ │ drive. │ │ ├─ /login, /app/* Web UI (SSR + JS) │ │ | |
| 79 | +│ │ spboucher.ai │ │ ├─ /api/v1/* JSON API (session auth) │ │ | |
| 80 | +│ └─────────────┘ │ ├─ /s/:token[/...] PUBLIC share pages │ │ | |
| 81 | +│ │ ├─ /dl/*, /stream/* Auth'd download/stream │ │ | |
| 82 | +│ │ └─ /thumb/* Thumbnails │ │ | |
| 83 | +│ └────────────────┬───────────────────────────────┘ │ | |
| 84 | +│ │ │ | |
| 85 | +│ ┌──────────────────────────────┼──────────────────────────────┐ │ | |
| 86 | +│ │ │ │ │ | |
| 87 | +│ ┌────▼─────────────┐ ┌────────────▼───────────┐ ┌──────────────▼─┐ │ | |
| 88 | +│ │ /srv/drive/files │ │ /srv/drive/db │ │ /srv/drive/ │ │ | |
| 89 | +│ │ content store: │ │ drive.sqlite (better- │ │ cache/ │ │ | |
| 90 | +│ │ blobs by sha256 │ │ sqlite3, WAL): │ │ thumbnails, │ │ | |
| 91 | +│ │ /ab/cd/abcd... │ │ nodes, shares, tags, │ │ transcodes, │ │ | |
| 92 | +│ │ (dedup natural) │ │ sessions, activity, FTS│ │ office→pdf │ │ | |
| 93 | +│ └──────────────────┘ └────────────────────────┘ └────────────────┘ │ | |
| 94 | +└────────────────────────────────────────────────────────────────────────┘ | |
| 95 | +``` | |
| 96 | + | |
| 97 | +**Principles:** | |
| 98 | +- **Content-addressed blob store** (`/srv/drive/files/<sha256[0:2]>/<sha256[2:4]>/<sha256>`): identical files stored once (free dedup); DB `nodes` table maps the virtual folder tree onto blobs. Deleting a node only deletes the blob when its refcount hits zero. | |
| 99 | +- **SQLite is the metadata brain** (`better-sqlite3`, WAL mode): tables `nodes` (id, parent_id, name, type file/folder, blob_sha, size, mime, created, modified, starred, color, trashed_at), `shares`, `tags`, `node_tags`, `sessions`, `activity`, plus an **FTS5** virtual table for search. | |
| 100 | +- **Everything streams.** Uploads and downloads never buffer whole files in memory. Range requests supported everywhere (video seeking!). | |
| 101 | +- **Preview/transcode work is queued** (tiny in-process job queue, concurrency 2) and cached in `/srv/drive/cache/` keyed by blob sha — thumbnails and conversions are computed once per unique file, ever. | |
| 102 | + | |
| 103 | +### 2.1 Repository layout of *this* project | |
| 104 | + | |
| 105 | +``` | |
| 106 | +spbdrive/ | |
| 107 | +├── CLAUDE.md | |
| 108 | +├── README.md # with badges | |
| 109 | +├── package.json | |
| 110 | +├── src/ | |
| 111 | +│ ├── server.mjs # Fastify bootstrap | |
| 112 | +│ ├── config.mjs # zod-validated env/config | |
| 113 | +│ ├── db/ | |
| 114 | +│ │ ├── schema.sql # full schema + FTS5 + indexes | |
| 115 | +│ │ └── db.mjs # migrations, prepared statements | |
| 116 | +│ ├── auth/ | |
| 117 | +│ │ ├── password.mjs # argon2id verify/change, lockout | |
| 118 | +│ │ └── session.mjs # cookie sessions (httpOnly, SameSite=Lax) | |
| 119 | +│ ├── storage/ | |
| 120 | +│ │ ├── blobs.mjs # CAS store: put(stream)→sha, get, refcount GC | |
| 121 | +│ │ ├── nodes.mjs # tree ops: mkdir, move, copy, rename, trash | |
| 122 | +│ │ └── upload.mjs # chunked/resumable upload endpoint | |
| 123 | +│ ├── preview/ | |
| 124 | +│ │ ├── router.mjs # mime → preview strategy dispatcher | |
| 125 | +│ │ ├── thumbs.mjs # sharp: image thumbs; ffmpeg: video poster | |
| 126 | +│ │ ├── transcode.mjs # ffmpeg audio/video web-safe transcodes | |
| 127 | +│ │ ├── office.mjs # libreoffice --headless → PDF conversion | |
| 128 | +│ │ ├── code.mjs # shiki highlighting + markdown-it (GFM) | |
| 129 | +│ │ └── archive.mjs # zip/tar listing (and inner-file preview) | |
| 130 | +│ ├── shares/ | |
| 131 | +│ │ └── shares.mjs # tokens, expiry, passwords, limits, zip-of-folder | |
| 132 | +│ ├── search/ | |
| 133 | +│ │ └── search.mjs # FTS5 queries + filters | |
| 134 | +│ ├── api/v1.mjs | |
| 135 | +│ └── web/ | |
| 136 | +│ ├── routes.mjs | |
| 137 | +│ ├── views/ # Nunjucks templates (login, app shell, share) | |
| 138 | +│ └── assets/ # tokens.css, app.css, app.js, fonts, icons | |
| 139 | +├── cli/ | |
| 140 | +│ ├── spbdrive.mjs # bin entry | |
| 141 | +│ └── commands/ | |
| 142 | +├── deploy/ | |
| 143 | +│ ├── ngrok.yml | |
| 144 | +│ ├── ecosystem.config.cjs # pm2: server + tunnel | |
| 145 | +│ ├── spbdrive.service # systemd alternative | |
| 146 | +│ ├── setup-m3u96b.sh # idempotent bootstrap (installs ffmpeg, libreoffice) | |
| 147 | +│ └── backup.sh | |
| 148 | +├── scripts/ | |
| 149 | +│ ├── inject-headers.mjs | |
| 150 | +│ ├── check-headers.mjs | |
| 151 | +│ └── reset-password.mjs | |
| 152 | +└── test/ # vitest unit + e2e (upload→preview→share round-trip) | |
| 153 | +``` | |
| 154 | + | |
| 155 | +--- | |
| 156 | + | |
| 157 | +## 3. Authentication & Sessions | |
| 158 | + | |
| 159 | +- **Login page** at `/login`: centered card, SPB monogram, single password field, "Remember me for 30 days" checkbox. Sharp, minimal, dark. | |
| 160 | +- Verify with argon2id. **Brute-force protection**: 5 failed attempts → 15-minute lockout per IP + global exponential backoff; every failure logged to `activity`. | |
| 161 | +- Session = random 256-bit id in `sessions` table; cookie `spbdrive_sid`, `httpOnly`, `Secure`, `SameSite=Lax`; 24 h TTL (30 d with remember-me), sliding renewal. | |
| 162 | +- Every route except `/login`, `/s/*` (shares), `/healthz`, and static assets requires a valid session → otherwise 302 to `/login?next=...`. | |
| 163 | +- Settings page: change password, view/revoke active sessions (device + last seen), toggle theme default. | |
| 164 | + | |
| 165 | +--- | |
| 166 | + | |
| 167 | +## 4. File Management Core | |
| 168 | + | |
| 169 | +### 4.1 Uploads (must feel instant and bulletproof) | |
| 170 | + | |
| 171 | +- **Drag & drop anywhere** in the app (full-window drop overlay), plus an Upload button (files *and* folders — use `webkitdirectory` and DataTransferItem tree walking so entire folder structures upload with hierarchy preserved). | |
| 172 | +- **Chunked + resumable**: client slices files into 8 MB chunks, `POST /api/v1/upload/init` → uploadId, `PUT /api/v1/upload/:id/chunk/:n`, `POST /api/v1/upload/:id/complete` (server assembles, sha256-streams into blob store, creates node). Interrupted uploads resume by asking the server which chunks it has. | |
| 173 | +- Upload panel (bottom-right, Google-Drive-style): per-file progress bars, speed, cancel, retry, aggregate progress, minimize. | |
| 174 | +- Paste-to-upload (Ctrl+V an image/screenshot into the app → lands in the current folder as `pasted-YYYYMMDD-HHmmss.png`). | |
| 175 | +- No file-size limit by design; test with a multi-GB file. | |
| 176 | + | |
| 177 | +### 4.2 Folder tree & organization ("groupe folder etc.") | |
| 178 | + | |
| 179 | +- Unlimited nested folders. Left sidebar: collapsible folder tree + quick sections **Recent**, **Starred**, **Shared**, **Trash**, per-tag views, and a storage usage meter (used space, per-type breakdown donut). | |
| 180 | +- **Folder colors** (8 palette choices) and **emoji/icon** per folder. | |
| 181 | +- **Tags/labels**: create colored tags, assign to any file/folder, filter by tag. | |
| 182 | +- **Starred/favorites** (toggle with `s`). | |
| 183 | +- Operations, all with multi-select (click, shift-click ranges, ctrl-click, drag-rectangle select): **move (drag & drop onto folders or via dialog with tree picker), copy, rename (F2, inline), delete → Trash, restore, delete forever, download (multi-select → server-zips on the fly), duplicate**. | |
| 184 | +- **Trash**: soft delete with original-path memory; auto-purge after 30 days (daily job); "Empty trash" with typed confirmation. | |
| 185 | +- Breadcrumb path with drag-onto-crumb to move; right-click **context menu** everywhere (custom-rendered, keyboard accessible). | |
| 186 | +- Conflict handling on move/upload: "Keep both (name-2)", "Replace", "Skip" — batch-applicable. | |
| 187 | + | |
| 188 | +### 4.3 Views | |
| 189 | + | |
| 190 | +- **Grid view** (thumbnail cards, 5 sizes via slider) and **List view** (name, size, type, modified, tags) — toggle persisted per folder. | |
| 191 | +- Sort: name / size / modified / type, asc-desc. Folders always first. | |
| 192 | +- **Keyboard-first**: arrows navigate, Enter opens, Space = quick-look preview overlay, Del = trash, Ctrl+A select all, `/` focuses search, `?` shows a shortcuts cheat-sheet modal. | |
| 193 | + | |
| 194 | +--- | |
| 195 | + | |
| 196 | +## 5. Universal Preview Engine — "all types imaginable" (Critical) | |
| 197 | + | |
| 198 | +Clicking a file opens the **Preview overlay**: full-screen modal, dark scrim, filename + size header, actions (Download · Share · Star · Info · Delete), ←/→ arrows to flip through siblings, Esc closes. Every strategy below is dispatched by `preview/router.mjs` from MIME + extension: | |
| 199 | + | |
| 200 | +| Category | Formats | Preview behavior | | |
| 201 | +|---|---|---| | |
| 202 | +| **Images** | jpg, png, gif, webp, avif, svg, bmp, ico, heic* | Zoom (wheel/pinch), pan, rotate, 1:1 toggle, EXIF panel (dimensions, camera, date, GPS→"open map" link). HEIC converted to jpg via sharp if supported, else download card. Animated gif/webp play. SVG sandboxed (served with strict CSP, no scripts). | | |
| 203 | +| **Video** | mp4, webm, mov, mkv, avi | Native `<video>` with **Range streaming** (instant seek). Poster frame + duration badge on thumbnails (ffmpeg). Non-web-safe codecs (mkv/avi/hevc) → background ffmpeg transcode to H.264/AAC mp4, cached by blob sha; UI shows "Optimizing for playback…" with progress, then plays. Playback speed control, PiP, keyboard (space, ←→ 5 s, ↑↓ volume, f fullscreen). | | |
| 204 | +| **Audio** | mp3, wav, flac, ogg, m4a, aac | Custom player: **waveform** (wavesurfer.js or pre-computed peaks via ffmpeg), ID3 tags + embedded cover art displayed, loop, speed. | | |
| 205 | +| **PDF** | | Rendered with **pdf.js**: page thumbnails rail, page nav, zoom, text selection + in-document search, print. | | |
| 206 | +| **Office** | docx, xlsx, pptx, odt, ods, odp, rtf | Converted to PDF via **libreoffice --headless** (queued, cached by sha) then shown in the pdf.js viewer with a note "Converted preview — download for original". xlsx additionally offers a native fast path: SheetJS → styled HTML table with sheet tabs. | | |
| 207 | +| **Code** | 100+ extensions (js, ts, py, go, rs, c, cpp, java, sh, sql, …) | **Shiki** highlighting, line numbers, wrap toggle, copy button, language auto-detect fallback. | | |
| 208 | +| **Markdown** | md | Full **GFM render** (same pipeline standard as SPB Git: tables, task lists, badges inline, heading anchors, mermaid blocks) with rendered/source toggle. | | |
| 209 | +| **Data** | csv, tsv | Virtualized table (fast on 100k rows), sticky header, column sort, cell search, "detected delimiter" smartness. | | |
| 210 | +| **Structured text** | json, yaml, toml, xml | Pretty-printed, syntax-highlighted, JSON gets a collapsible tree explorer toggle. | | |
| 211 | +| **Notebooks** | ipynb | Cells rendered: markdown cells via GFM, code cells highlighted, outputs (text/images) shown. | | |
| 212 | +| **Archives** | zip, tar, tar.gz, 7z*, rar* | **Browse inside the archive**: file tree with sizes, preview text/image members directly (streamed extraction of single member), "Extract to folder…" action. 7z/rar listing via `7z` binary if installed, else download card. | | |
| 213 | +| **Fonts** | ttf, otf, woff, woff2 | Specimen page: alphabet, pangram, size slider, weight info. | | |
| 214 | +| **Email** | eml | Parsed headers + HTML/plain body (sanitized) + attachment list (each previewable). | | |
| 215 | +| **Ebooks** | epub | epub.js reader with chapters. | | |
| 216 | +| **3D** *(nice-to-have)* | stl, glb | three.js orbit viewer. | | |
| 217 | +| **Everything else** | * | Elegant fallback card: big type icon, filename, size, mime, sha256, "Download" button — never an ugly error. | | |
| 218 | + | |
| 219 | +**Thumbnails** (`/thumb/:nodeId?size=`): images via sharp (256/512, webp), videos via ffmpeg frame @10%, PDFs via pdf render of page 1, office via converted-PDF page 1, code/text via generic type icons (crisp custom SVG icon set per extension family, color-coded). All cached by `(sha, size)`. | |
| 220 | + | |
| 221 | +--- | |
| 222 | + | |
| 223 | +## 6. Sharing System — clean public URLs | |
| 224 | + | |
| 225 | +Any file **or folder** → "Share" → modal: | |
| 226 | + | |
| 227 | +- Generates `https://drive.spboucher.ai/s/<token>` (token = 10-char base58, unguessable, signed). | |
| 228 | +- Options per share: **expiry** (1 h / 1 d / 7 d / 30 d / never / custom date), **password** (argon2-hashed; public visitor gets a minimal password gate page), **max downloads** (counter-enforced), **allow download** toggle (off = preview-only, download endpoints refuse), note-to-self label. | |
| 229 | +- **File share page** (public, no session): centered card with the full preview engine (same viewer as the app), file name/size, Download button, "Shared by Simon-Pierre Boucher · contact@spboucher.ai" footer. Proper Open Graph tags (thumbnail as og:image) so links unfurl nicely in iMessage/Slack/Twitter. | |
| 230 | +- **Folder share page**: read-only file browser of that subtree (grid/list, previews work, per-file download) + "Download all as ZIP" (server streams a zip built on the fly, `archiver`). | |
| 231 | +- **Share manager** (app section): table of all active shares — target, URL (copy button), visits, downloads, expiry countdown, revoke button, edit options. Expired/revoked links show a clean "This link has expired" page. | |
| 232 | +- Every share visit/download logged to `activity` (timestamp, IP, user-agent) and visible in the share's detail drawer. | |
| 233 | +- QR code button next to every share URL (generated server-side, SVG). | |
| 234 | + | |
| 235 | +--- | |
| 236 | + | |
| 237 | +## 7. Search, Recents & Activity | |
| 238 | + | |
| 239 | +- **Global search** (`/` shortcut): FTS5 over name + tags + extracted text. **Text extraction pipeline** (queued, cached): plain/code/md/csv indexed directly; PDF via `pdftotext`; docx/xlsx/pptx via converted text; results ranked, with snippet highlights. | |
| 240 | +- Filter chips in search: type (image/video/audio/doc/archive), tag, folder scope, date range, size range, starred, shared. | |
| 241 | +- **Recent** view: last 50 touched files. **Activity** page: chronological log (uploads, renames, moves, shares created, share visits, logins) with icons — this doubles as a security audit trail. | |
| 242 | + | |
| 243 | +--- | |
| 244 | + | |
| 245 | +## 8. Web UI Design Spec — must be sharp | |
| 246 | + | |
| 247 | +- **Same design DNA as SPB Git** (they're siblings): dark default `#0b0e14` background, surface `#11151c`, border `#1f2530`, text `#e6e9ef`, muted `#8b93a3`, accent `#4f8cff`, accent-2 `#22d3aa`, danger `#ff5d5d`; light theme toggle; `Inter` + `JetBrains Mono`, self-hosted woff2; radius 10px; borders over shadows. | |
| 248 | +- **App shell**: top bar (logo "SPB Drive", global search, upload button, view toggle, theme, settings) · left sidebar (New button with dropdown: folder/upload/paste, tree, sections, tags, storage meter) · main pane (breadcrumb, toolbar, content) · right **Info panel** (slides in: preview thumb, metadata, tags editor, share list, activity for that node). | |
| 249 | +- Micro-interactions: 150 ms ease transitions, skeleton loaders, optimistic UI on rename/move/star with rollback on error, toast notifications (undo on trash: "Moved to trash · Undo"). | |
| 250 | +- Empty states illustrated (custom minimal SVG art): empty folder, empty trash, no search results. | |
| 251 | +- Fully responsive: on mobile, sidebar becomes a drawer, grid adapts, upload via native picker, previews go full-screen. Touch: long-press = context menu. | |
| 252 | +- Custom 404/500/expired-share pages in the design system. Favicon + PWA manifest (installable, standalone display) — offline is out of scope, but the icon on a phone home screen must look pro. | |
| 253 | +- Lighthouse ≥ 90/95 (app) and ≥ 95/95 (share pages). No third-party CDN requests anywhere. | |
| 254 | + | |
| 255 | +--- | |
| 256 | + | |
| 257 | +## 9. `spbdrive` CLI | |
| 258 | + | |
| 259 | +Companion CLI (`npm i -g` / `npm link`), config in `~/.spbdrive/config.json` (server URL + an **API token** created in Settings — the CLI never stores the login password). Chmod 600. | |
| 260 | + | |
| 261 | +| Command | Behavior | | |
| 262 | +|---|---| | |
| 263 | +| `spbdrive init` | Wizard: server URL + API token paste, verifies with `/api/v1/me`. | | |
| 264 | +| `spbdrive ls [remote-path]` | List a folder (aligned, colorized; `--json`). | | |
| 265 | +| `spbdrive up <files...> [-d /remote/folder]` | Chunked upload with progress bars; directories recurse. | | |
| 266 | +| `spbdrive down <remote-path> [local]` | Download file, or folder as zip. | | |
| 267 | +| `spbdrive mkdir / mv / rm / restore` | Tree operations (rm → trash). | | |
| 268 | +| `spbdrive share <remote-path> [--expires 7d] [--password] [--max-dl N]` | Create share, print URL (and `--qr` renders an ANSI QR in the terminal). | | |
| 269 | +| `spbdrive shares` / `spbdrive revoke <token>` | Manage shares. | | |
| 270 | +| `spbdrive search "query"` | FTS search from terminal. | | |
| 271 | +| `spbdrive push <local-dir> <remote-dir>` | One-way sync mirror (hash-compare, upload changed, `--delete` optional). | | |
| 272 | +| `spbdrive doctor` | Config/token/server/ffmpeg-on-server diagnostics. | | |
| 273 | + | |
| 274 | +Exit codes 0/1/2/3 and `NO_COLOR` respected, same conventions as spbgit. | |
| 275 | + | |
| 276 | +--- | |
| 277 | + | |
| 278 | +## 10. JSON API (`/api/v1`) — session or Bearer API-token auth | |
| 279 | + | |
| 280 | +`GET /me` · `GET /nodes/:id` · `GET /nodes/:id/children?sort&view` · `POST /nodes` (mkdir) · `PATCH /nodes/:id` (rename/move/star/color/tags) · `DELETE /nodes/:id` (trash) + `/restore` + `?force=true` · upload trio (`/upload/init|chunk|complete`) · `GET /dl/:id` + `GET /stream/:id` (Range) · `GET /thumb/:id` · shares CRUD (`/shares`) · `GET /search?q&filters` · `GET /activity` · `GET /stats` (storage totals) · `POST /auth/login|logout`, `POST /auth/password`, `GET|DELETE /auth/sessions`, `POST /auth/api-tokens`. | |
| 281 | + | |
| 282 | +Errors as `{ "error": { "code", "message" } }`. Rate limit `/auth/login` hard (see §3) and public `/s/*` endpoints (100 req/min/IP). | |
| 283 | + | |
| 284 | +--- | |
| 285 | + | |
| 286 | +## 11. Deployment — node m3u96b + ngrok + `drive.spboucher.ai` | |
| 287 | + | |
| 288 | +### 11.1 `deploy/setup-m3u96b.sh` (idempotent) | |
| 289 | + | |
| 290 | +1. Install Node 20, **ffmpeg**, **libreoffice** (headless), **poppler-utils** (pdftotext), `7z`, ngrok. | |
| 291 | +2. Create `/srv/drive/{files,db,cache,backups,logs}` with correct ownership/permissions (700). | |
| 292 | +3. Clone/pull repo to `/srv/drive/app`, `npm ci --omit=dev`, run migrations. | |
| 293 | +4. First-boot: prompt for bootstrap password (or read `SPBDRIVE_BOOTSTRAP_PASSWORD`), hash, store; generate session/share keys. | |
| 294 | +5. Install pm2 config, start, `pm2 save`. | |
| 295 | + | |
| 296 | +### 11.2 ngrok (`deploy/ngrok.yml`) | |
| 297 | + | |
| 298 | +```yaml | |
| 299 | +version: 3 | |
| 300 | +agent: | |
| 301 | + authtoken: ${NGROK_AUTHTOKEN} | |
| 302 | +endpoints: | |
| 303 | + - name: spbdrive | |
| 304 | + url: https://drive.spboucher.ai | |
| 305 | + upstream: | |
| 306 | + url: http://127.0.0.1:7430 | |
| 307 | +``` | |
| 308 | + | |
| 309 | +Document: register `drive.spboucher.ai` in the ngrok dashboard, add the CNAME at the DNS provider, TLS terminates at ngrok, app runs `trustProxy: true` and reads client IP from `x-forwarded-for` (used by lockout + share logs). | |
| 310 | + | |
| 311 | +### 11.3 Process management & ops | |
| 312 | + | |
| 313 | +- pm2 apps `spbdrive-server` + `spbdrive-tunnel`, autorestart, logs to `/srv/drive/logs/`, `pm2 startup` documented (survives m3u96b reboots). systemd unit provided as alternative. | |
| 314 | +- `GET /healthz`: uptime, node count, storage used, queue depth. | |
| 315 | +- **Backups** (`deploy/backup.sh`, nightly cron): sqlite `.backup` snapshot + incremental rsync-style hardlink copy of `/srv/drive/files` into `backups/YYYY-MM-DD/`, keep 14 dailies + 8 weeklies; verify + log. Restore procedure documented in README. | |
| 316 | +- Daily maintenance job: purge 30-day trash, GC orphaned blobs (refcount 0), prune expired shares/sessions, vacuum FTS. | |
| 317 | + | |
| 318 | +--- | |
| 319 | + | |
| 320 | +## 12. Security Checklist (verify each before "done") | |
| 321 | + | |
| 322 | +- [ ] `sn18brady` appears nowhere in the repo; grep in CI (`check:secrets` script) proves it. | |
| 323 | +- [ ] argon2id everywhere (login, share passwords); constant-time token compares; lockout works (test it). | |
| 324 | +- [ ] All cookies `httpOnly` + `Secure` + `SameSite=Lax`; CSRF token on state-changing form posts. | |
| 325 | +- [ ] Share tokens ≥ 58 bits entropy; revocation immediate; expired links leak nothing (no filename in error page title). | |
| 326 | +- [ ] Path traversal impossible (nodes are DB ids, blobs are hashes — never trust client paths; validate names against `^[^/\\\0]{1,255}$`, reject `.`/`..`). | |
| 327 | +- [ ] Uploaded HTML/SVG never served same-origin as executable content: previews sandboxed (`Content-Security-Policy: sandbox`, `X-Content-Type-Options: nosniff`; raw HTML downloads as attachment). | |
| 328 | +- [ ] Strict CSP on the app (`default-src 'self'`); share pages likewise. | |
| 329 | +- [ ] ffmpeg/libreoffice invoked with `execFile` (no shell), timeouts, and memory/size guards; conversion of hostile files can't take the server down. | |
| 330 | +- [ ] Session revocation page works; logout everywhere button. | |
| 331 | +- [ ] No secrets/passwords in logs (pino redaction paths configured). | |
| 332 | + | |
| 333 | +--- | |
| 334 | + | |
| 335 | +## 13. Quality Bar & Definition of Done | |
| 336 | + | |
| 337 | +**Engineering standards:** ESM, small modules, JSDoc on exports, prepared statements only (no string-built SQL), vitest coverage on blob store + tree ops + share logic, and an e2e test that boots the server, logs in, chunk-uploads a file, fetches its thumbnail, creates a password-protected share, and downloads through it. `npm run lint` + `check:headers` + `check:secrets` all green. | |
| 338 | + | |
| 339 | +**The platform is DONE when every box is checked:** | |
| 340 | + | |
| 341 | +- [ ] `bash deploy/setup-m3u96b.sh` on a fresh m3u96b brings `https://drive.spboucher.ai` live and it survives a reboot. | |
| 342 | +- [ ] Wrong password 5× → locked out; right password → in; password changeable in Settings. | |
| 343 | +- [ ] Drag-dropping a **folder** with 500 nested files uploads with hierarchy intact, resumable, with a live progress panel. | |
| 344 | +- [ ] A 2 GB video uploads, gets a poster thumbnail, and **seeks instantly** in the player; an mkv transcodes then plays. | |
| 345 | +- [ ] docx, xlsx, pptx, pdf, epub, ipynb, zip (browse inside!), csv (100k rows), json, md (with badges), mp3 (waveform), heic, svg, and an unknown `.xyz` file all preview correctly or fall back elegantly. | |
| 346 | +- [ ] Any file/folder → share URL with expiry + password + max-downloads; folder share offers ZIP-all; link unfurls with a thumbnail in social apps; revoke kills it instantly. | |
| 347 | +- [ ] Multi-select drag-move, rename inline, trash + undo toast, restore, tags, folder colors, starred — all functional with keyboard shortcuts. | |
| 348 | +- [ ] Search finds a word **inside** an uploaded PDF. | |
| 349 | +- [ ] Storage meter, activity log, share visit logs all accurate. | |
| 350 | +- [ ] `spbdrive up ~/photos -d /Photos` and `spbdrive share /Photos --expires 7d` work end-to-end. | |
| 351 | +- [ ] Every generated file carries the **Simon-Pierre Boucher / contact@spboucher.ai** header (`check:headers` passes). | |
| 352 | +- [ ] Nightly backup ran and the documented restore procedure was tested once. | |
| 353 | + | |
| 354 | +--- | |
| 355 | + | |
| 356 | +## 14. Build Order (follow this sequence) | |
| 357 | + | |
| 358 | +1. **Skeleton**: config, Fastify, sqlite schema + migrations, logging, healthz, header/secret check scripts. | |
| 359 | +2. **Auth**: password bootstrap, sessions, login page, lockout, settings (change password, sessions list). | |
| 360 | +3. **Storage core**: blob store, nodes tree ops, chunked resumable upload, download/stream with Range, trash + GC. | |
| 361 | +4. **Web app shell**: design tokens, layout, folder browsing (grid+list), drag-drop upload panel, multi-select ops, context menus, keyboard nav. | |
| 362 | +5. **Preview engine**: thumbnails → images/video/audio → pdf.js → code/md/csv/json → office conversion → archives → fallbacks. | |
| 363 | +6. **Sharing**: tokens, options, public pages (file + folder + zip-all), share manager, OG unfurls, QR. | |
| 364 | +7. **Search & activity**: FTS, extraction pipeline, recents, activity log. | |
| 365 | +8. **CLI**: init → ls/up/down → share → push sync → doctor. | |
| 366 | +9. **Polish**: info panel, tags/colors/stars, empty states, PWA manifest, a11y, Lighthouse. | |
| 367 | +10. **Deploy**: setup script, ngrok, pm2, backups + restore drill on m3u96b; run the full DoD checklist. | |
| 368 | + | |
| 369 | +Work in small conventional commits (`feat:`, `fix:`, `chore:`). At the end of each phase, state which DoD boxes are now satisfied. | |
added
package-lock.json
+7109 −0
Large diff (7,109 lines) is not loaded by default to keep the page fast.Load diff
added
package.json
+57 −0
Large diff (57 lines) is not loaded by default to keep the page fast.Load diff
added
scripts/check-headers.mjs
+68 −0
Large diff (68 lines) is not loaded by default to keep the page fast.Load diff
added
scripts/check-secrets.mjs
+38 −0
Large diff (38 lines) is not loaded by default to keep the page fast.Load diff
added
scripts/inject-headers.mjs
+69 −0
Large diff (69 lines) is not loaded by default to keep the page fast.Load diff
added
scripts/reset-password.mjs
+53 −0
Large diff (53 lines) is not loaded by default to keep the page fast.Load diff
added
src/api/v1.mjs
+490 −0
Large diff (490 lines) is not loaded by default to keep the page fast.Load diff
added
src/auth/password.mjs
+185 −0
Large diff (185 lines) is not loaded by default to keep the page fast.Load diff
added
src/auth/session.mjs
+128 −0
Large diff (128 lines) is not loaded by default to keep the page fast.Load diff
added
src/config.mjs
+75 −0
Large diff (75 lines) is not loaded by default to keep the page fast.Load diff
added
src/db/db.mjs
+66 −0
Large diff (66 lines) is not loaded by default to keep the page fast.Load diff
added
src/db/schema.sql
+149 −0
Large diff (149 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/archive.mjs
+191 −0
Large diff (191 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/code.mjs
+104 −0
Large diff (104 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/handlers.mjs
+279 −0
Large diff (279 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/office-thumb.mjs
+51 −0
Large diff (51 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/office.mjs
+82 −0
Large diff (82 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/queue.mjs
+68 −0
Large diff (68 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/router.mjs
+86 −0
Large diff (86 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/thumbs.mjs
+122 −0
Large diff (122 lines) is not loaded by default to keep the page fast.Load diff
added
src/preview/transcode.mjs
+123 −0
Large diff (123 lines) is not loaded by default to keep the page fast.Load diff
added
src/search/extract.mjs
+0 −0
Binary file not shown.
added
src/search/index-sync.mjs
+62 −0
Large diff (62 lines) is not loaded by default to keep the page fast.Load diff
added
src/search/search.mjs
+99 −0
Large diff (99 lines) is not loaded by default to keep the page fast.Load diff
added
src/server.mjs
+146 −0
Large diff (146 lines) is not loaded by default to keep the page fast.Load diff
added
src/shares/shares.mjs
+180 −0
Large diff (180 lines) is not loaded by default to keep the page fast.Load diff
added
src/storage/blobs.mjs
+128 −0
Large diff (128 lines) is not loaded by default to keep the page fast.Load diff
added
src/storage/nodes.mjs
+346 −0
Large diff (346 lines) is not loaded by default to keep the page fast.Load diff
added
src/storage/upload.mjs
+130 −0
Large diff (130 lines) is not loaded by default to keep the page fast.Load diff
added
src/web/http-helpers.mjs
+113 −0
Large diff (113 lines) is not loaded by default to keep the page fast.Load diff
added
src/web/routes.mjs
+432 −0
Large diff (433 lines) is not loaded by default to keep the page fast.Load diff