backup.sh: no abort on empty rotation lists; worker SDK version detection; status docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5 changed files +50 −6
modified
docs/DECISIONS.md
+6 −0
@@ -61,3 +61,9 @@ Installed on the worker via Homebrew (no LaTeX exists on any node). Proprietary | |||||
| 61 | ## D-15 Deviation: no separate `apps/api`, `services/qa` and `services/publisher` are libraries invoked by the scheduler process | 61 | ## D-15 Deviation: no separate `apps/api`, `services/qa` and `services/publisher` are libraries invoked by the scheduler process | ||
| 62 | 62 | ||||
| 63 | They keep their own directories and tests but execute inside `autoscholar-scheduler` to avoid three extra pm2 processes with identical DB access. Responsibilities unchanged. | 63 | They keep their own directories and tests but execute inside `autoscholar-scheduler` to avoid three extra pm2 processes with identical DB access. Responsibilities unchanged. | ||
| 64 | + | ||||
| 65 | +## D-16 Greenfield development exception to remote-first | ||||
| 66 | +Until the first deployment the repository was authored on the operator laptop (fast iteration, local Postgres, Playwright). First deployment: `mld stage` → `mld deploy --node M3U96b` → `mld adopt` (node directory = git clone, origin spbgit `autoscholar.git`, commit e709d44). From then on changes are committed and pushed through spbgit and pulled on the node (`git pull && pnpm install && build && pm2 restart`); the worker is a separate bundle (`services/worker/install/bundle.sh`) rsynced to `/var/autoscholar/bundle` on M4M64c. After launch the standard remote-first rule applies (edit on the node). | ||||
| 67 | + | ||||
| 68 | +## D-17 Internal API source check | ||||
| 69 | +`next start` sets `X-Forwarded-For` to the socket address for direct connections; the gateway sets it to the public client address. The internal worker API therefore trusts the *first* XFF hop: it must be inside `INTERNAL_API_ALLOWED_CIDRS` (LAN/loopback), which excludes any request that came through the tunnel. | ||||
added
docs/IMPLEMENTATION_STATUS.md
+25 −0
@@ -0,0 +1,25 @@ | |||||
| 1 | +# Implementation status — AutoScholar.ai | ||||
| 2 | + | ||||
| 3 | +Last updated: 2026-10-30 (first build day). Phases follow the master build instructions §13. "Evidence" points to docs/TEST_REPORT.md. | ||||
| 4 | + | ||||
| 5 | +| Phase | Status | Evidence / notes | | ||||
| 6 | +|---|---|---| | ||||
| 7 | +| 0 Discovery | **done** | docs/INFRA_DISCOVERY.md, docs/DECISIONS.md; health probes of M3U96b/M4M64c; SDK docs verified. | | ||||
| 8 | +| 1 Foundation | **done** | pnpm monorepo, Biome, Vitest, zod schemas, Drizzle schema (42 tables) + migration, storage/auth/observability/research-contract/maclustr-adapter packages, `.env.example`. `pnpm lint`, `pnpm typecheck`, unit tests green. | | ||||
| 9 | +| 2 Public experience | **done** | All public routes, design system, PDF viewer, citation exports, JSON-LD, sitemap/robots/RSS/Atom, staging-only sample flagged synthetic and excluded in production. 25/25 Playwright e2e incl. axe a11y and responsive screenshots. | | ||||
| 10 | +| 3 Auth + admin | **done** | One-time bootstrap `admin123` → forced rotation (verified in production on 2026-10-30; admin123 now rejected), Argon2id, sessions/CSRF/rate limits/lockout/audit, full admin console (overview, new research, projects, runs with live timeline, MacLustr, manuscripts, quality, publications, templates, settings). | | ||||
| 11 | +| 4 Persistent orchestration | **done** | Postgres queue with SKIP LOCKED, leases + fencing tokens, heartbeats, checkpoints, cancel/retry, event ledger with reconnect cursors (SSE), artifact ingestion with digests, simulated worker for tests. 12 integration tests (duplicate launch, fencing, crash/expiry/resume, cancel, runtime cap, synthetic never publishes, idempotent publication). | | ||||
| 12 | +| 5 Real MacLustr execution | **done** | Worker installed on M4M64c as `asworker` (LaunchDaemon, pf egress rules, Seatbelt sandbox); enrolled with the control plane on M3U96b over the LAN; self-test proves CLAUDE.md loading, in-workspace output, and enforcement denials (hook + sandbox). Milestone research run launched from the admin API (see Phase 6). | | ||||
| 13 | +| 6 End-to-end research production | **in progress** | Stages 0–8 implemented as checkpointed SDK sessions (preflight → framing → literature → design → execute → robustness → manuscript → independent review → revision/reproduction), scholarly connectors (OpenAlex/Crossref/arXiv/Semantic Scholar/PubMed) with platform-side verification and generated BibTeX, tectonic builds, page previews. First real run (empirical, Okun's law G7) and an intentionally invalid run (clinical brief) launched 2026-10-30; a second-mode run follows. Results recorded in TEST_REPORT.md when complete. | | ||||
| 14 | +| 7 Quality + publication | **done (code), pending live evidence** | 16 deterministic gates incl. hard blockers; server-side decision; transactional idempotent publication with `AS-WP-YYYY-NNNN` sequence, promotion allow-list, corrections/retractions/withdrawal; founder halts. Live auto-publication awaits the milestone run. | | ||||
| 15 | +| 8 Hardening + deployment | **mostly done** | Deployed via `mld` (M3U96b:8470, pm2 web + scheduler + nightly backup), tunnel route `autoscholar.ai → M3U96b:8470` created on BHS64; restore test executed; security headers/CSP; docs/SECURITY.md. **Blocker:** public HTTPS is down because the GoDaddy zone for `autoscholar.ai` currently has **no A record** (the record seen at discovery was removed the same day; Let's Encrypt refuses issuance). Founder must re-add `A @ → 51.161.112.61` (no change made by the build, per rule 7). | | ||||
| 16 | +| 9 Launch readiness | **pending** | Founder guide + LAUNCH_REPORT after the milestone run and DNS fix; test papers to be removed from production before launch. | | ||||
| 17 | + | ||||
| 18 | +## Deviations from the build instructions (see DECISIONS.md) | ||||
| 19 | +- D-02 single Next.js app serves API + UI; D-07 filesystem artifact store (no S3 in the cluster); D-15 qa/publisher run inside the scheduler process; D-16 development happened on the laptop until first deployment, then `mld adopt` made `M3U96b:~/apps/autoscholar` the git source of truth (origin spbgit `autoscholar.git`); later changes were pushed through spbgit and pulled on the node (remote-first from launch on). | ||||
| 20 | + | ||||
| 21 | +## Known gaps | ||||
| 22 | +- `www.autoscholar.ai` has no DNS record (not created). | ||||
| 23 | +- No MFA yet (schema ready). | ||||
| 24 | +- Reproduction of analyses runs under a Seatbelt profile with network access (needed for package installs); no Linux container isolation available on the cluster. | ||||
| 25 | +- Opus 5 safety classifier can flag adversarial self-test prompts; the self-test uses Sonnet for the boundary session and the production policy prompt makes the research model decline boundary-crossing instructions on its own (enforcement still verified by hooks/sandbox). | ||||
modified
docs/INFRA_DISCOVERY.md
+3 −0
@@ -60,6 +60,9 @@ Ingress path at deploy time: `https://autoscholar.ai` → Caddy (BHS64, automati | |||||
| 60 | | DOI agency (Crossref/DataCite) | **no** | DOIs are not issued; UI shows none | | 60 | | DOI agency (Crossref/DataCite) | **no** | DOIs are not issued; UI shows none | | ||
| 61 | | Email / SMTP | no | not required in v1 | | 61 | | Email / SMTP | no | not required in v1 | | ||
| 62 | 62 | ||||
| 63 | +## 7b. Update 2026-10-30 afternoon — DNS regression (external blocker) | ||||
| 64 | +- **[verified]** `dig @ns43.domaincontrol.com autoscholar.ai A` → NOERROR with **no A record** (SOA serial 2026103001, i.e. the zone changed on 2026-10-30). The A record observed in the morning is gone. Caddy on BHS64 logs `no valid A records found for autoscholar.ai` when requesting the certificate, so `https://autoscholar.ai` cannot be served until the founder restores `A @ → 51.161.112.61` at GoDaddy. The tunnel route is already in place and will start working automatically once DNS resolves (Caddy retries issuance). | ||||
| 65 | + | ||||
| 63 | ## 8. Unknowns and blockers | 66 | ## 8. Unknowns and blockers | ||
| 64 | 67 | ||||
| 65 | - **[assumption]** The Agent SDK sandbox (`sandbox.enabled`) works under a LaunchDaemon-run non-GUI user on macOS 26.4. Verified in Phase 5 with a real run; `failIfUnavailable: true` makes the worker refuse to run unsandboxed. | 68 | - **[assumption]** The Agent SDK sandbox (`sandbox.enabled`) works under a LaunchDaemon-run non-GUI user on macOS 26.4. Verified in Phase 5 with a real run; `failIfUnavailable: true` makes the worker refuse to run unsandboxed. | ||
modified
scripts/backup.sh
+4 −4
@@ -6,10 +6,10 @@ export PATH=/opt/homebrew/opt/postgresql@17/bin:/opt/homebrew/bin:/usr/bin:/bin | |||||
| 6 | STAMP=$(date +%Y%m%d-%H%M) | 6 | STAMP=$(date +%Y%m%d-%H%M) | ||
| 7 | LOCAL="${ARTIFACT_ROOT%/artifacts}/backups"; mkdir -p "$LOCAL/daily" "$LOCAL/weekly" | 7 | LOCAL="${ARTIFACT_ROOT%/artifacts}/backups"; mkdir -p "$LOCAL/daily" "$LOCAL/weekly" | ||
| 8 | pg_dump --no-owner -Fc "$DATABASE_URL" > "$LOCAL/daily/autoscholar-$STAMP.dump" | 8 | pg_dump --no-owner -Fc "$DATABASE_URL" > "$LOCAL/daily/autoscholar-$STAMP.dump" | ||
| 9 | −[ "$(date +%u)" = 7 ] && cp "$LOCAL/daily/autoscholar-$STAMP.dump" "$LOCAL/weekly/" | 9 | +if [ "$(date +%u)" = 7 ]; then cp "$LOCAL/daily/autoscholar-$STAMP.dump" "$LOCAL/weekly/"; fi | ||
| 10 | −ls -1t "$LOCAL/daily"/*.dump | tail -n +8 | xargs -r rm -f | 10 | +(ls -1t "$LOCAL/daily"/*.dump 2>/dev/null | tail -n +8 | xargs rm -f) || true | ||
| 11 | −ls -1t "$LOCAL/weekly"/*.dump 2>/dev/null | tail -n +5 | xargs -r rm -f | 11 | +(ls -1t "$LOCAL/weekly"/*.dump 2>/dev/null | tail -n +5 | xargs rm -f) || true | ||
| 12 | −sha256sum "$LOCAL/daily/autoscholar-$STAMP.dump" 2>/dev/null || shasum -a 256 "$LOCAL/daily/autoscholar-$STAMP.dump" | 12 | +shasum -a 256 "$LOCAL/daily/autoscholar-$STAMP.dump" | ||
| 13 | ssh -o BatchMode=yes "${BACKUP_TARGET%%:*}" "mkdir -p ${BACKUP_TARGET#*:}/dumps ${BACKUP_TARGET#*:}/artifacts" | 13 | ssh -o BatchMode=yes "${BACKUP_TARGET%%:*}" "mkdir -p ${BACKUP_TARGET#*:}/dumps ${BACKUP_TARGET#*:}/artifacts" | ||
| 14 | rsync -a --delete "$LOCAL/daily/" "$BACKUP_TARGET/dumps/" | 14 | rsync -a --delete "$LOCAL/daily/" "$BACKUP_TARGET/dumps/" | ||
| 15 | rsync -a "$ARTIFACT_ROOT/" "$BACKUP_TARGET/artifacts/" | 15 | rsync -a "$ARTIFACT_ROOT/" "$BACKUP_TARGET/artifacts/" | ||
modified
services/worker/src/node-meta.ts
+12 −2
@@ -20,11 +20,21 @@ function tryExec(cmd: string, args: string[]): string | undefined { | |||||
| 20 | } | 20 | } | ||
| 21 | 21 | ||||
| 22 | export function sdkVersion(): string { | 22 | export function sdkVersion(): string { | ||
| 23 | + for (const p of [new URL("../node_modules/@anthropic-ai/claude-agent-sdk/package.json", import.meta.url), new URL("../../../node_modules/@anthropic-ai/claude-agent-sdk/package.json", import.meta.url)]) { | ||||
| 24 | + try { | ||||
| 25 | + return (JSON.parse(readFileSync(p, "utf8")) as { version: string }).version; | ||||
| 26 | + } catch { | ||||
| 27 | + /* try next */ | ||||
| 28 | + } | ||||
| 29 | + } | ||||
| 23 | try { | 30 | try { | ||
| 24 | − return (require("@anthropic-ai/claude-agent-sdk/package.json") as { version: string }).version; | 31 | + const entry = require.resolve("@anthropic-ai/claude-agent-sdk"); | ||
| 32 | + const m = entry.match(/claude-agent-sdk@(\d+\.\d+\.\d+)/); | ||||
| 33 | + if (m) return m[1]!; | ||||
| 25 | } catch { | 34 | } catch { | ||
| 26 | − return "unknown"; | 35 | + /* unknown */ | ||
| 27 | } | 36 | } | ||
| 37 | + return "unknown"; | ||||
| 28 | } | 38 | } | ||
| 29 | 39 | ||||
| 30 | export function workerVersion(): string { | 40 | export function workerVersion(): string { | ||
| 31 | 41 | ||||