Isolation audit — WP29
Boundaries (what actually separates executors from hidden material)
| Boundary | Mechanism | Verification |
|---|---|---|
| Physical/host | Executors run on cluster nodes (M3U96a, M3U96b, M4M64a, M4M64b); oracles, evaluators, this repository and the series exist only on the laptop (and ~/wp29-oracles/ on the laptop) |
nodes hold no copy (no rsync of oracles ever; tests/isolation/test_bundle_hygiene.py checks every public bundle for forbidden material); node → laptop SSH port 22 unreachable (nc -z from M3U96a, 2026-10-29) |
| Workspace | one fresh directory per run containing exactly the public bundle | isolation.prepare_workspace compares the remote SHA-256 manifest with the local bundle manifest before launch; mismatch ⇒ infrastructure error (this actually fired twice in the first smoke attempt because of an unexpanded ~, and the runs were recorded as error with no inference) |
| Instruction loading | --bare: no CLAUDE.md discovery, no hooks, plugins, MCP, auto-memory, skills; instructions injected only via --append-system-prompt-file; identical flags across arms |
sentinel probes: laptop normal mode loaded 49,077 tokens incl. the user's cluster CLAUDE.md; bare mode 144 tokens and no sentinel; on each node bare mode saw neither the project sentinel nor the node's ~/.claude/CLAUDE.md (reports/sentinel_probes_nodes.jsonl, 175–177 input tokens) |
| Tools | requested --tools Bash,Read,Write,Edit,Glob,Grep; delivered (init event of every session): Bash, Edit, Read only; no web tools, no MCP, no sub-agent tool; two built-in plugins (cc-plugin-agents-md, cc-plugin-plugin-authoring) still load under --bare; identical across arms and sessions (systems review, 497 init events) |
behaviour audit scans every tool_use input (path regex, ~ and $HOME normalised after the review found false positives): 4 of 384 runs referenced a path outside the workspace (RC0094 attempt 1 listed sibling run names; RA0008 read its own client transcript; RC0275 and RP0023 referenced ~/home in shell idioms); 0 network commands, 0 reads of other runs' files, 0 nested claude invocations |
| Prompt delivery | intended user prompt per role in protocol/arms/ |
defect A-006: the remote shell line expanded the prompt file unquoted, so only its first word reached the client in every frozen session ('Begin', 'Review', 'The', 'Your'); the continuation sessions did not read the feedback files either (1–2 turns, asked for the missing message); treatment texts (system prompt) unaffected; fixed and verified post hoc (full prompt in RD0005–RD0008, RT0025, RT0026, RR0016–RR0017) |
| Credentials | ~/wp29/.env (mode 600) on nodes, sourced by the launching shell; never written to logs or the repository; --bare never reads keychain/OAuth |
grep of runs/ for the key prefix is part of the release validation |
| Reviewer (arm D) | separate session id, separate system prompt, sees TASK.md + out/ |
residual leak path: the executor transcript exists under ~/.claude/projects on the node and could be read with Bash; no reviewer session touched .claude/projects (systems review); one reviewer (RC0159) wrote its verdict to out/out/review/review.json after a cd drift — recovered from the collected artefact (A-008); review-repair never executed in the frozen benchmark |
| Nested invocation | no Agent tool; PGAR_DEPTH guard; executors cannot launch controllers |
code review + probe |
Contamination of the task suite
Tasks were authored by Claude Code subagents (fresh contexts, no access to executor transcripts) under a frozen interface; the builder (this session) wrote the smoke task. Executor sessions are fresh (--bare, new session id, new workspace) and cannot access sibling runs, previous attempts or oracles. Pretraining familiarity with textbook methods is expected and reported per task (contamination_risk in each task card). Reference answers were accessible to: the authoring subagent (designer) and the evaluator host. No executor ever had access.
Residual risks
- Same user account on the nodes: an executor could read other runs' workspaces under
~/wp29/runs/(sibling outputs of other runs) via Bash. Behaviour audit flags any such path; none occurred in the smoke. A stricter per-run OS identity was not implemented (documented limitation). - The reviewer can in principle read the executor transcript (see above).
- The gate program is world-readable on the node; arms A/B were not told about it and never called it (0 invocations in 384 runs), which the audit records for every run.
- The nodes are not offline: they have Internet access; executors lacked web tools and issued no network command (audit), but the common preamble's sentence "There is no network access" was inaccurate.
- The harness changed after the freeze (A-004 scheduler, A-006 launcher); every run manifest records its code commit; no task, oracle, treatment text or analysis interface changed.
- Sibling exposure persists on the nodes after the study (workspaces,
.ctldirectories with arm texts, client transcript directories) under the shared user account.