spb/zyquo-agent Public MIT

1# AGENT-RESEARCH.md — How Modern Production AI Agents Work23**Zyquo Agent — Phase 0.A research document**4Compiled: 2026-07-30 · Sources: intensive web research (Anthropic engineering + platform docs, OpenAI docs, Google Gemini docs, academic papers, and production-agent write-ups — Claude Code, OpenAI Codex CLI, Cursor, Aider, OpenHands, Devin). All claims cite the source URL inline.56---78## Executive Summary910Modern production agents (Claude Code, Codex CLI, Devin, OpenHands) converge on the same skeleton: **a while-loop over a stateful conversation, keyed on the model's stop signal**. The model is given a system prompt + JSON-schema tool definitions + conversation history; it either emits tool calls (loop continues: execute, append results, re-send) or a final text answer (loop ends). Everything else — planning, memory, safety, workspaces — is machinery wrapped around that loop:1112- **The loop** is the ReAct pattern (reason → act → observe) made native via provider tool-calling APIs. The model signals "done" through its stop reason (`end_turn` vs `tool_use` for Anthropic; `stop` vs `tool_calls` for OpenAI Chat Completions).13- **Tools** are the agent-computer interface and deserve as much design effort as prompts (Anthropic reports spending *more* time on tools than on prompts for their SWE-bench agent). Few, consolidated, well-described tools beat many thin API wrappers.14- **Context is a finite resource** that degrades with length ("context rot"). Long-running agents survive via compaction (summarize old turns), output offloading (big tool outputs → files + in-context reference), persistent memory files (`MEMORY.md`/`NOTES.md`), and sub-agents with isolated context windows.15- **Workspaces** isolate each task in a working directory; file tracking + checkpoints make agent actions reversible and auditable.16- **Planning** is externalized into a visible, updatable todo list (Claude Code's TodoWrite/Task tools); re-planning after failure is a normal, expected path, not an error state.17- **Reliability** comes from loop guards (max iterations, budgets, repetition detection), and **safety** comes from a layered permission system — deny → ask → allow rule evaluation, read-only auto-approval, destructive-pattern circuit breakers, and human approval gates. The industry lesson (Cursor's bypassed denylist) is that pattern-matching alone is best-effort, not a security boundary; approvals + auditing must back it up.18- **Evaluation** is end-to-end and outcome-based: containerized task suites (SWE-bench, Terminal-Bench, OSWorld) with programmatic verifiers, plus LLM-judge rubrics and human trajectory inspection.1920The final section maps every finding onto Zyquo Agent's planned components (`AgentLoop`, `Planner`, `MemoryManager`, `LoopGuard`, `ToolRegistry`, `PolicyEngine`, `WorkspaceManager`).2122---2324## 1. The Core Agentic Loop2526### 1.1 Workflows vs. agents2728Anthropic's "Building Effective Agents" (<https://www.anthropic.com/engineering/building-effective-agents>) draws the canonical distinction:2930- **Workflows**: LLMs and tools orchestrated through *predefined code paths* (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer).31- **Agents**: systems where "LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks."3233Zyquo Agent is squarely the second kind. Anthropic's guidance for agents: they need **"ground truth from the environment at each step (such as tool call results or code execution) to assess progress"**, explicit **stopping conditions** to retain control, extensive sandboxed testing, guardrails, and human checkpoints — because autonomy compounds both cost and error.3435### 1.2 The research lineage3637- **ReAct** (Yao et al., 2022, <https://arxiv.org/abs/2210.03629>): interleave *reasoning traces* with *actions* in a thought → action → observation cycle. Reasoning induces and tracks plans; actions ground the model in external reality; observations feed exception handling. ReAct beat imitation/RL baselines on ALFWorld (+34% absolute) and WebShop (+10%) and reduces hallucination vs. pure chain-of-thought. Every modern tool-calling loop is ReAct with the "action" formalized as a native tool call.38- **Reflexion** (Shinn et al., 2023, <https://arxiv.org/abs/2303.11366>): agents improve via *verbal* reinforcement — after a failure, the agent reflects in natural language and stores that reflection in an **episodic memory buffer** consulted on the next attempt. No weight updates; 91% on HumanEval vs. GPT-4's 80% baseline. Production translation: when a step fails, have the model articulate *why* and keep that diagnosis in context (or in a notes file) before retrying.39- **Plan-and-Execute** (LangChain/LangGraph pattern, <https://blog.langchain.com/planning-agents/>, tutorial: <https://langchain-opentutorial.gitbook.io/langchain-opentutorial/17-langgraph/03-use-cases/05-langgraph-plan-and-execute>): a **planner** turns the objective into a structured checklist; an **executor** (often itself a small ReAct loop) works one step at a time; a **replanner** consumes `past_steps` and either refines the remaining plan or emits the final answer. Separating planning from execution is more reliable than pure step-by-step ReAct for complex multi-step tasks.4041Production agents blend all three: an up-front plan (Plan-and-Execute), a ReAct inner loop per step, and Reflexion-style self-critique on failure.4243### 1.3 Anatomy of one loop iteration (Anthropic wire format)4445From Anthropic's tool-use docs (<https://platform.claude.com/docs/en/agents-and-tools/tool-use/how-tool-use-works>), the canonical shape is a `while` loop keyed on `stop_reason`:46471. **Send** a request: system prompt + `tools` array (JSON schemas) + full message history.482. **Model responds.** If it wants to act, the response has `stop_reason: "tool_use"` and one or more `tool_use` content blocks, each carrying `{id, name, input}` (input is a JSON object matching the tool's schema). The response may also contain `text` blocks (the "thought" — this is the ReAct reasoning trace) before the tool calls.493. **Execute** each tool in your runtime. The model *never* executes anything itself — "it emits a structured request, your code runs the operation, and the result flows back."504. **Append** the assistant message verbatim, then a **user** message containing one `tool_result` block per call: `{type: "tool_result", tool_use_id: <matching id>, content: <output>, is_error: <bool>}`.515. **Repeat** from step 2 while `stop_reason == "tool_use"`.5253**How the model signals "I'm done":** the loop exits on any other stop reason — `"end_turn"` (final answer produced), or abnormal ones the app must handle: `"max_tokens"`, `"stop_sequence"`, `"refusal"` (see <https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons>). There is no separate "done" tool needed: *not calling a tool* is the done-signal. Provider equivalents:5455| Provider | "I want to act" | "I'm done" |56|---|---|---|57| Anthropic Messages | `stop_reason: "tool_use"` + `tool_use` blocks | `stop_reason: "end_turn"` |58| OpenAI Chat Completions | `finish_reason: "tool_calls"` + `message.tool_calls[]` | `finish_reason: "stop"` |59| OpenAI Responses API | `function_call` output items | plain message output items |60| Gemini | candidate parts contain `functionCall` | parts contain only text |6162A useful robustness convention on top of stop reasons (used by several agent frameworks and recommended in agent-loop tutorials, e.g. <https://claude-world.com/tutorials/s01-the-agent-loop/>): also require the final text to contain an explicit completion statement, and treat an empty final answer after tool activity as a stall.6364**Worked example — one full Anthropic iteration** (wire shapes from <https://platform.claude.com/docs/en/agents-and-tools/tool-use/how-tool-use-works>):6566```jsonc67// → request N68{ "system": "...", "tools": [{ "name": "bash", "description": "...", "input_schema": {...} }],69  "messages": [ ...history..., { "role": "user", "content": "Set up the project and run the tests" } ] }7071// ← response N: the model thinks, then acts72{ "stop_reason": "tool_use",73  "content": [74    { "type": "text", "text": "I'll check whether a package manifest exists first." },   // ReAct thought75    { "type": "tool_use", "id": "toolu_01A", "name": "bash",76      "input": { "command": "ls -la" } } ] }                                              // action7778// → request N+1: assistant turn appended verbatim + observation79{ "messages": [ ...,80    { "role": "assistant", "content": [ /* the two blocks above */ ] },81    { "role": "user", "content": [82      { "type": "tool_result", "tool_use_id": "toolu_01A",83        "content": "Package.swift\nSources\nTests", "is_error": false } ] } ] }8485// ← eventually: no tool_use blocks ⇒ done86{ "stop_reason": "end_turn", "content": [ { "type": "text", "text": "All 12 tests pass. ..." } ] }87```8889The same skeleton, re-skinned: OpenAI puts the action in `message.tool_calls` and the observation in a `role:"tool"` message; Gemini puts them in `functionCall`/`functionResponse` parts. The `AgentStep` model should capture exactly this triple — thought text, tool call(s), tool result(s) — plus the stop reason.9091### 1.4 System-prompt anatomy for an agent9293The loop is only as good as the contract the system prompt establishes. Findings across the sources:9495- **Altitude**: Anthropic's context-engineering post prescribes the "Goldilocks zone" — not brittle hardcoded if-then logic, not vague platitudes; "specific enough to guide behavior effectively, yet flexible enough to provide the model with strong heuristics" (<https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents>).96- **Canonical sections** (visible in Claude Code's prompt structure and Anthropic's agent guidance): (1) role and capabilities; (2) the environment — workspace path, OS, what the tools can touch; (3) tool-usage policy — when to use which tool, parallel-call rules, output-size discipline; (4) planning discipline — maintain the todo list, mark items in progress/completed as you go; (5) safety expectations — actions are gated, never attempt to bypass approval, prefer workspace-relative paths, never `sudo`; (6) the done-convention — "when the task is complete, respond without calling tools, summarizing what was done and verifying success criteria."97- **Standing instructions live outside the transcript**: Claude Code re-injects CLAUDE.md and memory after every compaction (<https://code.claude.com/docs/en/best-practices>); the agent's system prompt + plan must likewise be compaction-immune (see §3).98- **Ground the model in verification**: instruct it to verify its own work with tools (run the tests, list the directory) before declaring done — this operationalizes "ground truth from the environment at each step" (<https://www.anthropic.com/engineering/building-effective-agents>).99100### 1.5 Native tool calling vs. structured output101102Native tool calling (function schemas + typed call/response blocks) is strictly preferred over asking the model to emit parseable text/JSON in prose:103104- Models are **fine-tuned on the native format**, so calls are better-formed and error recovery is better. Anthropic explicitly notes its published `bash`/`text_editor` schemas are "trained-in": "Claude has been optimized on thousands of successful trajectories that use these exact tool signatures, so it calls them more reliably" (<https://platform.claude.com/docs/en/agents-and-tools/tool-use/how-tool-use-works>).105- The docs' rule of thumb: "if you're writing a regex to extract a decision from model output, that decision should have been a tool call."106- OpenAI's **strict mode** (`strict: true`) goes further: constrained decoding guarantees the arguments match the schema exactly (requires `additionalProperties: false` and all fields in `required`; optionals expressed as `"type": ["string","null"]`) (<https://developers.openai.com/api/docs/guides/function-calling>).107108Structured output (JSON mode) remains useful for *final* answers of a fixed shape, not for actions.109110### 1.6 Streaming the loop111112Everything in an iteration can and should stream to the UI:113114- **Anthropic**: text arrives as `text_delta`; tool-call arguments arrive as `input_json_delta` events carrying `partial_json` string fragments that the client concatenates ("the chunks do not respect JSON boundaries") (<https://platform.claude.com/docs/en/build-with-claude/streaming>). The beta `fine-grained-tool-streaming-2025-05-14` header removes server-side buffering so large arguments stream immediately — at the cost that the accumulated JSON may be *invalid/partial* if the stream ends early (e.g. `max_tokens`), so clients must parse defensively (<https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/fine-grained-tool-streaming>).115- **OpenAI**: `response.function_call_arguments.delta` / `.done` events (Responses API), or `delta.tool_calls[i].function.arguments` fragments (Chat Completions) — accumulate per tool-call index until `finish_reason` arrives.116- **Gemini**: function-call arguments arrive as deltas that must be aggregated before execution (<https://ai.google.dev/gemini-api/docs/function-calling>).117118Implication: the normalized `ProviderClient` interface must expose a unified stream of events — `textDelta`, `thinkingDelta`, `toolCallStarted(id, name)`, `toolCallArgumentsDelta(id, fragment)`, `toolCallCompleted(id, input)`, `turnCompleted(stopReason)` — and the UI renders tool arguments live as they stream.119120---121122## 2. Tool Design123124### 2.1 How tools are defined125126All three provider families use the same conceptual triple — **name, description, JSON-Schema parameters**:127128- **Anthropic**: `tools: [{name, description, input_schema}]` where `input_schema` is JSON Schema (<https://platform.claude.com/docs/en/agents-and-tools/tool-use/how-tool-use-works>).129- **OpenAI**: `tools: [{type: "function", function|name/description/parameters, strict}]` — full JSON Schema features: "property types, enums, descriptions, nested objects, and recursive objects" (<https://developers.openai.com/api/docs/guides/function-calling>).130- **Gemini**: `tools: [{functionDeclarations: [{name, description, parameters}]}]` — parameters use an **OpenAPI-subset** schema, notably more restricted than full JSON Schema (<https://ai.google.dev/gemini-api/docs/function-calling>).131132Practical consequence for a multi-provider agent: define tools once in an internal type, generate provider-specific schemas from it, and keep schemas within the **lowest common denominator** (flat-ish objects, `type`/`enum`/`description`/`required` — avoid `oneOf`, `$ref`, deep recursion) so a single tool definition works everywhere.133134### 2.2 Threading tool calls and results, per provider135136The three wire formats the loop must normalize:137138**Anthropic** — content blocks inside normal messages:139```140assistant: [ {type:"text", ...thought...}, {type:"tool_use", id:"toolu_1", name:"bash", input:{...}} ]141user:      [ {type:"tool_result", tool_use_id:"toolu_1", content:"...", is_error:false} ]142```143All `tool_result` blocks must come **first** in the next user message, one per `tool_use` id, and results for *all* parallel calls go in **one** user message.144145**OpenAI Chat Completions** — dedicated roles:146```147assistant: { content:null, tool_calls:[{id:"call_1", type:"function", function:{name:"bash", arguments:"{\"command\":...}"}}] }148tool:      { role:"tool", tool_call_id:"call_1", content:"..." }149```150Arguments are a **JSON string** (must be parsed; may rarely be malformed without strict mode). The Responses API instead uses `function_call` output items answered by `function_call_output` items keyed on `call_id` (<https://developers.openai.com/api/docs/guides/function-calling>).151152**Gemini** — parts inside contents:153```154model: { parts:[{functionCall:{name:"bash", args:{...}}}] }155user:  { parts:[{functionResponse:{name:"bash", response:{...}}}] }156```157Modes via config: `AUTO` (model decides), `ANY` (must call a function), `NONE` (<https://ai.google.dev/gemini-api/docs/function-calling>). Anthropic's equivalent is `tool_choice: auto|any|tool|none`; OpenAI's is `tool_choice: auto|required|none|{function}`.158159**Error results:** all providers support signaling tool failure back to the model — Anthropic via `is_error: true` on the `tool_result`, OpenAI/Gemini by putting the error text in the result content. Best practice (universal across Claude Code/Aider/OpenHands): return *actionable* error text ("file not found: /x/y — did you mean /x/z?") so the model can self-correct in the next iteration, rather than a bare stack trace.160161### 2.3 Parallel vs. sequential tool calls162163- All three providers can emit **multiple tool calls in one assistant turn** when the calls are independent (Anthropic: multiple `tool_use` blocks; OpenAI: `tool_calls[]`, disable with `parallel_tool_calls: false`; Gemini: parallel `functionCall` parts + separate "compositional" chaining across turns).164- The runtime may execute them concurrently, but **all results must be returned together** in the next message, correlated by id.165- Dependent calls are inherently sequential — the model needs observation N before choosing action N+1. Production agents run read-only calls (multiple file reads, searches) in parallel and mutating calls sequentially; Zyquo Agent should make parallel execution a per-settings toggle and never parallelize two mutating shell commands.166167### 2.4 Best practices for tool definitions168169From Anthropic's "Writing effective tools for agents" (<https://www.anthropic.com/engineering/writing-tools-for-agents>), the ACI section of "Building Effective Agents", and OpenAI's function-calling guide:1701711. **Fewer, consolidated tools.** "More tools don't always lead to better outcomes" — build a few tools targeting high-impact workflows (`schedule_event`, not `list_users`+`list_events`+`create_event`). OpenAI: "aim for fewer than 20 functions available at the start of a turn."1722. **Descriptions are onboarding docs.** Make implicit context explicit; state when to use the tool and when *not* to; unambiguous parameter names (`user_id`, not `user`); the intern test: "an intern can correctly use the function given nothing but what you gave the model."1733. **Namespace related tools** (`file_read`, `file_write`, `file_search`) to reduce selection confusion.1744. **Token-efficient results.** Pagination/filtering/truncation with sensible defaults — Claude Code truncates tool responses at ~25,000 tokens by default; truncation messages should steer the model ("output truncated; use targeted searches instead").1755. **Return meaningful context**: semantic identifiers over UUIDs; support `concise`/`detailed` response formats.1766. **Poka-yoke**: design arguments so misuse is hard (e.g. require absolute paths to eliminate cwd ambiguity — a change Anthropic reports fixed a whole error class in their SWE-bench agent).1777. **Make invalid states unrepresentable** with enums and structure; don't make the model fill arguments the app already knows (OpenAI).1788. **Evaluate tools like code**: prototype, run realistic multi-call tasks, read the agent's reasoning to find rough edges, iterate.179180### 2.5 How leading agents design the three core tools181182**Shell/bash tool.** Anthropic's trained-in `bash_20250124` tool (<https://platform.claude.com/docs/en/agents-and-tools/tool-use/bash-tool>) has a deliberately tiny schema — `{command}` plus a `restart` flag — and the *application* owns a **persistent bash session**: "one bash process alive across tool calls, so state persists between commands. The working directory, environment variables, and any files a command creates are still there for the next command." Implementation guidance in the doc: per-command timeouts, output truncation, and treating sandboxing/command validation as the app's job. Claude Code's own Bash tool adds a per-call `timeout`, an output cap, background execution, and a natural-language `description` field the UI shows the user. OpenHands equivalently exposes `CmdRunAction` (bash) plus an IPython cell action, executed inside a Docker sandbox, with results returned as typed `Observation` events (<https://arxiv.org/abs/2407.16741>).183184**File-edit tool.** Two dominant designs:185- **String-replacement editing** — Anthropic's `str_replace_based_edit_tool` (<https://platform.claude.com/docs/en/agents-and-tools/tool-use/text-editor-tool>) with commands `view`, `create`, `str_replace`, `insert`. `str_replace` requires the `old_str` to match **exactly and uniquely** in the file — a non-unique or non-matching string is an error returned to the model. This is the same contract as Claude Code's Edit tool, and it's the key safety property: the model must prove it knows the current file content before changing it.186- **Diff formats** — Aider's edit formats (<https://aider.chat/docs/more/edit-formats.html>): `whole` (rewrite the file — simple but token-expensive), `diff` (SEARCH/REPLACE blocks styled like git conflict markers — the default because it balances token efficiency with explicit before/after context), `udiff` (unified diffs, adopted for GPT-4 Turbo to fight "lazy coding" elisions), and per-model variants (`diff-fenced` for Gemini). Lesson: the *edit format must match what the model executes reliably*, and exact-match search/replace with clear failure errors is the most robust default for tool-calling models.187188**OS-automation tools.** Anthropic's `computer` tool drives GUI via screenshots + mouse/keyboard; for macOS-native automation the practical pattern (used by Mac agent projects) is an **osascript tool**: schema `{script, language: applescript|jxa, timeout}`, executed via `/usr/bin/osascript`, with the app pre-declaring `NSAppleEventsUsageDescription` and surfacing TCC Automation prompts to the user. Because AppleScript can do anything the user can (send mail, delete files), production designs treat it like a mutating shell command: always subject to the approval gate, with the exact script shown to the user. Anthropic's computer-use guidance similarly stresses human confirmation for consequential actions and isolated environments.189190---191192## 3. Memory & Context Compression193194### 3.1 Why: context is finite and rots195196Anthropic's "Effective context engineering for AI agents" (<https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents>): "context must be treated as a finite resource with diminishing marginal returns." **Context rot** — accuracy degrading as the window fills — stems from attention stretching across n² token relationships and sparse training on very long sequences. So a long-running agent needs an active context-management strategy, not just a big window.197198### 3.2 Compaction (summarizing old turns)199200The reference implementation is **Claude Code's auto-compact** (<https://claudelog.com/faqs/what-is-claude-code-auto-compact/>, <https://okhlopkov.com/claude-code-compaction-explained/>, <https://howaiworks.ai/blog/claude-code-auto-compact-context-management>):201202- Token usage is monitored continuously; near ~95% of the effective window (threshold ≈ window minus a fixed reserve) the turn pauses.203- A **summarization pass over the whole history** produces a structured summary (task state, decisions made, files touched, next steps) that *replaces* the older turns; the session continues from the summary.204- Content loaded from disk (CLAUDE.md project instructions, memory) is **re-injected after compaction** — compaction only touches conversation history, never standing instructions.205- Manual `/compact` triggers the same pipeline on demand, optionally with focus instructions.206- **Thrashing guard**: if a single huge file/tool output refills the context immediately after each summary, Claude Code stops auto-compacting after a few attempts and surfaces an error instead of looping.207208Anthropic's context-engineering post confirms the design and the hard part: "passing the message history to the model to summarize and compress the most critical details" — the art is choosing what to preserve (decisions, unresolved bugs, plan state) vs. discard (raw tool outputs, dead-end exploration).209210### 3.3 Concrete compaction strategy (synthesized)211212The strategy production systems converge on, and the one Zyquo Agent's `MemoryManager` should implement:2132141. **Never compact**: system prompt, tool schemas, the current plan/todo list, standing instructions/memory files (re-inject after compaction).2152. **Keep verbatim**: the most recent N steps (recency matters most for the next decision).2163. **Summarize into compact structured records**: completed sub-tasks and older turns — e.g. `✔ Step 2: created venv, installed pandas 2.2 (3 commands, all exit 0)` instead of three full command transcripts.2174. **Offload large tool outputs to files**: write long stdout/logs/file dumps into the workspace (`.zyquo/outputs/step-014-stdout.txt`) and replace them in-context with a one-line reference + summary ("output 48KB, saved to …; key line: 3 tests failed in test_parser.py"). This mirrors Anthropic's context-editing result: clearing stale tool results enabled 100-turn workflows and cut token use 84% (<https://www.anthropic.com/news/context-management>).2185. **Compact preemptively** at a threshold (~80–90%), never mid-tool-execution, and always via a dedicated summarization call with an explicit "preserve: plan, key facts, open problems" prompt.2196. **Guard against thrashing** as Claude Code does.220221### 3.4 Hierarchical memory: scratchpad vs. notes files222223Two tiers, both external to the context window:224225- **Structured note-taking / agentic memory** (short-to-medium term): the agent maintains a `NOTES.md`/`MEMORY.md`/todo file **in its workspace**, writing down progress, learned facts, and open questions, and re-reading it after compaction or restart. Anthropic cites this as a core technique — memory persists "with minimal overhead" while the working context stays lean (<https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents>).226- **The memory tool** (long term, cross-session): Anthropic's `memory_20250818` tool gives the model file operations (view/create/str_replace/insert/delete/rename) over a client-managed `/memories` directory that survives conversations; combined with context editing it improved agentic-search performance 39% (<https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool>, <https://www.anthropic.com/news/context-management>). Claude Code's CLAUDE.md files are the same idea applied to user/project preferences.227- **Sub-agents as context compression**: delegating exploration to a sub-agent with its own clean window, which returns only a 1–2K-token distilled summary, keeps detail out of the orchestrator's context entirely (<https://www.anthropic.com/engineering/multi-agent-research-system>). Claude Code's Task tool works exactly this way: "the parent agent receives only the sub-agent's final output, not its internal reasoning history" (<https://code.claude.com/docs/en/sub-agents>).228- **Just-in-time retrieval over pre-loading**: keep lightweight identifiers (paths, ids) in context and load content via tools when needed, rather than stuffing everything up front.229230---231232## 4. Working Repositories / Workspaces233234### 4.1 Why isolation matters235236Every serious agent gives the model a **bounded working directory** (and often a stronger sandbox) for three reasons: blast-radius containment (a bad `rm` hits scratch space, not the user's home), reproducibility/auditability (everything the task produced lives in one folder), and clean state per task (no cross-task contamination).237238Reference points:239240- **OpenHands**: all code/bash execution happens inside a **Docker-sandboxed runtime**; agent↔environment interaction is an event stream of typed Actions and Observations, and each conversation binds to a workspace (local dir mounted into the sandbox, or a remote workspace) (<https://arxiv.org/abs/2407.16741>, SDK: <https://arxiv.org/html/2511.03690v1>).241- **Codex CLI**: OS-level sandboxing — **macOS Seatbelt** (`sandbox-exec` kernel-enforced profiles) and Linux **Landlock + seccomp** — restricting file writes to the workspace and blocking network unless granted; sandbox level is chosen independently of the approval mode (<https://www.vincentschmalbach.com/how-codex-cli-flags-actually-work-full-auto-sandbox-and-bypass/>, <https://agent-safehouse.dev/docs/agent-investigations/codex>).242- **SWE-bench / Terminal-Bench**: evaluation itself runs each task in a pinned Docker container — the "workspace = container + repo checkout" mapping is now the standard unit of agent work (<https://openai.com/index/introducing-swe-bench-verified/>, <https://arxiv.org/abs/2601.11868>).243- **Claude Code**: softer model — permissions scope reads to the launch directory + `additionalDirectories`, writes require approval, and an optional OS sandbox can enforce path restrictions on *all* subprocesses (the docs note plain Read/Edit deny rules "don't apply to arbitrary subprocesses… For OS-level enforcement… enable the sandbox") (<https://code.claude.com/docs/en/permissions>).244245### 4.2 Session ↔ folder mapping, file tracking, checkpoints246247- **Session→folder**: one task = one directory is the norm (OpenHands conversation↔workspace; Devin's cloud VM per session; Zyquo Agent: `~/Library/Application Support/ZyquoAgent/Workspaces/<task-id>/`). The workspace is the shell tool's cwd and FileTools' root; reopening a task reattaches its folder.248- **File tracking**: agents record every file they create/modify — Claude Code tracks file state per session (its Edit tool refuses to edit a file that wasn't Read first, a cheap way to prevent blind overwrites); OpenHands records `FileEditObservation`s in the event stream. This powers UI badges (created/modified) and audit.249- **Checkpoints**: Claude Code **automatically checkpoints file state at every user prompt**; `/rewind` (or Esc-Esc) restores *code*, *conversation*, or both, with checkpoints persisted across sessions (~30-day retention) (<https://code.claude.com/docs/en/checkpointing>, <https://claudelog.com/faqs/how-to-use-checkpoints-in-claude-code/>). Cursor pioneered per-request checkpoints; Claude Code's separation of code-restore vs. conversation-restore is the more refined design. Implementation options: git-based shadow snapshots or copy-on-write file snapshots inside the workspace; the key property is that checkpoints capture only agent-touched files, cheaply, at step boundaries.250251---252253## 5. Planning & Task Decomposition254255### 5.1 Externalized todo lists256257Claude Code makes the plan a first-class, model-maintained artifact: the **TodoWrite** tool (now evolved into structured **TaskCreate/TaskUpdate/TaskGet/TaskList** tools as of v2.1.142) has the agent write a task list with per-item states `pending → in_progress → completed`, updated *as it works*, and the harness renders it live to the user (<https://code.claude.com/docs/en/agent-sdk/todo-tracking>, <https://claudelog.com/faqs/what-is-todo-list-in-claude-code/>). Benefits documented: the plan is observable (user sees progress), it disciplines the model (the harness nudges it to keep exactly one item `in_progress` and mark items done immediately), and it survives compaction because it lives outside raw conversation text.258259Anthropic's context-engineering post frames the same practice as memory: to-do lists are "structured note-taking" that maintains coherence across long horizons.260261### 5.2 Plan-first modes and mid-run re-planning262263- **Plan mode**: Claude Code's `plan` permission mode lets the agent read/explore but not mutate, producing a plan the user approves before execution (<https://code.claude.com/docs/en/permissions>). The recommended workflow is explicitly **explore → plan → code → commit** (<https://code.claude.com/docs/en/best-practices>).264- **Re-planning is the normal path**: the LangGraph plan-and-execute pattern includes a replanner node that revises remaining steps after each execution (<https://blog.langchain.com/planning-agents/>). Devin write-ups make the same point: "Devin reasons about the situation using its full context — what the task is, what the test failures say, what it has done so far — and chooses the most appropriate path forward. The plan changes, and that's not a failure state — it's the system working correctly" (<https://cognition.com/blog/how-cognition-uses-devin-to-build-devin>). Devin's design also stresses long-horizon memory of "what it tried, what worked, what failed, and why."265- Trigger re-planning on: a step failing twice, an assumption invalidated by an observation, or the user editing the plan.266267### 5.3 Sub-agents and delegation268269- **Claude Code Task tool / subagents**: the orchestrator spawns a subagent with a fresh context window containing only the delegation prompt; the subagent runs its own tool loop and returns a concise report; up to ~10 run concurrently (<https://code.claude.com/docs/en/sub-agents>). Used for exploration, research, and parallelizable independent work.270- **Anthropic's multi-agent research system** (<https://www.anthropic.com/engineering/multi-agent-research-system>): orchestrator-workers at scale. Hard-won lessons: the lead agent must **save its plan to memory** before spawning workers (context may overflow); delegation prompts must carry *objective, output format, tool guidance, and boundaries* (vague prompts caused duplicated work); **scale effort to complexity** (one agent for a simple lookup, 10+ for open research); errors compound in stateful multi-step systems, so agents must be able to *resume from checkpoints* rather than restart.271- **Counterpoint — Cognition's "Don't Build Multi-Agents"** (<https://cognition.ai/blog/dont-build-multi-agents>): for *coding/acting* tasks (vs. read-only research), parallel agents that can't see each other's context make conflicting decisions; principles: share full context ("actions carry implicit decisions"), prefer a single continuous agent with strong context compression. Synthesis for Zyquo Agent: **one primary loop**; sub-agents only for read-only exploration/summarization, never for concurrent mutation of the same workspace.272273---274275## 6. Reliability & Control276277### 6.1 Loop guards278279Anthropic's guidance: agents need explicit **stopping conditions** — max iterations and checkpoints — "to maintain control" (<https://www.anthropic.com/engineering/building-effective-agents>). Concrete guards used across production systems and frameworks:280281- **Max steps/iterations**: a hard per-task cap (framework defaults range ~10–50; agent SDKs expose `max_turns`). Hitting it should pause-and-ask, not silently die.282- **Token/cost budget**: track cumulative input+output tokens per task; warn at a threshold, pause at the cap. Anthropic's multi-agent post notes agents can burn ~15× the tokens of a chat, so budgets are economic guards too.283- **Wall-clock budget** and **per-command timeout** (the bash-tool docs make per-command timeouts the app's responsibility).284- **Repetition detection**: same tool + same (normalized) arguments failing repeatedly ⇒ trip. Reflexion's insight applies: force a self-critique turn ("the last two attempts failed with X; state a different approach") before allowing a retry.285- **Stall / no-progress detection**: N consecutive iterations with no plan-item state change, no file mutation, and no new information ⇒ pause and ask the user. Claude Code's compaction thrashing detector is the same pattern applied to memory.286- **Server-side analogue**: Anthropic's own internal loop caps iterations and returns `stop_reason: "pause_turn"` so the client can choose to continue — evidence that "pause, hand control back" is the correct trip behavior, not abort (<https://platform.claude.com/docs/en/agents-and-tools/tool-use/how-tool-use-works>).287288### 6.2 Error recovery289290- Feed tool errors back as observations (Anthropic `is_error: true`) with actionable text; the model self-corrects on the next iteration — this is the loop's built-in recovery mechanism.291- Distinguish *retryable* (transient network, timeout) from *diagnostic* (compile error — the model should read it) from *fatal* (permission denied by policy — surface to user).292- **Resume, don't restart**: build the system so a crashed/interrupted task can resume from its transcript + workspace + checkpoints (<https://www.anthropic.com/engineering/multi-agent-research-system>).293- Handle abnormal stop reasons explicitly: `max_tokens` mid-tool-call means an incomplete action that must not be executed; `refusal` should surface to the user (<https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons>).294295### 6.3 Human-in-the-loop approval gates: how the leaders do it296297**Claude Code** (<https://code.claude.com/docs/en/permissions>) — the most instructive design, worth reproducing in detail:298299- **Tiered by tool type**: read-only tools (file reads, Grep) run without approval *within the working directory + additional directories*; **Bash commands require approval except a built-in read-only set** (`ls`, `cat`, `pwd`, `grep`, `head`, `tail`, `which`, `diff`, read-only `git`, …); file edits require approval (remembered until session end); Bash approvals can be remembered permanently per repository.300- **Rules**: `allow` / `ask` / `deny` lists evaluated in strict order **deny → ask → allow**, first match wins; specificity does *not* override order (a broad `Bash(aws *)` deny beats a narrow allow). Syntax: `Tool` or `Tool(specifier)` with globs — `Bash(npm run test *)`, `Read(~/.zshrc)`, `Edit(/src/**)` (gitignore-style path patterns with `//` = filesystem root, `~/` = home, `/` = settings-source anchor), `WebFetch(domain:example.com)`.301- **Compound-command awareness**: shell operators are parsed — `Bash(safe-cmd *)` does **not** authorize `safe-cmd && other-cmd`; each subcommand must independently match. Known wrappers (`timeout`, `nice`, `nohup`, env-var prefixes, bare `xargs`) are stripped before matching; exec-capable wrappers (`watch`, `find -exec`, `devbox run`) deliberately can't be prefix-approved.302- **Modes**: `default` (prompt on first use per tool), `acceptEdits` (auto-accept file edits + benign fs commands in-workspace), `plan` (read-only exploration), `dontAsk` (auto-deny anything not pre-approved), `bypassPermissions` (skip prompts — docs say to use it only "in isolated environments like containers or VMs") — and even bypass keeps **circuit breakers**: `rm -rf /` and `rm -rf ~` *always* prompt, including when hidden inside `$(...)`/backtick substitutions.303- **Fragility honesty**: the docs warn that argument-constraining patterns (`Bash(curl http://github.com/ *)`) are bypassable via flags/redirects/variables, and recommend structural fixes (deny curl entirely; use a domain-scoped fetch tool; hooks for validation).304- **Risk explanation UI**: on a Bash prompt, Ctrl+E asks the model itself to explain the command and label it Low/Med/High risk.305306**Codex CLI** — approvals and sandbox are **orthogonal axes**: approval policies from `untrusted`/suggest (approve everything) through `on-request`/auto-edit (edits auto, commands ask) to `never`/full-auto; sandbox levels `read-only` → `workspace-write` → `danger-full-access`, enforced at the **OS level** (Seatbelt/Landlock) so even "full-auto" runs are contained unless the user explicitly disables the sandbox (<https://inventivehq.com/knowledge-base/openai/how-to-configure-sandbox-modes>, <https://www.vincentschmalbach.com/how-codex-cli-flags-actually-work-full-auto-sandbox-and-bypass/>).307308**Cursor** — Run Modes: Auto-review (default; allowlisted commands run, shell is sandboxed when possible, everything else goes through an **LLM safety classifier** that checks the call against the user's intent), Allowlist, and Run Everything; plus user allowlists/denylists (<https://cursor.com/docs/agent/security>). **The cautionary tale**: security researchers found four distinct bypasses of Cursor's denylist (subshells, string manipulation, etc.), and Cursor's own docs concede the guardrails are "best-effort… rather than a hard security boundary" (<https://www.backslash.security/blog/cursor-ai-security-flaw-autorun-denylist>, <https://www.theregister.com/2025/07/21/cursor_ai_safeguards_easily_bypassed/>).309310**Design conclusions for a policy engine** (synthesized):3111. Deny → ask → allow, first-match, no specificity override — simple to reason about, hard to misconfigure.3122. Parse shell structure (subcommands, wrappers, substitutions) before matching; never regex the raw string only.3133. Auto-allow only a **curated read-only command set**; everything mutating asks by default.3144. Destructive patterns (`rm -rf` at scale, `sudo`, `curl | sh`, writes to system paths, `mkfs`/`diskutil`, `killall`, `launchctl`, `defaults write` outside app domain) are **always-ask circuit breakers in every mode** — exactly like Claude Code's `rm -rf ~` breaker surviving bypassPermissions.3155. Treat pattern-matching as UX, not security: pair it with approvals, workspace scoping, and a complete audit trail (the Cursor lesson).3166. "Approve & remember" should save the *narrowest* rule that covers the action (Claude Code saves per-subcommand rules, max 5, for compound commands).317318### 6.4 Deciding an action is risky319320Signals used across these systems: (a) static classification — command family (read/mutate/destroy), target path (inside vs. outside workspace, system paths, dotfiles), privilege (`sudo`), network + execution combos (`curl | sh`), irreversibility; (b) contextual classification — does the action match the user's stated intent (Cursor's classifier, Claude Code's Ctrl+E explainer); (c) mode — the same action can be auto-run in autonomous mode but asked in manual mode, *except* the always-ask class. A hybrid — fast rule-based classifier with an always-ask floor, optionally augmented by a model-generated explanation at prompt time — is the state of the art.321322---323324## 7. Evaluation325326### 7.1 Benchmark task suites (the pattern to copy)327328- **SWE-bench Verified** (<https://openai.com/index/introducing-swe-bench-verified/>, <https://www.swebench.com>): 500 human-validated real GitHub issues from 12 Python repos. The agent gets a Docker container with the repo at the pre-fix commit + the issue text; success = the produced patch makes hidden **FAIL_TO_PASS** tests pass (proves the fix) while **PASS_TO_PASS** tests still pass (proves no regression). Key ideas: real tasks, hermetic pinned environments, and a *deterministic programmatic oracle* — success is judged by end state, not by how the agent got there.329- **Terminal-Bench** (<https://arxiv.org/abs/2601.11868>, <https://www.tbench.ai>): 89+ hand-crafted end-to-end terminal tasks (compiling, training, sysadmin, security, data science). Each task = natural-language instruction + Docker environment + **verification test suite** + an oracle solution proving solvability. This is the closest published analogue to Zyquo Agent's domain (arbitrary shell work, not just code patches).330- **OSWorld / OSWorld 2.0** (<https://os-world.github.io>, <https://arxiv.org/abs/2606.29537>): real-OS computer-use tasks; 2.0 has 108 long-horizon workflows (~1.6 human-hours, ~318 tool calls avg) with execution-based verification scripts inspecting final OS/app state.331332Common skeleton: **instruction + isolated environment + programmatic success checker + oracle solution**. Metrics: task resolution rate (primary), plus steps/tokens/time-to-completion (efficiency).333334### 7.2 Beyond pass/fail: judges and trajectory inspection335336From Anthropic's multi-agent research system (<https://www.anthropic.com/engineering/multi-agent-research-system>):337338- **End-state evaluation** for tasks with mutable state: judge whether the final state is correct, not whether the agent followed an expected path — agents legitimately find alternate valid routes.339- **LLM-as-judge** with a rubric (accuracy, completeness, source/tool quality, efficiency) scales grading of free-form outcomes; single-call judges with 0–1 scores worked best.340- **Human trajectory inspection remains essential**: humans caught failure modes rubrics missed (e.g. preferring SEO content farms over authoritative sources). Reading transcripts of *how* the agent worked — wrong tool choices, ignored errors, loops — is the debugging method.341- **Start small**: ~20 representative tasks catch most regressions early; don't wait for a big eval harness.342- Also evaluate **safety properties as test cases**: "destructive command always prompts in every mode," "cancel actually kills the process," "no sudo runs silently" — assert them like unit tests (this mirrors Phase 7's mandate).343344---345346## 8. Implications for Zyquo Agent Architecture347348| Finding (section) | Component | Design consequence |349|---|---|---|350| While-loop on stop_reason; done = non-tool response (§1.3) | `AgentLoop` | Actor loop: send → stream → if toolCalls: execute via gate, append results, repeat; exit on `end_turn`/`stop`; handle `max_tokens`/`refusal`/`pause` explicitly. |351| Provider stop/threading formats differ (§1.3, §2.2) | `ProviderClient` protocol |352| System prompt: altitude, done-convention, verify-before-done (§1.4) | `AgentLoop` system prompt | Sectioned prompt (role, environment, tool policy, planning discipline, safety, done-convention); compaction-immune; instructs the model to verify with tools before declaring done. | Normalize to `AssistantTurn {text, thinking, [ToolCall], stopReason}` + `ToolResultMessage`; each client (OpenAI-compatible, Anthropic, Gemini) maps to its wire format; results for parallel calls returned in one message, correlated by id. |353| Streaming deltas incl. partial tool JSON (§1.6) | `ProviderClient` → UI | Unified event stream (`textDelta`, `toolCallArgsDelta`, …); accumulate partial JSON per call id; UI renders command text as it streams; never execute until arguments finalize. |354| Trained-in tool shapes; few consolidated tools (§2.4–2.5) | `ToolRegistry`, `Tools/` | Small tool set: `bash` (persistent session semantics, `{command, timeout?}`), `osascript`, `read_file`/`write_file`/`edit_file` (exact-unique `str_replace` contract)/`list`/`search`. Schemas kept to the JSON-Schema/OpenAPI common subset; descriptions written as onboarding docs; 25K-token result truncation with steering messages. |355| Exact-match str_replace + read-before-edit (§2.5, §4.2) | `FileTools`, `WorkspaceManager` | `edit_file` fails loudly on zero/multiple matches; require the file to have been read this task before editing; track every created/modified file. |356| Compaction: keep plan+recent verbatim, summarize old, re-inject standing context (§3.2–3.3) | `MemoryManager` | Live token accounting; compact at ~85% via a summarization call; plan, system prompt, MEMORY.md never summarized away and re-injected post-compaction; thrashing guard. |357| Offload big outputs to files (§3.3) | `MemoryManager` + `WorkspaceManager` | Tool outputs > threshold written to `workspace/.zyquo/outputs/…`, replaced in-context by path + auto-summary; searchable later via FileTools. |358| Memory files / memory tool (§3.4) | `MemoryManager` | Agent-maintained `MEMORY.md` in each workspace (read at task start, updated as facts are learned); persists across compactions and sessions. |359| Workspace = task folder; sandbox where possible (§4) | `WorkspaceManager`, `ExecutionService` | One dir per task under `Workspaces/`; bash cwd pinned there; FileTools scoped there by default, escape = explicit permission; consider optional Seatbelt profile for autonomous mode later. |360| Checkpoints at step boundaries, code/conversation restore (§4.2) | `WorkspaceManager` | Snapshot agent-touched files per user prompt / per step; restore-files, restore-transcript, or both. |361| Externalized todo list with pending/in_progress/completed (§5.1) | `Planner` | Plan drafted at task start, persisted, rendered in the Plan panel, editable by user; the loop updates item states as steps complete; plan survives compaction by construction. |362| Re-planning is normal; Reflexion on failure (§1.2, §5.2) | `Planner` + `AgentLoop` | After a step fails twice: forced self-critique turn, then plan revision; plan changes surfaced in UI, never silent. |363| Single continuous agent; sub-agents read-only only (§5.3) | `AgentLoop` | v1: one loop, no concurrent mutating sub-agents (Cognition's context-sharing argument); optional later: read-only explorer sub-agent returning summaries. |364| Max steps, token/time budgets, repetition & stall detection, pause-not-abort (§6.1) | `LoopGuard` | Configurable caps (Settings › Agent); repetition = same tool+normalized args failing; stall = N iterations without plan/file/info change; on trip → pause task, Awaiting-input state, ask user. |365| Deny→ask→allow, parsed subcommands, read-only auto-allow, always-ask circuit breakers (§6.3) | `PolicyEngine` | Rule engine with that exact precedence; shell parser splits `&&`/`;`/`|` and strips wrapper commands before matching; curated read-only allowset; destructive class (`rm -rf` scale, `sudo`, `curl|sh`, system paths, disk ops, `launchctl`…) always asks in **all three modes** incl. Autonomous; "Approve & remember" saves narrowest per-subcommand rules. |366| Patterns are UX, not security (§6.3) | `PolicyEngine` + `AuditLog` | Every executed action (approved or auto) appended to the audit log with timestamp, cwd, exit code, truncated output; approval cards show exact command + explanation + risk label. |367| Modes mirror industry (§6.3) | `PolicyEngine` | Manual ≈ Claude Code `default`/Codex suggest; Guarded ≈ `acceptEdits`+classifier (auto read-only/safe, ask mutating); Autonomous ≈ bounded full-auto — still budgeted, audited, and circuit-breakered. |368| Eval = instruction + env + programmatic checker; trajectory reading; safety as tests (§7) | Phase 7 harness | ≥8 scenario tasks in temp workspaces with scripted success checkers; per-model tool-call conformance table; safety assertions (always-prompt, cancel-kills, no-silent-sudo) as automated tests; keep transcripts for inspection. |369370**The one-sentence architecture:** Zyquo Agent is a single Anthropic-style tool-use while-loop (`AgentLoop`) whose every action passes a Claude-Code-style deny→ask→allow gate (`PolicyEngine`) into a per-task folder (`WorkspaceManager`), kept honest by an externalized todo plan (`Planner`), kept alive by compaction + memory files + output offloading (`MemoryManager`), and kept bounded by step/token/time/repetition guards that pause rather than abort (`LoopGuard`) — with everything streamed to the UI and appended to an audit log.371372---373374## Source Index375376**Agentic loop & patterns**377- Anthropic, *Building Effective Agents* — <https://www.anthropic.com/engineering/building-effective-agents>378- Anthropic docs, *How tool use works* (agentic loop, stop reasons, server-side loop) — <https://platform.claude.com/docs/en/agents-and-tools/tool-use/how-tool-use-works>379- Anthropic docs, *Stop reasons and fallback* — <https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons>380- Yao et al., *ReAct: Synergizing Reasoning and Acting in Language Models* — <https://arxiv.org/abs/2210.03629>381- Shinn et al., *Reflexion: Language Agents with Verbal Reinforcement Learning* — <https://arxiv.org/abs/2303.11366>382- LangChain, *Plan-and-Execute agents* — <https://blog.langchain.com/planning-agents/>; tutorial: <https://langchain-opentutorial.gitbook.io/langchain-opentutorial/17-langgraph/03-use-cases/05-langgraph-plan-and-execute>383384**Tool design & provider APIs**385- Anthropic, *Writing effective tools for agents* — <https://www.anthropic.com/engineering/writing-tools-for-agents>386- OpenAI, *Function calling guide* — <https://developers.openai.com/api/docs/guides/function-calling>387- Google, *Gemini function calling* — <https://ai.google.dev/gemini-api/docs/function-calling>388- Anthropic docs, *Bash tool* — <https://platform.claude.com/docs/en/agents-and-tools/tool-use/bash-tool>389- Anthropic docs, *Text editor tool* — <https://platform.claude.com/docs/en/agents-and-tools/tool-use/text-editor-tool>390- Anthropic docs, *Streaming* / *Fine-grained tool streaming* — <https://platform.claude.com/docs/en/build-with-claude/streaming>, <https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/fine-grained-tool-streaming>391- Aider, *Edit formats* — <https://aider.chat/docs/more/edit-formats.html>392393**Memory & context**394- Anthropic, *Effective context engineering for AI agents* — <https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents>395- Anthropic, *Managing context on the Claude Developer Platform* (context editing + memory tool, 84%/39% results) — <https://www.anthropic.com/news/context-management>396- Anthropic docs, *Memory tool* — <https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool>397- Claude Code auto-compact analyses — <https://claudelog.com/faqs/what-is-claude-code-auto-compact/>, <https://okhlopkov.com/claude-code-compaction-explained/>, <https://howaiworks.ai/blog/claude-code-auto-compact-context-management>398399**Workspaces, planning, agents in production**400- OpenHands platform paper — <https://arxiv.org/abs/2407.16741>; OpenHands Agent SDK — <https://arxiv.org/html/2511.03690v1>401- Claude Code docs: *Checkpointing* — <https://code.claude.com/docs/en/checkpointing>; *Sub-agents* — <https://code.claude.com/docs/en/sub-agents>; *Todo/Task tracking* — <https://code.claude.com/docs/en/agent-sdk/todo-tracking>; *Best practices* — <https://code.claude.com/docs/en/best-practices>402- Anthropic, *How we built our multi-agent research system* — <https://www.anthropic.com/engineering/multi-agent-research-system>403- Cognition, *Don't Build Multi-Agents* — <https://cognition.ai/blog/dont-build-multi-agents>; *How Cognition uses Devin to build Devin* — <https://cognition.com/blog/how-cognition-uses-devin-to-build-devin>404405**Safety & control**406- Claude Code docs, *Configure permissions* (rules, modes, read-only set, circuit breakers) — <https://code.claude.com/docs/en/permissions>407- Codex CLI sandbox/approvals — <https://inventivehq.com/knowledge-base/openai/how-to-configure-sandbox-modes>, <https://www.vincentschmalbach.com/how-codex-cli-flags-actually-work-full-auto-sandbox-and-bypass/>, <https://agent-safehouse.dev/docs/agent-investigations/codex>408- Cursor, *Agent Security* — <https://cursor.com/docs/agent/security>; Backslash Security, *The Denylist Delusion* — <https://www.backslash.security/blog/cursor-ai-security-flaw-autorun-denylist>; The Register coverage — <https://www.theregister.com/2025/07/21/cursor_ai_safeguards_easily_bypassed/>409410**Evaluation**411- OpenAI, *Introducing SWE-bench Verified* — <https://openai.com/index/introducing-swe-bench-verified/>; SWE-bench — <https://www.swebench.com>412- *Terminal-Bench* — <https://arxiv.org/abs/2601.11868>, <https://www.tbench.ai>413- *OSWorld* — <https://os-world.github.io>; *OSWorld 2.0* — <https://arxiv.org/abs/2606.29537>414