SPB Git

Repository Public

README.md 13.5 KB

# @aos/server

Fastify server of Agent OS (agent.spboucher.ai): authentication, REST + events WebSocket under /api/v1, static console, Postgres. Adapted from the Claude Remote server (auth, sessions, CSRF, throttling, CSP, audit and janitor are unchanged in behaviour).

The server never runs agent code: it reads the kernel's tables with plain SQL and delegates every write the kernel owns to @aos/kernel through a narrow port. It talks to the worker only through Postgres. See docs/CONTRACT.md §1, §2, §8 (binding).

# Layout

Path Role
src/app.ts buildServer(config, {kernel, providers}): the whole server as a factory
src/main.ts entry point: config → migrate (kernel) → adapters → listen
src/kernel-port.ts KernelPort: the six kernel functions the server calls
src/kernel-adapter.ts the only file touching @aos/kernel (typed imports)
src/providers-port.ts / providers-adapter.ts same pattern for @aos/providers (catalogue only)
src/dto-auth.ts auth / notification / audit DTOs + branding (the file the web app imports)
src/dto.ts server-local DTOs: SystemStatus, UpdateProject, ArtifactDto, ModelInfoEx, SteeringMessage
src/events-hub.ts kernel event feed → WebSockets (replay, hand-over, slow consumers, notifications)
src/paths.ts path safety under AOS_WORKSPACE_ROOT
src/security/*, src/modules/auth password, TOTP, WebAuthn, sessions, CSRF/Origin guard, throttle, setup token
src/modules/* projects, models, missions (+ runs, steps, artifacts), waits, policy, spend, system, events-ws, notifications, settings, openapi
src/browser-view.ts strips the provider-native transcript from what the browser receives
src/env-file.ts loader of the file named by AOS_ENV_FILE
src/testkit/* harness, fake-kernel.ts, software passkey

# Environment

Variable Default Notes
NODE_ENV development production enforces the three requirements marked *
AOS_PORT / AOS_HOST 8640 / 127.0.0.1
AOS_PUBLIC_ORIGIN * http://localhost:8640 exact origin; https in production. Cookie scope, Origin check, WebAuthn RP ID (= hostname)
DATABASE_URL * postgres://localhost/agent_os_dev
AOS_SECRET_KEY * dev: generated into $AOS_DATA_DIR/dev-secret-key 64 hex chars; AES-256-GCM key for TOTP secrets
AOS_DATA_DIR ~/agent-os-data holds setup-token (0600) until the administrator exists; artifacts under artifacts/ (kernel)
AOS_WORKSPACE_ROOT ~/workspace projects live in <root>/<rel_path>
AOS_TRUSTED_PROXIES 127.0.0.1,::1,10.67.0.1 IPs/CIDRs whose X-Forwarded-For is honoured (tunnel gateway)
AOS_WEB_DIR ../web/out static export; missing ⇒ API only
AOS_LOG_LEVEL info pino JSON
AOS_ENV_FILE – optional KEY=VALUE file loaded at start; never overrides the environment
ANTHROPIC_API_KEY, OPENAI_API_KEY – the server only records their presence (booleans) and masks their values in everything it sends or logs; it never calls a model

# API (/api/v1)

Every route needs a full session (password + second factor) except the six public / pre-auth ones (GET /health, GET /auth/status, POST /auth/setup, POST /auth/login, passkey login options / verify). Mutations need X-CSRF-Token (from GET /auth/status) and Origin = AOS_PUBLIC_ORIGIN. Errors are {error:{code,message,details?}}. Creations answer 201.

Area Endpoints
health GET /health → {ok, version, agent:'aos'} · GET /system/status → SystemStatus (dto.ts) · GET /system/audit
auth unchanged from Claude Remote; GET /auth/status adds branding:{agent:'aos', productName:'Agent OS', agentName:'Agent'}
projects GET/POST /projects · GET/PATCH/DELETE /projects/:id
models GET /models → ModelInfo[] (+ optional stale, note)
missions GET /missions?status=&projectId=&limit= · POST /missions · GET /missions/:id · POST /missions/:id/cancel · GET /missions/:id/runs
runs GET /runs/:id · GET /runs/:id/steps?after=&limit= · POST /runs/:id/message {text} · GET /runs/:id/artifacts · GET /artifacts/:sha256?name=
waits GET /waits?status=&kind=&missionId= · POST /waits/:id/resolve
policy GET/POST /policy/rules · DELETE /policy/rules/:id
spend GET /spend → SpendSummary
misc GET/PUT /settings (AosSettings) · GET /notifications · POST /notifications/read · GET /openapi.json
events WS /events?cursor=<lastEventId>&missionId=<optional>

Behaviour worth knowing:

  • POST /missions — the project must belong to the caller and its directory must exist (409 workspace_missing); provider / model / effort / budget default to the owner's AosSettings; the model must be in the provider catalogue (400 unknown_model, never a silent substitute) and the provider key configured (409 provider_not_configured); refused with 409 daily_cap_reached when spend_days for CURRENT_DATE ≥ dailyBudgetUsd. workspace_abs = realpath(AOS_WORKSPACE_ROOT/rel_path). The kernel builds the root DelegationContract; done criteria may be empty and are passed through as an empty list (contract.doneCriteria = [], missions.done_criteria = []) — that is the signal for the engine to propose criteria first. No placeholder criterion is used.
  • Steps — request, result and error are each bounded to 20 000 characters. The JSON structure is kept (long strings are cut with a …[truncated N chars] marker, surplus array items / object keys are dropped with a marker) and the step carries truncated: true. idem_key, request_hash, leases, fencing tokens and workspace_abs are never sent.
  • Artifacts — reachable only through a run of a mission the caller owns. Always Content-Type: application/octet-stream, Content-Disposition: attachment, nosniff, Content-Security-Policy: default-src 'none'; sandbox: nothing renders inline.
  • Waits — default listing: pending approvals and questions, oldest first (kind=all includes timers / child joins). Resolving goes through the kernel; a wait that is no longer pending is 409 wait_not_pending. remember: true is passed in the resolution (approve only): the engine turns it into an allow rule; the server inserts no rule.
  • Projects — create = mkdir + git init; existing = register a directory; clone = git clone (whitelist https://, ssh://, git@host:path, gitsrv:path; argv array, no shell; minimal environment with GIT_TERMINAL_PROMPT=0 and no API key; 10 min timeout; the partial directory is removed on failure). The clone runs inside the request. Paths are clean relative paths under AOS_WORKSPACE_ROOT; the realpath of every existing component must stay inside the root (symlink escapes ⇒ 400 path_outside_root); .git / .aos segments are refused. Credentials embedded in a remote URL are stripped before it is stored or shown. DELETE unregisters only (refused with 409 project_busy while a mission is live).
  • Models — live listing per provider when its key is configured, cached 10 min (1 min after a failure), 10 s timeout. A provider without a key, or whose listing fails, is answered from the static catalogue with available:false, stale:true, note.
  • System status — db {ok, latencyMs}, worker {alive, lastSeenAt, source, workers[]} (table workers first, a row < 60 s old; otherwise the lease heartbeat of a running run; alive: null when nothing can tell), providers {anthropic, openai} as booleans, counts.
  • Spend — byModel covers the last 7 days, from the usage of model_call steps.

# Events WebSocket

Same authentication as every other route (cookie + second factor, Origin checked at upgrade; query validated before the upgrade). Server → browser only, one JSON AosEvent per text frame.

  1. With cursor, persisted events with id > cursor are replayed from Postgres (owner-scoped, optionally one mission), at most 5 000. If more are missing, the first 5 000 are followed by system.notice {code:'gap', message:'gap, refetch', latestId}. Without cursor: live only.
  2. Live events from the kernel's listenEvents are buffered during the replay and flushed after it; ids are monotonic per socket, so the hand-over neither loses nor repeats an event.
  3. Transient model.delta events are forwarded with id: -1 and never stored.
  4. A consumer with more than 1 MiB buffered is dropped (close 4429, then terminated).
  5. When the auth session ends (logout, revocation, expiry — re-checked every 60 s) the socket receives system.notice {code:'auth.expired'} and is closed with 4401.

Notices use id: -1 and missionId: ''. wait.created (approval / question) and a terminal mission.status also create an in-app notification (title only; mission title when notifications.detail = 'titles').

# Security model in one page

  • Route auth levels (config.auth, default full): public (GET /health only), preauth (auth/status, setup, login, passkey login), restricted (valid session, second factor pending: status, logout, TOTP step, enrolment), full. One global onRequest guard (src/security/guard.ts) enforces them, also for WebSocket upgrades. http.test.ts iterates the live route table: every route answers 401 without a session unless it is on the documented list, and every mutating route refuses a missing CSRF token or a foreign Origin.
  • Sessions: random 32-byte cookie token (__Host-aos_sid; aos_sid only outside production on http://localhost), sha256 stored, idle 12 h, absolute 30 d, 15 min before the second factor, rotated on upgrade. Pre-auth passkey challenge cookie: __Host-aos_pre.
  • Request rate limits (per minute, config.rateLimit): credential checks share a strict per-IP bucket (20); anonymous GET /auth/status per IP (240); GET /health per IP (600); everything else per session cookie (600).
  • Login throttling in login_attempts: user 5, IP 10, global 100 failures / 15 min; lock 60 s doubling per extra failure (cap 15 min).
  • CSP on every response; script-src never contains unsafe-inline (per-document sha256 hashes of the static export's inline scripts).
  • Secrets: the model API keys are read by @aos/providers / the worker, not used here. The server keeps only booleans, and a scrubber replaces any occurrence of the key values with [redacted] in every API response body and header, every WebSocket frame and every log line (step bodies are scrubbed before truncation). Artifact content is streamed as stored. leak.test.ts pollutes the journal with the keys and greps every GET route of the route table.
  • Owner scoping: every read joins on owner_id; another owner's rows are 404.
  • Audit on every mutation; never objectives, answers, steering text or URLs with credentials.

# Kernel and providers integration

KernelPort (createMission, requestCancel, resolveWait, postOwnerMessage, listenEvents, openArtifact) is implemented by kernel-adapter.ts with ordinary typed imports of @aos/kernel (a workspace dependency): a signature change in the kernel is a compile error here. The adapter creates its own kernel handle (createKernel({databaseUrl, dataDir})) and passes it to every call. Migrations belong to the kernel: main.ts calls its migrate(databaseUrl) at startup, and the test harness uses the same function. The server inserts no event itself (notices sent to a socket are never persisted). providers-adapter.ts does the same for @aos/providers (catalogue and live model listing only).

src/kernel-adapter.test.ts runs in the default suite: the adapters against the real kernel and providers (vitest resolves both packages to their sources, vitest.config.ts; no model call): migrate, create mission, live events, steering, wait resolution, artifact download, cancel.

browser-view.ts removes the provider-native transcript (native) from model_call step results and from step.completed events before anything is sent to the browser.

main.ts first loads the file named by AOS_ENV_FILE, if set (env-file.ts: KEY=VALUE lines, never overriding a variable already in the environment) — this is how production gets DATABASE_URL, AOS_SECRET_KEY and the presence of the provider keys without putting them in the PM2 environment.

# Run

bash
pnpm --filter @aos/shared build && pnpm --filter @aos/kernel build && pnpm --filter @aos/providers build
pnpm --filter @aos/server build
node dist/main.js            # migrates (kernel), then listens on AOS_PORT

First run: read the one-time token on the host (cat $AOS_DATA_DIR/setup-token; it is never logged), open the site, create the administrator, then enrol TOTP or a passkey. A second factor is mandatory: until one exists the session can only reach the enrolment endpoints.

# Tests

bash
pnpm --filter @aos/server test     # needs local Postgres with createdb/dropdb on PATH, and git

Each test file creates its own database aos_test_<random>, applies db/migrations and drops it afterwards (also when the harness fails to start). The kernel is src/testkit/fake-kernel.ts: SQL-backed, it writes the rows the real kernel writes for the six port functions and plays the worker through seed* / emit* fixtures. Migrations are applied by the kernel's migrate. The model catalogue in tests is a labelled fixture (FIXTURE_CATALOGUE). src/testkit/soft-authenticator.ts is a software passkey (ES256, real signatures).

Files main
Menu

Theme