1# AKOS architecture23AKOS (API Knowledge Operating System) is one Python service + one worker + one egress proxy + one PostgreSQL database, deployed on a MacLustr node (M4M64c) and exposed through the MacLustr Tunnel (BHS64 Caddy → WireGuard → node). It is deliberately a modular monolith: the six subsystems of the specification are Python packages inside `akos/`, sharing one schema and one job queue.45```6 Internet ──TLS──▶ Caddy (BHS64) ──wg0/wg1──▶ 10.67.0.33:8520 akos-web (FastAPI: UI, /api/v1, /mcp, /docs)7 127.0.0.1:8521 akos-proxy (governed egress, CONNECT/forward proxy, loopback only)8 akos-worker (durable jobs: discovery, compile, verify, export, drift, backup)9 127.0.0.1:5432 PostgreSQL 17 (database akos) + artifact store ~/apps/akos/data/artifacts/<sha256>10 macOS keychain (service "akos") = credential vault, referenced as vault://keychain/akos/<name>11```1213## Subsystems → packages1415| Spec subsystem | Package | Notes |16|---|---|---|17| Knowledge Enforcement Gateway (KEG) | `akos/gateway/` (`executor.py`, `policy.py`, `egress_proxy.py`, `sandbox.py`) | Every mediated call: intent row persisted → policy decision → SSRF check → HTTP call (httpx) with credential injected from keychain → response hashed, snapshot stored (if license permits) → outcome row. Sandboxed worker runs under `sandbox-exec` with network limited to loopback; the only way out is the egress proxy which allows only hosts of a live execution grant. |18| API Knowledge Registry (AKR) | `akos/registry/` (`models.py`, `lifecycle.py`, `search.py`, `graph.py`) | PostgreSQL, versioned rows, no in-place overwrite of a published version; FTS (`tsvector` + `pg_trgm`). |19| Archaeologist & Compiler | `akos/compiler/` (`archaeologist.py`, `openapi.py`, `probe.py`, `akp.py`, `codegen_python.py`, `codegen_ts.py`, `codegen_mcp.py`, `docs.py`) | Deterministic code, no LLM required. Evidence rows for every fetched document (URL, hash, time). |20| Credential Genome Vault | `akos/vault/` (`keychain.py`, `refs.py`, `redact.py`) | `security` CLI; values never persisted in DB; redaction filter on logs and API responses. |21| Executable Provenance Engine | `akos/provenance/` (`manifest.py`, `lineage.py`, `export.py`, `quality.py`) | Immutable manifests, lineage DAG, `akos research export`. |22| Knowledge Composition & Debt | `akos/composition/` (`recipes.py`, `joins.py`), `akos/debt.py`, `akos/drift.py` | Join validation (cardinality, units, granularity), debt records, drift checks. |23| Jobs | `akos/jobs/` (`queue.py`, `worker.py`, `handlers/`) | Postgres outbox table, `FOR UPDATE SKIP LOCKED`, heartbeats, retries with backoff, idempotency keys, dead-letter. |24| Auth / audit | `akos/auth.py`, `akos/audit.py` | argon2id passwords, signed cookie sessions, hashed bearer tokens with scopes + project binding, hash-chained audit events. |25| Interfaces | `akos/api/` (REST routers), `akos/mcp_server.py` (Streamable HTTP JSON-RPC), `akos/cli.py` (`akos …`), `akos/sdk.py` (Python SDK) | |26| UI | `web/` (Vite + React 19 + TypeScript, served as static files by FastAPI) | mobile-first, dark mode, all data from `/api/v1`. |2728## Data model (PostgreSQL, migrations in `akos/migrations/`)2930Core tables (all have `id uuid`, `created_at timestamptz`):3132- `users`, `api_tokens` (sha256 of token, scopes, optional `project_id` binding, `revoked_at`), `projects` (slug, `policy_mode` ∈ discovery/development/research/production), `agents` (kind, project, budget, permissions).33- `providers`, `apis` (kind ∈ rest/graphql/sdk/mcp, `opaque_internal_implementation bool`, `lifecycle`, `knowledge_state`), `api_versions` (immutable `contract jsonb` + `contract_hash`), `endpoints`, `variables`.34- `credential_refs` (`ref` = `vault://keychain/akos/<name>`, scheme, scopes, `allowed_projects`, `version`, `rotated_at`, `revoked_at`; **no value column**).35- `knowledge_packages` (AKP: `akp_id` stable + `version`, `manifest jsonb`, `contract_yaml`, `recipe jsonb`, `readme`, `license jsonb`, `hash`, `lifecycle`, `knowledge_state`, `superseded_by`), `client_artifacts` (language, artifact sha, static-scan result), `verification_evidence` (kind, result, command, output hash, artifact).36- `executions` (intent persisted first: project, agent, api/akp/endpoint, sanitized params, purpose, fingerprint; then outcome: state, status code, response hash, artifact, duration, error, retries; `observed bool`), `policy_decisions`, `egress_grants` (execution → host:port allowlist consumed by the proxy).37- `datasets`, `transformations`, `lineage_edges`, `recipes`, `knowledge_debt`, `drift_alerts`, `artifacts` (content-addressed), `jobs`, `worker_heartbeats`, `audit_events` (prev_hash/hash chain).3839Lifecycle (`akos/registry/lifecycle.py`, enforced in code):40`discovered → inspected → modeled → implemented → tested → registered → monitored`, side states `quarantined`, `stale`, `deprecated`, `revoked`. Knowledge state (`observed, documented, implemented, tested, independently_reproduced, deprecated, unknown`) is **derived from evidence rows only** (`akos/registry/lifecycle.py:knowledge_state_from_evidence`); the API refuses to set it directly.4142## Policy modes (gateway)4344| Mode | Requirement to execute | Outcome |45|---|---|---|46| discovery | URL passes SSRF/allow checks; project budget | execution recorded as `provisional`, knowledge-debt `undocumented_endpoint` created if no AKP |47| development | endpoint belongs to an API version with lifecycle ≥ inspected | normal execution |48| research | AKP lifecycle ≥ tested, license recorded, credential scoped, lineage enabled; **fails closed** if intent cannot be persisted | execution + dataset eligibility |49| production | AKP lifecycle = registered/monitored, approved version pinned in project, credential scoped, rate limit | monitored execution |5051## Enforcement boundary (honest statement)5253- Covered: calls made through `POST /api/v1/execute`, the MCP tool `execute_api`, the CLI, the Python SDK, and any code run by `akos-worker` inside `sandbox-exec` (network restricted to loopback → only the egress proxy, which only allows hosts granted to a live execution).54- Not covered: an unmanaged shell on the node or laptop, Claude Code `Bash` tool calls outside the sandbox, remote MCP servers (Apify, Tavily…) that make their own HTTP calls. These are recorded as `unobserved` when declared and are excluded from verified exports. Claude Code hooks can only *nudge* (SessionStart context, PreToolUse warning on raw `curl`/`requests` usage); they cannot intercept arbitrary networking. See `docs/threat-model.md`.5556## REST API contract (`/api/v1`, JSON, cookie or `Authorization: Bearer akos_…`)5758Auth: `POST /auth/login {email,password}` → `Set-Cookie: akos_session` (HttpOnly, Secure, SameSite=Lax) · `POST /auth/logout` · `GET /auth/me` · `GET/POST /auth/tokens` (`{name, scopes[], project_id?}` → token shown once) · `DELETE /auth/tokens/{id}` · `POST /auth/password`.5960Read: `GET /overview` · `GET /search?q=` · `GET /providers[/{id}]` · `GET /apis?q&kind&lifecycle` · `GET /apis/{id}` (versions, endpoints, akps, evidence, debt) · `GET /versions/{id}` · `GET /endpoints/{id}` · `GET /akps[/{id}]` · `GET /akps/{id}/files/{name}` · `GET /akps/{id}/bundle.zip` · `GET /executions?project_id&state&limit` · `GET /executions/{id}` · `GET /credentials[/{id}]` (metadata only) · `GET /credentials/{id}/audit` · `GET /projects[/{id}]` · `GET /agents` · `GET /policy/decisions` · `GET /datasets[/{id}]` · `GET /datasets/{id}/lineage` · `GET /datasets/{id}/quality` · `GET /recipes[/{id}]` · `GET /debt` · `GET /drift` · `GET /audit?limit&target_type&target_id` · `GET /audit/verify` · `GET /jobs[/{id}]` · `GET /graph?focus=<type>:<id>&depth=2` · `GET /ops/health` · `GET /ops/metrics` · `GET /ops/backups` · `GET /health` (public: `{status, version}`).6162Write: `POST /providers` · `POST /apis` · `POST /apis/{id}/lifecycle {to, reason}` · `POST /discover {source_url?, openapi_url?, probe_url?, name?, provider?, project_id}` → `{job_id}` · `POST /akps/{id}/compile` · `POST /akps/{id}/verify` · `POST /execute {...}` → execution · `POST /credentials {name, provider_id?, scheme, placement, value, scopes[], allowed_projects[], environment, expires_at?}` · `POST /credentials/{id}/rotate {value}` · `POST /credentials/{id}/revoke` · `POST /projects` · `PATCH /projects/{id} {policy_mode, approved_akps}` · `POST /agents` · `POST /datasets {name, execution_ids[], project_id}` · `POST /datasets/{id}/export {verified}` → `{job_id}` · `POST /recipes/compose {name, project_id, left, right, join}` → validation report · `POST /recipes/{id}/run` · `PATCH /debt/{id}` · `POST /drift/check {api_version_id}` · `POST /jobs/{id}/retry` · `POST /ops/backup`.6364Errors: `{"error": {"code": "...", "message": "...", "correlation_id": "..."}}`; 401 unauthenticated, 403 forbidden/policy denied, 404, 409 conflict (lifecycle/idempotency), 422 validation, 429 rate limit.6566## MCP (`POST /mcp`, Streamable HTTP, JSON responses, Bearer token)6768Tools: `search_knowledge`, `inspect_api`, `discover_api`, `execute_api`, `generate_client`, `verify_package`, `list_knowledge_debt`, `compose_recipe`, `export_research_dataset`, `get_execution`, plus aliases `akos.search/inspect/register/acquire/compile/validate/reproduce/lineage/debt/status`. Every tool returns IDs, versions, evidence and status — never raw datasets.6970## Listener and proxy trust7172`akos-web` binds **127.0.0.1:8520 and the wg1 address (10.67.0.x):8520 only** (two sockets, `uvicorn.Server.serve(sockets=…)`), never the LAN interface. `X-Forwarded-*` is honoured only from 127.0.0.1 and 10.67.0.1 (the Caddy hub). Cookies are `Secure`; HSTS is set by the app. Rate limiting is per token/IP in-process (token bucket) plus Postgres-backed budgets per project.73