SPB Git

spb/akos

Public
9.9 KB

# AKOS architecture

AKOS (API Knowledge Operating System) is one Python service + one worker + one egress proxy + one PostgreSQL database, deployed on a MacLustr node (M4M64c) and exposed through the MacLustr Tunnel (BHS64 Caddy → WireGuard → node). It is deliberately a modular monolith: the six subsystems of the specification are Python packages inside akos/, sharing one schema and one job queue.

text
 Internet ──TLS──▶ Caddy (BHS64) ──wg0/wg1──▶ 10.67.0.33:8520  akos-web   (FastAPI: UI, /api/v1, /mcp, /docs)
                                                         127.0.0.1:8521  akos-proxy (governed egress, CONNECT/forward proxy, loopback only)
                                                                         akos-worker (durable jobs: discovery, compile, verify, export, drift, backup)
                                                         127.0.0.1:5432  PostgreSQL 17 (database akos)  + artifact store ~/apps/akos/data/artifacts/<sha256>
                                                         macOS keychain (service "akos")  = credential vault, referenced as vault://keychain/akos/<name>

# Subsystems → packages

Spec subsystem Package Notes
Knowledge Enforcement Gateway (KEG) akos/gateway/ (executor.py, policy.py, egress_proxy.py, sandbox.py) Every mediated call: intent row persisted → policy decision → SSRF check → HTTP call (httpx) with credential injected from keychain → response hashed, snapshot stored (if license permits) → outcome row. Sandboxed worker runs under sandbox-exec with network limited to loopback; the only way out is the egress proxy which allows only hosts of a live execution grant.
API Knowledge Registry (AKR) akos/registry/ (models.py, lifecycle.py, search.py, graph.py) PostgreSQL, versioned rows, no in-place overwrite of a published version; FTS (tsvector + pg_trgm).
Archaeologist & Compiler akos/compiler/ (archaeologist.py, openapi.py, probe.py, akp.py, codegen_python.py, codegen_ts.py, codegen_mcp.py, docs.py) Deterministic code, no LLM required. Evidence rows for every fetched document (URL, hash, time).
Credential Genome Vault akos/vault/ (keychain.py, refs.py, redact.py) security CLI; values never persisted in DB; redaction filter on logs and API responses.
Executable Provenance Engine akos/provenance/ (manifest.py, lineage.py, export.py, quality.py) Immutable manifests, lineage DAG, akos research export.
Knowledge Composition & Debt akos/composition/ (recipes.py, joins.py), akos/debt.py, akos/drift.py Join validation (cardinality, units, granularity), debt records, drift checks.
Jobs akos/jobs/ (queue.py, worker.py, handlers/) Postgres outbox table, FOR UPDATE SKIP LOCKED, heartbeats, retries with backoff, idempotency keys, dead-letter.
Auth / audit akos/auth.py, akos/audit.py argon2id passwords, signed cookie sessions, hashed bearer tokens with scopes + project binding, hash-chained audit events.
Interfaces akos/api/ (REST routers), akos/mcp_server.py (Streamable HTTP JSON-RPC), akos/cli.py (akos …), akos/sdk.py (Python SDK)
UI web/ (Vite + React 19 + TypeScript, served as static files by FastAPI) mobile-first, dark mode, all data from /api/v1.

# Data model (PostgreSQL, migrations in akos/migrations/)

Core tables (all have id uuid, created_at timestamptz):

  • users, api_tokens (sha256 of token, scopes, optional project_id binding, revoked_at), projects (slug, policy_mode ∈ discovery/development/research/production), agents (kind, project, budget, permissions).
  • providers, apis (kind ∈ rest/graphql/sdk/mcp, opaque_internal_implementation bool, lifecycle, knowledge_state), api_versions (immutable contract jsonb + contract_hash), endpoints, variables.
  • credential_refs (ref = vault://keychain/akos/<name>, scheme, scopes, allowed_projects, version, rotated_at, revoked_at; no value column).
  • knowledge_packages (AKP: akp_id stable + version, manifest jsonb, contract_yaml, recipe jsonb, readme, license jsonb, hash, lifecycle, knowledge_state, superseded_by), client_artifacts (language, artifact sha, static-scan result), verification_evidence (kind, result, command, output hash, artifact).
  • executions (intent persisted first: project, agent, api/akp/endpoint, sanitized params, purpose, fingerprint; then outcome: state, status code, response hash, artifact, duration, error, retries; observed bool), policy_decisions, egress_grants (execution → host:port allowlist consumed by the proxy).
  • datasets, transformations, lineage_edges, recipes, knowledge_debt, drift_alerts, artifacts (content-addressed), jobs, worker_heartbeats, audit_events (prev_hash/hash chain).

Lifecycle (akos/registry/lifecycle.py, enforced in code): discovered → inspected → modeled → implemented → tested → registered → monitored, side states quarantined, stale, deprecated, revoked. Knowledge state (observed, documented, implemented, tested, independently_reproduced, deprecated, unknown) is derived from evidence rows only (akos/registry/lifecycle.py:knowledge_state_from_evidence); the API refuses to set it directly.

# Policy modes (gateway)

Mode Requirement to execute Outcome
discovery URL passes SSRF/allow checks; project budget execution recorded as provisional, knowledge-debt undocumented_endpoint created if no AKP
development endpoint belongs to an API version with lifecycle ≥ inspected normal execution
research AKP lifecycle ≥ tested, license recorded, credential scoped, lineage enabled; fails closed if intent cannot be persisted execution + dataset eligibility
production AKP lifecycle = registered/monitored, approved version pinned in project, credential scoped, rate limit monitored execution

# Enforcement boundary (honest statement)

  • Covered: calls made through POST /api/v1/execute, the MCP tool execute_api, the CLI, the Python SDK, and any code run by akos-worker inside sandbox-exec (network restricted to loopback → only the egress proxy, which only allows hosts granted to a live execution).
  • Not covered: an unmanaged shell on the node or laptop, Claude Code Bash tool calls outside the sandbox, remote MCP servers (Apify, Tavily…) that make their own HTTP calls. These are recorded as unobserved when declared and are excluded from verified exports. Claude Code hooks can only nudge (SessionStart context, PreToolUse warning on raw curl/requests usage); they cannot intercept arbitrary networking. See docs/threat-model.md.

Auth: POST /auth/login {email,password} → Set-Cookie: akos_session (HttpOnly, Secure, SameSite=Lax) · POST /auth/logout · GET /auth/me · GET/POST /auth/tokens ({name, scopes[], project_id?} → token shown once) · DELETE /auth/tokens/{id} · POST /auth/password.

Read: GET /overview · GET /search?q= · GET /providers[/{id}] · GET /apis?q&kind&lifecycle · GET /apis/{id} (versions, endpoints, akps, evidence, debt) · GET /versions/{id} · GET /endpoints/{id} · GET /akps[/{id}] · GET /akps/{id}/files/{name} · GET /akps/{id}/bundle.zip · GET /executions?project_id&state&limit · GET /executions/{id} · GET /credentials[/{id}] (metadata only) · GET /credentials/{id}/audit · GET /projects[/{id}] · GET /agents · GET /policy/decisions · GET /datasets[/{id}] · GET /datasets/{id}/lineage · GET /datasets/{id}/quality · GET /recipes[/{id}] · GET /debt · GET /drift · GET /audit?limit&target_type&target_id · GET /audit/verify · GET /jobs[/{id}] · GET /graph?focus=<type>:<id>&depth=2 · GET /ops/health · GET /ops/metrics · GET /ops/backups · GET /health (public: {status, version}).

Write: POST /providers · POST /apis · POST /apis/{id}/lifecycle {to, reason} · POST /discover {source_url?, openapi_url?, probe_url?, name?, provider?, project_id} → {job_id} · POST /akps/{id}/compile · POST /akps/{id}/verify · POST /execute {...} → execution · POST /credentials {name, provider_id?, scheme, placement, value, scopes[], allowed_projects[], environment, expires_at?} · POST /credentials/{id}/rotate {value} · POST /credentials/{id}/revoke · POST /projects · PATCH /projects/{id} {policy_mode, approved_akps} · POST /agents · POST /datasets {name, execution_ids[], project_id} · POST /datasets/{id}/export {verified} → {job_id} · POST /recipes/compose {name, project_id, left, right, join} → validation report · POST /recipes/{id}/run · PATCH /debt/{id} · POST /drift/check {api_version_id} · POST /jobs/{id}/retry · POST /ops/backup.

Errors: {"error": {"code": "...", "message": "...", "correlation_id": "..."}}; 401 unauthenticated, 403 forbidden/policy denied, 404, 409 conflict (lifecycle/idempotency), 422 validation, 429 rate limit.

# MCP (POST /mcp, Streamable HTTP, JSON responses, Bearer token)

Tools: search_knowledge, inspect_api, discover_api, execute_api, generate_client, verify_package, list_knowledge_debt, compose_recipe, export_research_dataset, get_execution, plus aliases akos.search/inspect/register/acquire/compile/validate/reproduce/lineage/debt/status. Every tool returns IDs, versions, evidence and status — never raw datasets.

# Listener and proxy trust

akos-web binds 127.0.0.1:8520 and the wg1 address (10.67.0.x):8520 only (two sockets, uvicorn.Server.serve(sockets=…)), never the LAN interface. X-Forwarded-* is honoured only from 127.0.0.1 and 10.67.0.1 (the Caddy hub). Cookies are Secure; HSTS is set by the app. Rate limiting is per token/IP in-process (token bucket) plus Postgres-backed budgets per project.

Files · main
Menu

Theme