| A1 |
Unknown public API → provisional AKP with real evidence + recorded execution |
PASS |
Frankfurter probe (akp frankfurter.frankfurter-exchange-rates v1→v3, inferred_schema debt open), World Bank probe, Open-Meteo OpenAPI (openapi/forecast.yml, doc_fetch + openapi_parse evidence). Discovery executions recorded in project akos-discovery; verification: contract_test PASS (sandboxed pytest), live_call PASS, schema_validation PASS; lifecycle tested. Visible in UI → API Atlas. |
| A2 |
Generated Python client performs a real request; schema and errors tested |
PASS |
test_cross_project_reuse_without_rediscovery runs the downloaded bundle's client.py in direct transport against the mock (200 1.15…, validate()==[]) and its generated tests/ (error, unknown op, gateway-token, credential tests). Live: verification live calls through gateway for 3 real providers (HTTP 200, hashes recorded). |
| A3 |
Knowledge persists after restart / reboot / fresh Claude session |
PASS (restart, fresh session) / NOT RUN (reboot) |
pm2 restart akos-web akos-worker akos-proxy: counts identical before/after (apis 4, packages 9, executions 52, datasets 4, audit 167), AKP v3 tested with 6 evidence rows. Fresh Claude Code project (/tmp/akos-fresh-project, claude -p … --allowedTools mcp__akos__*): reused AKP v3, client constants read, governed execution ba7cd910… succeeded, no discover_api call. Node reboot not performed: M4M64c also hosts chat.spboucher.ai and note.spboucher.ai; PM2 startup + mld heal cover boot (verified for the other apps on 2026-09-10 mass reboot). |
| A4 |
Second clean project reuses the AKP without rediscovery |
PASS |
test_cross_project_reuse_without_rediscovery (project-bound token B, API count unchanged) + fresh Claude session above. |
| A5 |
Opaque MCP labelled; no fabricated internals |
PASS |
test_opaque_mcp_labelled_and_blocked, demo_opaque.py: kind=mcp, opaque_internal_implementation=true, critical opaque_mcp debt, invocations observed=false, verified export BLOCKED with reasons (observed, akp_tested, license, not_opaque); Open-Meteo AKP named as a tested alternative, explicitly not proof. |
| A6 |
Controlled-network bypass blocked; uncovered routes disclosed |
PASS (node) |
tests/security/test_enforcement.py: inside sandbox-exec curl/urllib/subprocess/bash direct egress blocked; proxy returns 403 without grant, 200 with grant (api.frankfurter.dev), other hosts still blocked; writes outside workdir denied; egress_log has allow+block rows. Uncovered: host shell outside sandbox, Claude Code Bash, remote MCP servers (docs/threat-model.md). |
| A7 |
Secret rotation/revocation; no leakage |
PASS |
test_credentials_scoped_rotation_revocation_no_leak: scoped fake secret works (200), missing → 401, wrong project → 403, rotate → v2 used, revoke → policy deny; fake value absent from 7 tables, API responses, logs (caplog), repo grep, export manifest. |
| A8 |
Research export reruns from stored inputs; live revision limits reported |
PASS |
Export zip replayed in a clean directory with python3 -I transform.py --offline → REPRODUCED (hash equal), tests/test_replay.py 1 passed; acquire.py re-runs live via AKOS and reports identical/REVISED per execution; METHODS.md distinguishes computational / retrieval / upstream reproducibility. |
| A9 |
Cross-source join validated; incompatible unit/granularity refused |
PASS |
demo_research.py: World Bank GDP (country-year, USD) × Frankfurter daily USD/CAD aggregated to yearly mean → 1:1, 9 matched keys, 0/1 unmatched, units gdp_cad: CAD, output 9 rows verified, 16-node lineage; refusals: granularity mismatch (no aggregate), unit mismatch (USD + CAD/USD). Unit tests test_join_refuses_*. |
| A10 |
Drift: schema change detected, dependents flagged, debt, versioned fix revalidated |
PASS |
test_drift_detection_versioning_revalidation: mock schema change → alert + stale_contract debt + affected project id + API/AKP/version stale; re-verification → tested, alert resolved, debt resolved. |
| A11 |
Worker crash recovery; DB backup/restore |
PASS |
test_worker_crash_recovery_and_audit_intact (stale lock requeued, attempts=2, dead-letter + retry); test_backup_and_restore_scripts (node): pg_dump + artifacts tgz + sha256, restore into akos_restore_test with row counts. Production backups exist in ~/backups/akos/ (e.g. akos-20261101-195741.dump, 205 KB). |
| A12 |
Auth, tenant separation, SSRF, prompt injection, rate limits, audit logging |
PASS |
test_auth_tenant.py, test_ssrf_injection_limits.py, audit chain/trigger tests; concurrency fix verified by test_audit_concurrent_appends_stay_linear. Known limitation: the production audit log contains historical forks (first at seq 31) produced before the fix by parallel benchmark jobs; all 167 content hashes verify, rows are append-only and were not rewritten; GET /audit/verify reports content_ok=true, linear=false with the fork list. |
| A13 |
Public DNS, TLS, UI, REST, MCP from outside the host |
PASS |
scripts/acceptance_external.sh 13/13 from the laptop; claude mcp list → akos … ✔ Connected; CLI akos login/status/doctor/search/inspect/execute against the public URL. |
| A14 |
Mobile UI workflows at iPhone width, no placeholders |
see note |
Playwright iPhone-14 suite against the public URL: first run 3/4 passed (the 4-page navigation test exceeded the 60 s default over the WAN); timeout raised to 180 s and rerun — result recorded in docs/build-log.md (final line of the 2026-11-01 entry). No placeholder features: every page reads /api/v1. |
| A15 |
Benchmarks (measured) |
PASS |
scripts/bench.py --n 30 --jobs 10 from the laptop via the tunnel: search p50 24 ms / p95 64 ms; AKP inspect p50 39 / p95 54 ms; overview API p50 34 / p95 37 ms; UI index p50 70 / p95 441 ms; health p50 76 / p95 179 ms (WAN); job throughput 10 drift checks (live provider call + schema validation each) in 8.0 s = 75 jobs/min with worker concurrency 3. |