Security
- Private by default. The API binds
127.0.0.1:8300; workers bind127.0.0.1:8310-8399; the console binds127.0.0.1:8301. Only the WireGuard-connected gateway (Caddy on BHS64) reaches port 8300 over the tunnel; nothing else is exposed. - Admin auth: single admin user, argon2 password hashes, signed session cookie (
itsdangerous, HttpOnly, Secure behind HTTPS, SameSite=Lax, 14 days). First user is seeded fromADMIN_EMAIL/ADMIN_PASSWORDor created once via/api/auth/setup(only when no user exists). Login rate limit 8/min/IP. Password change requires the current password. - CSRF: session-authenticated mutations require header
X-LLM-CSRF: 1and, when anOriginis present, it must match the public URL / host. - API keys:
llm_live_+ 40 random chars, stored as SHA-256; only the prefix is displayed after creation. Scopesinference/admin. Revocation is immediate. Per-key last-used and request counters. - Request validation: JSON bodies ≤ 20 MB (middleware + parser), sampling ranges checked, ≤ 8 stop sequences, ≤ 256 embedding inputs, ≤ 200 rerank documents,
n=1only. - Path safety: repository ids validated (
org/name, no..); model deletion refuses paths outside the model root; scanned paths are only read. - No shell from the API: workers are started with
subprocess.Popen(list)(no shell); user-supplied extra llama args are filtered (`|;&$`` rejected) and only settable by an admin. - Secrets:
.envis600and outside the repo;HF_TOKENonly reaches the downloader; the session secret is generated intodata/.secret(600). Keys, passwords and tokens are never logged. - Prompt privacy:
LOG_PROMPTS=falseby default; when enabled, prompts/completions are stored truncated ininference_requests. - Audit log: login/failed login/password change, key create/revoke, model load/unload/update/delete, downloads, harvest scans, settings changes, server start/stop — with actor and IP.
- Headers:
X-Content-Type-Options: nosniff,X-Frame-Options: DENY,Referrer-Policy: same-origin,Cache-Control: no-storeon API paths. Console isnoindex. - Memory safety is part of security here: a model that would exhaust unified memory is refused before any allocation; swap use raises a warning alert; workers are killed on unload so memory always returns.