Troubleshooting
| Symptom | Where to look | Likely cause / fix |
|---|---|---|
MODEL_NOT_FOUND |
GET /v1/models, Models page |
id/alias typo; model files missing (installed=false after rescan) |
MODEL_TOO_LARGE / not_recommended |
model page → Memory by context | budget 45 GB exceeded; lower context (overrides.context), use a smaller quant, or raise the budget in Settings (never above RAM − 8 GB) |
MODEL_INCOMPATIBLE (architecture) |
pip show mlx-lm, llama-server --version |
runtime too old for the architecture: upgrade mlx-lm / llama.cpp, rescan |
MODEL_LOAD_FAILED with log tail |
~/llm-api/logs/workers/worker-<id>.log |
corrupted download (re-download with force), missing tokenizer file, unsupported quant |
MODEL_LOAD_TIMEOUT |
same log + Activity Monitor | very large model on a cold SSD cache; raise LOAD_TIMEOUT_SECONDS |
WORKER_CRASHED |
worker log, dmesg-style Metal errors |
out-of-memory at the Metal level → lower context / KV bits (overrides.kv_bits=8) |
CONTEXT_TOO_LARGE |
request | prompt longer than the loaded context; reload with a larger context (model page) |
GENERATION_TIMEOUT |
Settings | GENERATION_TIMEOUT_SECONDS; check thermal throttling in System |
Swap warning / memory pressure critical |
Dashboard alert, System | model exceeds the envelope; unload, lower budget; never leave it swapping |
| Download fails 401/403 | Settings → HF token | gated repo: set HF_TOKEN, accept the license on Hugging Face |
| Download fails 429 | Downloads history | HF rate limit: retry later |
INSUFFICIENT_DISK |
System → Storage | MIN_FREE_DISK_GB reserve; delete unused models |
| Dashboard 503 "not reachable" | pm2 logs llm-api-web, port 8301 |
Next not running / build failed (pnpm build) |
Public site 502 while localhost:8300 is fine |
ssh BHS64 curl http://10.67.0.40:8300/health, pm2 ls (llm-api-wgfwd) |
WireGuard relay down: restart llm-api-wgfwd (managed firewall blocks non-Apple listeners on the wg address, see deployment.md) |
| Login loop | cookies | behind HTTPS set SECURE_COOKIES=true; ensure the proxy passes X-Forwarded-Proto |
| Model shows loaded after a crash | restart server | stale state is cleared at startup (workers.json) |
GPU shows — |
ioreg -r -c IOAccelerator |
metric unavailable on this macOS; degrades gracefully |
Logs: ~/llm-api/logs/llm-api.log (server), ~/llm-api/logs/workers/*.log (per model), pm2 logs. Events: /api/logs/events, audit /api/logs/audit.
Handy commands on the node:
bash
source ~/apps/llm-api/server/.venv/bin/activate && cd ~/llm-api
llm-api status # hardware + telemetry + model count
llm-api scan # rescan and list registry with compatibility
pgrep -fl "mlx_worker|llama-server"
curl -s localhost:8300/health