1# Troubleshooting23| Symptom | Where to look | Likely cause / fix |4|---|---|---|5| `MODEL_NOT_FOUND` | `GET /v1/models`, Models page | id/alias typo; model files missing (`installed=false` after rescan) |6| `MODEL_TOO_LARGE` / `not_recommended` | model page → *Memory by context* | budget 45 GB exceeded; lower context (`overrides.context`), use a smaller quant, or raise the budget in Settings (never above RAM − 8 GB) |7| `MODEL_INCOMPATIBLE` (architecture) | `pip show mlx-lm`, `llama-server --version` | runtime too old for the architecture: upgrade mlx-lm / llama.cpp, rescan |8| `MODEL_LOAD_FAILED` with log tail | `~/llm-api/logs/workers/worker-<id>.log` | corrupted download (re-download with force), missing tokenizer file, unsupported quant |9| `MODEL_LOAD_TIMEOUT` | same log + Activity Monitor | very large model on a cold SSD cache; raise `LOAD_TIMEOUT_SECONDS` |10| `WORKER_CRASHED` | worker log, `dmesg`-style Metal errors | out-of-memory at the Metal level → lower context / KV bits (`overrides.kv_bits=8`) |11| `CONTEXT_TOO_LARGE` | request | prompt longer than the loaded context; reload with a larger `context` (model page) |12| `GENERATION_TIMEOUT` | Settings | `GENERATION_TIMEOUT_SECONDS`; check thermal throttling in System |13| Swap warning / memory pressure `critical` | Dashboard alert, System | model exceeds the envelope; unload, lower budget; never leave it swapping |14| Download fails 401/403 | Settings → HF token | gated repo: set `HF_TOKEN`, accept the license on Hugging Face |15| Download fails 429 | Downloads history | HF rate limit: retry later |16| `INSUFFICIENT_DISK` | System → Storage | `MIN_FREE_DISK_GB` reserve; delete unused models |17| Dashboard 503 "not reachable" | `pm2 logs llm-api-web`, port 8301 | Next not running / build failed (`pnpm build`) |18| Public site 502 while `localhost:8300` is fine | `ssh BHS64 curl http://10.67.0.40:8300/health`, `pm2 ls` (`llm-api-wgfwd`) | WireGuard relay down: restart `llm-api-wgfwd` (managed firewall blocks non-Apple listeners on the wg address, see deployment.md) |19| Login loop | cookies | behind HTTPS set `SECURE_COOKIES=true`; ensure the proxy passes `X-Forwarded-Proto` |20| Model shows loaded after a crash | restart server | stale state is cleared at startup (`workers.json`) |21| GPU shows `—` | `ioreg -r -c IOAccelerator` | metric unavailable on this macOS; degrades gracefully |2223Logs: `~/llm-api/logs/llm-api.log` (server), `~/llm-api/logs/workers/*.log` (per model), `pm2 logs`. Events: `/api/logs/events`, audit `/api/logs/audit`.2425Handy commands on the node:2627```bash28source ~/apps/llm-api/server/.venv/bin/activate && cd ~/llm-api29llm-api status # hardware + telemetry + model count30llm-api scan # rescan and list registry with compatibility31pgrep -fl "mlx_worker|llama-server"32curl -s localhost:8300/health33```34