1# CLAUDE.md — LLM API (www.llm-api.io)23Private OpenAI-compatible local model server for Apple Silicon. Full product spec: see `docs/` and README.4Runs on **M1M64** (Mac Studio M1 Max 64 GB, rented, user `simon`), deployed with `mld deploy llm-api --node M1M64`5(manifest `M1M32:~/dispatch/apps/llm-api.json`), public via MacLustr Tunnel (BHS64) at https://www.llm-api.io.67## Layout8- `server/` Python 3.13 FastAPI (`llm_api`): gateway, registry, manager, workers (`llm_api/worker/mlx_worker.py`, llama-server), tests (`pytest`, fake workers, no MLX needed).9- `web/` Next.js 16 console (proxied by the API in production; `pnpm dev` proxies /api,/v1 to `API_URL`).10- Data root on the node: `~/llm-api/` (`models/` library, `data/` SQLite + `.secret` + `.env`, `logs/`). Code: `~/apps/llm-api/`.1112## Rules (non-negotiable)131. Never load a model whose estimate exceeds the budget; never rely on swap; never fake metrics.142. Workers only on 127.0.0.1; memory release = kill the worker process.153. Never mark ready before warm-up; never delete model files automatically.164. Verify MLX / mlx-lm / mlx-vlm / llama.cpp / huggingface-hub APIs against the installed versions before changing runtime code.175. Always test load → inference → unload → memory recovery after touching the manager or workers.1819## Dev loop20```bash21cd server && python -m compileall -q llm_api && rsync -az --delete --exclude __pycache__ --exclude .venv ./ M1M64:~/apps/llm-api/server/22ssh M1M64 'source ~/llm-api/.venv/bin/activate; cd ~/apps/llm-api/server && python -m pytest -q --basetemp=/tmp/llmapi-tests; rm -rf /tmp/llmapi-tests'23cd web && pnpm exec tsc --noEmit && pnpm build24```25Gotchas: pytest temp dirs must stay sparse (`--basetemp` + sparse fixtures — a real 400-layer fixture once filled the 1.8 TB SSD); `socket.getfqdn` hangs on the rented Mac (never use stock `HTTPServer.server_bind`); MLX work must stay on one thread per worker.26