"use client"; import { Code, PageHeader } from "@/components/ui"; export default function DocsPage() { const origin = typeof location !== "undefined" ? location.origin : "https://www.llm-api.io"; return (
Base URL
{`${origin}/v1`}
Endpoints: GET /v1/models, POST /v1/chat/completions, POST /v1/completions, POST /v1/embeddings, POST /v1/rerank. Interactive OpenAPI schema at /openapi.
Python (openai SDK)
{`from openai import OpenAI client = OpenAI(base_url="${origin}/v1", api_key="llm_live_xxxxx") # Any installed model id, an alias (fast, coder, reasoning…) or "auto" r = client.chat.completions.create( model="default", messages=[{"role": "user", "content": "Explain monetary policy in two sentences."}], ) print(r.choices[0].message.content) # Streaming for chunk in client.chat.completions.create(model="default", messages=[{"role":"user","content":"Count to 10"}], stream=True): if chunk.choices and chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True) # Embeddings (embedding model or alias "embedding") e = client.embeddings.create(model="embedding", input=["hello world"])`}
curl
{`curl ${origin}/v1/models -H "Authorization: Bearer llm_live_xxxxx" curl ${origin}/v1/chat/completions \\ -H "Authorization: Bearer llm_live_xxxxx" \\ -H "Content-Type: application/json" \\ -d '{"model": "default", "messages": [{"role": "user", "content": "Hello"}], "stream": true}'`}
Behaviour
  • If the requested model is not loaded, the request waits while it is loaded from SSD (the previous model is unloaded first if memory requires it). Load times are typically 1–30 s depending on size.
  • Model switches are serialized. Concurrent requests for the same model are queued at the worker.
  • Reasoning models return reasoning_content in the message / delta. Disable thinking with {"\"chat_template_kwargs\": {\"enable_thinking\": false}"}.
  • Tool calling: pass tools; calls come back as tool_calls for models with a tool-capable chat template.
  • Every response includes a timings object (TTFT, tokens/s, peak memory) in addition to usage.
  • Errors follow the OpenAI shape: {"{\"error\": {\"message\", \"type\", \"code\"}}"}, e.g. MODEL_NOT_FOUND, MODEL_TOO_LARGE, CONTEXT_TOO_LARGE, WORKER_CRASHED.
Management API
Same host, prefix /api, requires the dashboard session or an API key with the admin scope: GET /api/models, POST /api/models/:id/load|unload|benchmark|pin, POST /api/models/download, GET /api/system, GET /api/events (SSE).
); }