Architecture
Apps/LocalChatMac Apps/LocalChatIOS (scenes, commands, menu bar, lifecycle)
\ /
LocalChatUI — AppModel, ChatController, every screen, design system
┌──────────┬───────────┼────────────┬───────────────┬─────────────┐
Inference ModelHub Conversations Knowledge Agents Benchmarks Sync RemoteInference LocalAPIServer
└──────────┴───────────┴─────┬──────┴───────────────┴─────────────┘
LocalChatCore (+ Security, Networking)Rules
- Feature modules depend on
LocalChatCore, never on each other. They meet through Core protocols:InferenceBackend/ModelRuntime,TextGenerating/InferenceService,EmbeddingProvider,KnowledgeSearching,SyncableStore,HTTPRouteHandler. - Only
LocalChatUIknows every module.AppModelbuilds the services once; views read its observable state. - One on-device generation at a time:
InferenceEnginequeues requests, loads models on demand, unloads the least recently used model when memory is needed, and releases idle models on memory pressure. Remote generations run concurrently. - Every number shown comes from a measurement or is labelled as an estimate (
Provenance, "≈", "est.").
A chat turn
ChatController.send → store the user message → pick the model (or ModelRouter when Auto) → knowledge search when collections are attached → ContextAssembler fits system prompt, summary, sources and recent turns in the token budget (exact counts from the loaded tokenizer) → InferenceEngine.generate streams events → text is shown live and flushed to SQLite every 0.6 s → final message saved with measured metrics → automatic title → compaction (a summary record; messages are never modified).
Edits and regenerations create sibling messages (parentID), so the conversation is a tree; ConversationTree computes the active branch.
Data on disk
Application Support/LocalChat/ (inside the app container): Data/localchat.sqlite (conversations, FTS5), Data/knowledge.sqlite (chunks, vectors), Data/metrics.sqlite, Models/ (excluded from backups), Downloads/, Attachments/, AgentRuns/. Secrets (node API keys, Hugging Face token, sync key) are in the Keychain.
Privacy defaults
Nothing leaves the device unless the user acts: browsing or downloading models contacts huggingface.co; a remote node receives the conversations that use one of its models (shown on the conversation); sync is off until configured and is encrypted before upload; the API server listens on 127.0.0.1 only.