SPB Git

Repository Public

TEST_REPORT.md 2.8 KB

# Test report — 2026-11-20, MacBook Pro M5 Max (48 GB), macOS 27.0, Xcode 27.2

# Executed

swift test --disable-keychain — 200 tests, 0 failures (log: build/logs/all-tests.log).

Target Tests What is real
LocalChatCoreTests 29 crypto known-answer vectors, Keychain (really executed), path sandbox, SSE, URL policy
LocalChatDataTests 88 SQLite stores, branching, FTS5, export / import, sync records, extraction of generated PDF / DOCX / CSV, vector search (100 000 × 384: 8–12 ms warm), stubbed Hub + resumable downloads, GGUF header, per-device MLX catalog, Apple sentence embeddings end to end
LocalChatServiceTests 73 API server over real sockets against the remote-node client (stream, auth 401, rate limit 429, cancel), agent loop with scripted model (permissions, approvals, limits, shell deny-list, SSRF guard), encrypted sync convergence
LocalChatInferenceTests 9 real GPU inference: MLX streaming answer, KV-cache reuse on turn 2, cancellation, tool call (get_weather(city: Montreal)), engine load / observe / unload, refusal of a model that cannot fit; llama.cpp answer with prefix reuse
LocalChatAppTests 1 end to end through the app layer: real download of mlx-community/Qwen3-0.6B-4bit (351 MB, SHA-256 verified), chat, automatic title, regenerate → sibling branch, branch switch, second turn, full-text search, metrics recorded, document indexed and cited in a grounded answer

Also run once with LC_LIVE=1: live Hugging Face search, starter catalog resolution, one real small download.

# Checked by hand (screenshots)

  • macOS Debug app: launch, real chat with Markdown, LaTeX formula, highlighted code, context bar, metrics, inspector; Models and Observatory screens with measured values (95.6 tokens/s for Qwen3-0.6B-4bit).
  • macOS Release app (Developer ID, sandbox, hardened runtime): launches; spctl → "accepted, Notarized Developer ID".
  • iOS app in the iPhone 17 Pro Simulator (iOS 26.5): launch, tab layout, new chat, send; the system model's failure in the Simulator is reported in the transcript.

# Bugs found by these runs and fixed

  • A late database observation event could replace the conversation state with an older one, so a turn was generated without the newest message. The controller now always re-reads the store.
  • The system inspector caused an AppKit constraint loop (crash) during streaming; replaced by a plain side panel.

# Not executed

  • LocalChatMacUITests / LocalChatIOSUITests (XCUITest): the Xcode test runner timed out before starting on this machine while it was under heavy load (load average above 50). They are written, never run.
  • Anything on a physical iPhone or iPad. See LIMITATIONS.md.
Files main
Menu

Theme