SPB Git

Repository Public

0004-agent-transport-mtls-lan.md 3.2 KB

# 0004 — Agents connect outbound over the LAN with mTLS; commands are signed by the orchestrator

Context (observed). The public entry (Caddy on the gateway) terminates TLS, so client certificates cannot reach the backend through it. 12 Macs have no WireGuard tunnel; all are on one LAN whose DHCP addresses have already been renumbered once.

Decision.

  • telemetry-ingest listens on the host node (LAN + tunnel address), TLS 1.3, client certificates issued by a private CA created by the product. It is never routed by the public gateway.
  • Agents open outbound connections only; they listen on nothing. They try a list of endpoints and authenticate the server by certificate (fixed name ingest.mcc.internal), not by address.
  • Enrollment uses a single-use token (15 min) and a CSR; the private key never leaves the node. Certificates last 30 days and renew over mTLS.
  • Commands are signed (Ed25519) by the orchestrator, which holds the only signing key. The ingest service merely transports them: if it is compromised it can drop or delay commands but cannot forge or alter one. Agents verify signature, recipient, expiry (≤ 120 s) and a persisted nonce, then apply their local allowlist.

Consequences. A node outside the LAN (the Linux gateway) reaches ingest through its tunnel address. A revoked agent is cut off within seconds (certificate serial checked per request).

# Amendment (2026-11-18) — signed HTTPS through the public origin is the default transport

Measured. On node m4ml (macOS 26.7), a Go binary started as a LaunchAgent gets connect: no route to host when it dials LAN peers (192.168.2.x), while the same binary succeeds from an SSH session. From the LaunchAgent the Internet and the WireGuard hub are reachable. This is macOS Local Network Privacy: consent can only be given in the graphical session of each Mac, per binary, and is not available to launchd daemons. Direct-LAN mTLS is therefore unusable for a launchd-started agent on these machines.

Decision. Agents talk to https://clustr.spboucher.ai/agent/v1/… (public address, not subject to that restriction, reachable from every node including those without a tunnel). Because the gateway terminates TLS, client certificates cannot be used at the TLS layer; instead every request carries a proof of possession of the agent's enrolled certificate key: the certificate, a timestamp, a nonce and an ECDSA signature over method, URI, timestamp, nonce and body hash. telemetry-ingest verifies chain, validity, revocation, freshness, replay and signature before processing. control-api only relays /agent/v1/ to ingest on loopback and strips user credentials; it cannot forge an agent request (no agent key) nor a command (no orchestrator key). The enrollment CLI still checks the private CA fingerprint returned at enrollment. The mTLS listener remains available as an alternative endpoint type for environments where direct connections are permitted.

Trade-off. Telemetry now depends on the Internet path to the gateway. During an outage agents buffer up to 6 hours and replay in order; nothing is lost within that window, and the dashboard shows nodes as stale/offline — which is also what the operator would experience, since the UI uses the same entry point.

Files main
Menu

Theme