# CLAUDE.md — Company Atlas (repo guide)

Product: Company Atlas — The Live Atlas of Global Companies. A distributed public-web sensor network attached to companies: observations → snapshots → changes → structured events → metrics → intelligence. The accumulated history is the product. Full product specification (200 sections, non-negotiable principles): docs/PRODUCT-SPEC.md. Read it before changing behaviour.

Domains: canonical https://www.company-atlas.co today (DNS A → MacLustr Tunnel 51.161.112.61). www.company-atlas.com is the spec's primary hostname but its nameservers are at Vercel, not GoDaddy — once its A records point to the gateway, add it as a tunnel route/redirect and flip CA_SITE_URL. Apex → www 308.

# Stack

• Python 3.12 src/companyatlas (FastAPI, SQLAlchemy Core + asyncpg, Alembic SQL forward-only, httpx, selectolax, feedparser, zstd), CLI catlas (commands auto-discovered from companyatlas/commands/*.py).
• Postgres 17 only (no Redis): queue = queue_jobs with SKIP LOCKED; caches in-process (api/common.TTLCache).
• Objects: content-addressed zstd store CA_DATA_DIR/objects/ab/cd/<sha256>.zst (archive.py).
• Web: apps/web Next 16 + React 19 + Tailwind v4 (Geist), SSR, rewrites /api/v1/* → FastAPI loopback. Ports: prod web 8360 / api 8361; dev web 8370 / api 8371 (see .env.example).
• LLM enrichment: OpenAI-compatible CA_LLM_BASE_URL (MacLustr llm-api.io, key "company-atlas" in deploy/.llm-key, git-ignored). Optional; deterministic first.

# Layout

src/companyatlas/   config.py taxonomy.py ids.py urls.py fetch.py archive.py db/ logging.py cli.py
                    sdk/ (models, normalize, diff, connector)  connectors/  services/ (discovery, pipeline, scheduler, events, llm, metrics…)
                    api/ (main, common, routers/*)  commands/ (CLI groups)  registry/ (seed loader)
migrations/versions/0001_initial.py   registry/ (companies ndjson, industries.yaml, countries.csv)   prompts/   fixtures/   tests/
apps/web/   docs/ (PRODUCT-SPEC, ARCHITECTURE, API, DATA-MODEL, CONNECTORS, SCORING, DEPLOY, OPERATIONS)   deploy/ (mld manifest, scripts)

Architecture, ownership map and boundaries: docs/ARCHITECTURE.md. API contract: docs/API.md.

# Operating rules (spec §182)

1. Inspect existing architecture first; preserve working functionality; avoid rewrites; reuse the SDK.
2. Never fabricate data. No inference presented as fact: careful language (detected, no longer listed, signal), confidence labels.
3. Historical-first: never overwrite or delete history; new versions, status columns, forward-only migrations.
4. Raw vs interpreted stay separate (objects ↔ snapshots ↔ changes ↔ events). Reprocessing must never require re-fetching.
5. LLMs are enrichment, not the crawler: deterministic fetch → normalize → hash → diff → significance → LLM only if useful, budgeted.
6. Only fetch.Fetcher talks to the network: SSRF guard, robots, per-domain rate/concurrency, size and redirect caps. Never bypass authentication or challenges; never collect private data.
7. No magic numbers: tunables in config.Settings / taxonomy.py; bump formula/prompt/connector versions when behaviour changes.
8. Mobile is first-class; dark and light both designed; dense, readable, terminal-grade UI.
9. Tests use fixtures (fixtures/), never the live web (-m live opt-in). Run .venv/bin/pytest -q, .venv/bin/ruff check src tests, pnpm typecheck after meaningful changes.
10. Secrets only in env / git-ignored files (deploy/.admin-token, deploy/.llm-key); the rendered manifest lives on M1M32.

# Dev quickstart

uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -e '.[dev]'
createdb -O companyatlas companyatlas   # role companyatlas/companyatlas, extensions pg_trgm + uuid-ossp
cp .env.example .env && .venv/bin/catlas migrate && .venv/bin/catlas seed
.venv/bin/catlas onboard --limit 50      # discovery + sensors for the first companies
.venv/bin/catlas schedule                # scheduler + workers (Ctrl-C to stop)
.venv/bin/catlas api                     # http://127.0.0.1:8371/api/v1/docs
pnpm install && pnpm dev:web             # http://localhost:8370

# Deploy (see docs/DEPLOY.md)

Since 2026-09-14: OVH server BHS128b (ssh BHS128b, Ubuntu 24.04), outside mld — dir ~/apps/company-atlas (uv Python 3.12 venv), data ~/company-atlas-data, Postgres 17 native (companyatlas), PM2 under systemd (~/apps/pm2.company-atlas.config.cjs, generated from the mld manifest kept in ~/apps/.manifests/): company-atlas-api / company-atlas-scheduler / company-atlas-web. Route https://www.company-atlas.co → wg1 10.67.0.62:8360. Release = rsync + uv pip install -e . + catlas migrate + web build + pm2 restart (exact command in docs/DEPLOY.md). Former home: M2U64 via mld (deploy/render-manifest.sh --push → mld stage → mld deploy), cold copy kept.