SPB Git forge

spb/rareindex

Public
54commits 1branches 0releases
7.1 MBsize
maindefault branch
10 days agolast push
TypeScript 61.9% HTML 37.2% SQL 0.7%
1.8 KB

# ADR 0001 — TypeScript end-to-end, pnpm monorepo, PostgreSQL-backed queue

Date: 2026-09-07 · Status: accepted

# Context

CLAUDE.md allows Node/TypeScript or Python services (§167) and suggests a layout (§195). The launch wedge must ship quickly on a single node while the architecture must scale to hundreds of millions of observations.

# Decision

  1. TypeScript everywhere (web, API, workers, connectors). One runtime, shared Zod schemas between pipeline and UI, Drizzle typed queries, and the connector interface exactly as specified (§102).
  2. pnpm workspaces, packages consumed as TS sources (transpilePackages in Next, tsx for workers).
  3. PostgreSQL 17 as system of record (pg_trgm, unaccent, pgvector). No Timescale for now; time series tables (price_snapshots, index_values) use composite PKs and date indexes; partitioning can be added by migration.
  4. pg-boss as the queue (Postgres-backed), wrapped by a small Queue interface in workers/lib so Redis Streams/Kafka can be substituted without touching connectors or handlers (§140).
  5. Admin UI lives inside apps/web under /admin (token-protected) instead of a separate apps/admin, to avoid duplicating the design system in Phase 1. It can be split later.
  6. Registry from meta files: connectors/registry.json is generated from per-connector meta.json (avoids merge conflicts between parallel contributors) — it remains the artefact §106 asks for.

# Consequences

  • Data-science-heavy methods (hedonic regression, repeat-sales) are implemented in TS with small numerical helpers; a Python research service can be added later behind the same tables.
  • Everything deploys as PM2 processes on one node via mld; horizontal scale = more worker processes pointed at the same Postgres/pg-boss.