# Architecture

text

Browser ──HTTPS──▶ BHS64 Caddy ──wg1──▶ M3U96b :8560  wp-web (Next.js 16, PM2)
                                              │            └─ sqlPublic (role wp_reader → public_* views)  / sql (role wp_app)
                                              │
                                          wp-worker (tsx, PM2) ── jobs table ── poppler, OpenAI embeddings (optional), pg_dump
                                              │
                                          PostgreSQL 17 wp_spboucher (pgvector, pg_trgm, unaccent)   ~/data/wp-spboucher-ai (files, previews)

# Data model (drizzle/0001_init.sql)

series (WP, FIRC) · papers (permanent identifier/slug, number, status draft|reserved|published|withdrawn|superseded, publication status, dates with precision, structured sections as JSONB, keywords, JEL, generated fts tsvector and search_blob, embedding vector(1536), lexvec vector(512)) · paper_versions (immutable history, is_current, withdrawn) · files (content-addressed, SHA-256, integrity flag, pdfinfo metadata) · paper_chunks (full text by page range, tsvector, embeddings).
Taxonomy: authors, topics (kind domain|topic), methods, datasets and join tables carrying assigned_by = editor|author|inferred.
Intelligence: paper_relations (topic|method|dataset|author|semantic|lexical|curated, weight, bilingual explanation, inferred).
Operations: users, sessions, login_attempts, audit_log, settings, redirects, events (one per visitor/day/paper, bot-flagged), paper_stats_daily, search_log, ingest_candidates, jobs, backups.
Public views public_papers, public_paper_versions, public_paper_chunks, public_paper_relations, public_files, public_settings are the only relations the read-only role may read, so unpublished records cannot leak through any public code path.

# Request paths

Public pages are server components under src/app/(public)/[locale]; src/proxy.ts rewrites canonical EN URLs to /en/... internally and leaves /fr/... as is, so ISR caches never mix languages. /en/... requested directly is redirected (308) to the canonical URL.
Paper pages revalidate every 120 s; admin mutations call revalidatePath for the affected routes.
Search (src/lib/papers/search.ts): lexical candidates from papers.fts (weighted A–D) + trigram similarity on search_blob + full-text chunk hits with ts_headline, fused by reciprocal rank with semantic candidates (paper embedding and chunk embeddings) when embeddings are available. Every hit reports the signals that matched.
Downloads stream from disk with Range/ETag support; events counts one download per visitor per paper per day; daily_stats aggregates and updates counters.

# Worker

Polls jobs (SKIP LOCKED, retries with back-off, 30-min stale lock recovery). Schedules: daily stats 00:10, relations 03:00, backup 04:40 (node local time). process_pdf → integrity, metadata, preview, full text → chunks → lexical vector → embed_paper → relations.

# Relations

Deterministic links from shared topics, methods, datasets and non-primary co-authors (the series' primary author is on every paper and would add no information). Inferred links: top-4 lexical neighbours (hashed TF-IDF cosine ≥ 0.22) and top-4 semantic neighbours (embedding cosine ≥ 0.45). Curated links are added in /admin and never overwritten.

# Ingestion

ingest_scan walks the inbox (+ configured directories), checksums every PDF, stores new files, creates candidates; ingest_candidate runs integrity, pdfinfo, title-page heuristics (title, authors, abstract, keywords, JEL, WP number, version, month/year), preview and duplicate detection (checksum, identifier, title similarity, author overlap). The administrator approves (draft paper or new version) or rejects; nothing is published without an explicit status change.

spb/wp-spboucher-ai