spb/wp-spboucher-ai

1# Architecture23```4Browser ──HTTPS──▶ BHS64 Caddy ──wg1──▶ M3U96b :8560  wp-web (Next.js 16, PM2)5                                              │            └─ sqlPublic (role wp_reader → public_* views)  / sql (role wp_app)6                                              │7                                          wp-worker (tsx, PM2) ── jobs table ── poppler, OpenAI embeddings (optional), pg_dump8                                              │9                                          PostgreSQL 17 wp_spboucher (pgvector, pg_trgm, unaccent)   ~/data/wp-spboucher-ai (files, previews)10```1112## Data model (drizzle/0001_init.sql)13- `series` (WP, FIRC) · `papers` (permanent `identifier`/`slug`, `number`, status `draft|reserved|published|withdrawn|superseded`, publication status, dates with precision, structured sections as JSONB, keywords, JEL, generated `fts` tsvector and `search_blob`, `embedding vector(1536)`, `lexvec vector(512)`) · `paper_versions` (immutable history, `is_current`, `withdrawn`) · `files` (content-addressed, SHA-256, integrity flag, pdfinfo metadata) · `paper_chunks` (full text by page range, tsvector, embeddings).14- Taxonomy: `authors`, `topics` (kind `domain|topic`), `methods`, `datasets` and join tables carrying `assigned_by = editor|author|inferred`.15- Intelligence: `paper_relations` (`topic|method|dataset|author|semantic|lexical|curated`, weight, bilingual explanation, `inferred`).16- Operations: `users`, `sessions`, `login_attempts`, `audit_log`, `settings`, `redirects`, `events` (one per visitor/day/paper, bot-flagged), `paper_stats_daily`, `search_log`, `ingest_candidates`, `jobs`, `backups`.17- Public views `public_papers`, `public_paper_versions`, `public_paper_chunks`, `public_paper_relations`, `public_files`, `public_settings` are the only relations the read-only role may read, so unpublished records cannot leak through any public code path.1819## Request paths20- Public pages are server components under `src/app/(public)/[locale]`; `src/proxy.ts` rewrites canonical EN URLs to `/en/...` internally and leaves `/fr/...` as is, so ISR caches never mix languages. `/en/...` requested directly is redirected (308) to the canonical URL.21- Paper pages revalidate every 120 s; admin mutations call `revalidatePath` for the affected routes.22- Search (`src/lib/papers/search.ts`): lexical candidates from `papers.fts` (weighted A–D) + trigram similarity on `search_blob` + full-text chunk hits with `ts_headline`, fused by reciprocal rank with semantic candidates (paper embedding and chunk embeddings) when embeddings are available. Every hit reports the signals that matched.23- Downloads stream from disk with Range/ETag support; `events` counts one download per visitor per paper per day; `daily_stats` aggregates and updates counters.2425## Worker26Polls `jobs` (SKIP LOCKED, retries with back-off, 30-min stale lock recovery). Schedules: daily stats 00:10, relations 03:00, backup 04:40 (node local time). `process_pdf` → integrity, metadata, preview, full text → chunks → lexical vector → `embed_paper` → `relations`.2728## Relations29Deterministic links from shared topics, methods, datasets and non-primary co-authors (the series' primary author is on every paper and would add no information). Inferred links: top-4 lexical neighbours (hashed TF-IDF cosine ≥ 0.22) and top-4 semantic neighbours (embedding cosine ≥ 0.45). Curated links are added in `/admin` and never overwritten.3031## Ingestion32`ingest_scan` walks the inbox (+ configured directories), checksums every PDF, stores new files, creates candidates; `ingest_candidate` runs integrity, pdfinfo, title-page heuristics (title, authors, abstract, keywords, JEL, WP number, version, month/year), preview and duplicate detection (checksum, identifier, title similarity, author overlap). The administrator approves (draft paper or new version) or rejects; nothing is published without an explicit status change.33