SPB Git

wp-spboucher-ai: initial release of the Research Working Papers observatory

Next.js 16 + PostgreSQL 17/pgvector platform for wp.spboucher.ai: publication library (permanent identifiers,
versioned manuscripts, citations), discovery (filters, hybrid search, topics/methods/datasets/authors, timeline,
research map), administration (auth, ingestion queue, versions, taxonomy, audit), public API, OAI-PMH, RSS, backups.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Nov 2, 2026) Browse files

149 changed files +9,812 −0

149 changed files
Unified Split
added .env.example +19 −0
@@ -0,0 +1,19 @@
1 +APP_URL=https://wp.spboucher.ai
2 +PORT=8560
3 +NODE_ENV=production
4 +DATABASE_URL_ADMIN=postgres://<superuser>@localhost:5432/wp_spboucher
5 +DATABASE_URL=postgres://wp_app:<pw>@localhost:5432/wp_spboucher
6 +DATABASE_URL_PUBLIC=postgres://wp_reader:<pw>@localhost:5432/wp_spboucher
7 +SESSION_SECRET=<64 random chars>
8 +OWNER_EMAIL=owner@example.org
9 +OWNER_INITIAL_PASSWORD=<initial password, must be changed at first login>
10 +STORAGE_DIR=/Users/<user>/data/wp-spboucher-ai
11 +AI_PROVIDER=none # none | openai (embeddings only; the platform works without AI)
12 +OPENAI_API_KEY=
13 +OPENAI_EMBEDDING_MODEL=text-embedding-3-small
14 +AI_MONTHLY_BUDGET_USD=10
15 +BACKUP_DIR=/Users/<user>/backups/wp-spboucher-ai
16 +BACKUP_PASSPHRASE=<passphrase>
17 +BACKUP_REMOTE= # optional rsync target host:path
18 +PG_BIN=/opt/homebrew/opt/postgresql@17/bin
19 +WORKER_POLL_MS=5000
added .gitignore +14 −0
@@ -0,0 +1,14 @@
1 +node_modules/
2 +.next/
3 +.env
4 +.env.*
5 +!.env.example
6 +tsconfig.tsbuildinfo
7 +next-env.d.ts
8 +test-results/
9 +playwright-report/
10 +storage/
11 +backups/
12 +*.log
13 +.DS_Store
14 +docs/_verification-*.json
added CLAUDE.md +25 −0
@@ -0,0 +1,25 @@
1 +# wp-spboucher-ai — Research Working Papers (wp.spboucher.ai)
2 +
3 +Public working-paper observatory of Simon-Pierre Boucher (UQO). Deployed on MacLustr node **M3U96b** by `mld` (app `wp-spboucher-ai`, PM2 `wp-web` :8560 + `wp-worker`, Postgres 17 DB `wp_spboucher`, manuscripts in `~/data/wp-spboucher-ai`). Public route: BHS64 Caddy → wg1 M3U96b:8560.
4 +
5 +**Remote-first**: the source of truth is the clone on the node (`M3U96b:~/apps/wp-spboucher-ai`, origin spbgit `wp-spboucher-ai.git`). Edit there, `pnpm build && pm2 restart wp-web wp-worker`, then commit/push on the node; `git pull` on the laptop mirror (`~/Desktop/Cluster/projets/wp-spboucher-ai`).
6 +
7 +## Map
8 +- `drizzle/*.sql` — schema (checksummed SQL migrations, `scripts/migrate.ts`); `public_*` views are the only objects the read-only role sees.
9 +- `src/db` — postgres-js clients: `sql` (app role) / `sqlPublic` (read-only role; all public pages and API).
10 +- `src/lib/papers` — public queries, hybrid search (tsvector + trigram + pgvector RRF), citations (APA/Chicago/BibTeX/RIS), identifiers (`WP38` ⇄ `wp38`, permanent), Scholar/DC/JSON-LD meta, OpenAPI.
11 +- `src/lib/pdf` — poppler pipeline (pdfinfo/pdftotext/pdftoppm), SHA-256 content-addressed storage, metadata proposal heuristics (suggestions only), chunking.
12 +- `src/lib/jobs` — worker handlers: `process_pdf`, `embed_paper` (OpenAI, optional, model verified at runtime), `relations` (deterministic + inferred, labelled), `daily_stats`, `backup` (pg_dump + tar, AES-256, 14 d, rsync), `ingest_scan` / `ingest_candidate`.
13 +- `src/app/(public)/[locale]` — public pages; canonical URLs are unprefixed EN, FR under `/fr/...` (rewrite in `src/proxy.ts`, separate caches).
14 +- `src/app/admin` + `src/lib/admin/actions.ts` — administration (session cookie, Argon2id, TOTP, CSRF, same-origin, audit log).
15 +- `src/app/api` — public API v1, files (range, ETag, download counting), OAI-PMH, events beacon, health, admin upload.
16 +- `content/seed` — verified catalog (from www.spboucher.ai, 2026-11-01) + editorial taxonomy; `pnpm seed:import --publish` is idempotent.
17 +
18 +## Rules
19 +- Never fabricate metadata; dates keep their precision (`day|month|year`); reserved identifiers (WP39, WP40) stay non-public until approved in `/admin`.
20 +- Working paper numbers are permanent; redirects table handles historical slugs.
21 +- Statistics are computed live; relations inferred from similarity are flagged `inferred`.
22 +- Secrets only in `.env` (node) and `~/Desktop/Cluster/secrets/wp-spboucher-ai/.env` (laptop copy).
23 +
24 +## Commands
25 +`pnpm test` (vitest) · `pnpm test:e2e` (Playwright, `E2E_BASE_URL`) · `pnpm test:security [url]` · `pnpm verify:deploy [url]` · `pnpm backup` · `pnpm restore:drill` · `pnpm ingest:scan [dir]` · `pnpm relations:rebuild`.
added README.md +28 −0
@@ -0,0 +1,28 @@
1 +# Research Working Papers — Simon-Pierre Boucher Research Observatory
2 +
3 +Public scientific publishing platform for the UQO working papers of Simon-Pierre Boucher: **https://wp.spboucher.ai**.
4 +
5 +Three layers: a **publication library** (stable identifiers, versioned manuscripts, rich metadata, citation exports), a **discovery engine** (topics, methods, datasets, authors, timeline, typo-tolerant and semantic search) and a **research-intelligence layer** (knowledge graph of papers linked by methods, data, topics, authors and labelled textual similarity).
6 +
7 +| | |
8 +|---|---|
9 +| Stack | Next.js 16 (App Router, React 19, TypeScript, Tailwind 4), PostgreSQL 17 + pgvector + pg_trgm + unaccent, postgres-js, poppler, PM2, OpenAI embeddings (optional) |
10 +| Node | MacLustr **M3U96b** (`~/apps/wp-spboucher-ai`), deployed with `mld`; public route through the MacLustr Tunnel (BHS64, Caddy, Let's Encrypt) |
11 +| Languages | English (canonical URLs) and French (`/fr/...`) |
12 +| Discoverability | Highwire Press (Google Scholar) + Dublin Core meta tags, JSON-LD `ScholarlyArticle`, OAI-PMH (`/api/oai`), RSS (`/feed.xml`), sitemap, public JSON API (`/api/docs`) |
13 +| Administration | `/admin` — password (Argon2id) + optional TOTP, CSRF, audit trail, ingestion queue, versions, taxonomy, settings, analytics, backups |
14 +
15 +## Development
16 +
17 +```bash
18 +pnpm install
19 +cp .env.example .env # fill in (see docs/operations.md)
20 +pnpm db:migrate && pnpm db:roles && pnpm bootstrap
21 +pnpm seed:import --publish # verified catalog + manuscripts from $STORAGE_DIR/inbox
22 +pnpm worker # in another terminal
23 +pnpm dev # http://localhost:8560
24 +```
25 +
26 +Tests: `pnpm test` (unit), `pnpm test:e2e` (Playwright, set `E2E_BASE_URL`), `pnpm test:security [url]`, `pnpm verify:deploy [url]`.
27 +
28 +Documentation: `docs/infrastructure-report.md` (discovery and compatibility), `docs/architecture.md`, `docs/operations.md` (runbook), `docs/api.md`, `docs/verification.md` (test evidence).
added content/seed/catalog.json +5567 −0
@@ -0,0 +1,5567 @@
1 +{
2 + "fields": [
3 + {
4 + "id": "real-estate",
5 + "label": "Real Estate & Urban Economics",
6 + "short": "Real estate",
7 + "blurb": "Hedonic pricing, automated valuation, property taxation, housing indices and housing theory, mostly on large Quebec and Canadian transaction datasets."
8 + },
9 + {
10 + "id": "finance",
11 + "label": "Finance & Economics",
12 + "short": "Finance",
13 + "blurb": "High-frequency markets, derivatives, prediction markets, volatility forecasting, distributed estimation, and the general-equilibrium economics of AI agents."
14 + },
15 + {
16 + "id": "ai",
17 + "label": "AI Systems",
18 + "short": "AI systems",
19 + "blurb": "Measured systems studies of language models: multi-agent cache transfer, sparse small-model architectures, and distributed mixture-of-experts inference on Apple Silicon."
20 + },
21 + {
22 + "id": "methods",
23 + "label": "Research Methods",
24 + "short": "Methods",
25 + "blurb": "Controlled studies of protocol-guided autonomous research with coding agents: verification, reproducibility, and the cost of delegation."
26 + },
27 + {
28 + "id": "math",
29 + "label": "Mathematics",
30 + "short": "Mathematics",
31 + "blurb": "Computational discovery with certified proofs — SAT certificates and formal verification in combinatorics."
32 + }
33 + ],
34 + "papers": [
35 + {
36 + "slug": "wp38",
37 + "kind": "wp",
38 + "num": "UQO Working Paper No. 38",
39 + "number": 38,
40 + "field": "finance",
41 + "title": "The Economic Geography of Digital Retail Supply: Product Diversity, Regional Specialization, and the Hidden Structure of Quebec's Online Economy",
42 + "subtitle": "Evidence from 390,108 product listings of 1,569 Quebec online stores across 17 administrative regions (Fabri-Ka snapshot, 1 November 2026)",
43 + "version": "1.0",
44 + "date": "November 2026",
45 + "year": 2026,
46 + "pages": "51 pages",
47 + "pageCount": 51,
48 + "authors": [
49 + "Simon-Pierre Boucher"
50 + ],
51 + "description": "Tests, under a preregistered design with merchant-reshuffling nulls and three adversarial referee rounds, whether online catalogues of 1,569 Quebec stores reveal regional specialization: the signal is about who sells, not how much is listed (Cramér's V 0.112 vs 0.086 under merchant-equal weights), catalogues are extremely concentrated (Gini 0.85) and the complexity indicator fails validation and is withheld.",
52 + "pdf": "/papers/wp38_uqo.pdf",
53 + "repo": "https://git.spboucher.ai/wp38_uqo"
54 + },
55 + {
56 + "slug": "wp37",
57 + "kind": "wp",
58 + "num": "UQO Working Paper No. 37",
59 + "number": 37,
60 + "field": "finance",
61 + "title": "The Hidden Architecture of Grocery Prices: Promotions, Retail Competition, and the Cost of Living in Quebec",
62 + "subtitle": "Evidence from 300,613 posted-price events across 54 Quebec grocery feeds, August–November 2026",
63 + "version": "1.0",
64 + "date": "November 2026",
65 + "year": 2026,
66 + "pages": "45 pages",
67 + "pageCount": 45,
68 + "authors": [
69 + "Simon-Pierre Boucher"
70 + ],
71 + "description": "Assembles an auditable panel of 124,579 grocery listings from 54 Quebec feeds (300,613 price events) and shows that identical products differ by a median 0.154 log points across banners, that promotions account for half of all price changes and double measured volatility, that sales return exactly to their reference price, and that naive indices are off by 10–30 points for banners with assortment turnover; publishes a five-component Quebec Grocery Price and Affordability Index.",
72 + "pdf": "/papers/wp37_uqo.pdf",
73 + "repo": "https://git.spboucher.ai/wp37_uqo"
74 + },
75 + {
76 + "slug": "wp36",
77 + "kind": "wp",
78 + "num": "UQO Working Paper No. 36",
79 + "number": 36,
80 + "field": "finance",
81 + "title": "The Hidden Dynamics of Used-Car Markets: Price Discovery, Inventory Liquidity, and Strategic Dealer Behavior",
82 + "subtitle": "Evidence from a Daily Panel of 273,265 Quebec Listings",
83 + "version": "1.0",
84 + "date": "November 2026",
85 + "year": 2026,
86 + "pages": "44 pages",
87 + "pageCount": 44,
88 + "authors": [
89 + "Simon-Pierre Boucher"
90 + ],
91 + "description": "Follows 273,265 Quebec dealer listings about seven times a day for eleven weeks: depreciation is nonlinear, one listing in five is marked down (median 2.8%) with a hazard that doubles over three weeks, same-day repricing co-moves among neighbours without any lagged reaction, regional premia are within 4% of Montréal, and a five-component Automobile Market Liquidity Index explains 56% of the common variation in regional turnover.",
92 + "pdf": "/papers/wp36_uqo.pdf",
93 + "repo": "https://git.spboucher.ai/wp36_uqo"
94 + },
95 + {
96 + "slug": "wp35",
97 + "kind": "wp",
98 + "num": "UQO Working Paper No. 35",
99 + "number": 35,
100 + "field": "finance",
101 + "title": "The Social Compression Hypothesis",
102 + "subtitle": "A Pre-Registered Test of Coordination Demand, Institutional Capacity, and Instability in Historical and Modern Societies, with a Formal Framework and Agent-Based Experiments",
103 + "version": "1.0",
104 + "date": "November 2026",
105 + "year": 2026,
106 + "pages": "80 pages",
107 + "pageCount": 80,
108 + "authors": [
109 + "Simon-Pierre Boucher"
110 + ],
111 + "description": "Reads institutions as compression devices that lower the communication cost of coordination, proves five theorems and runs 6,048 agent-based experiments, then tests the empirical corollary on Seshat (626 polities), the Ethnographic Atlas and a 14,637 country-year panel under a frozen pre-registration: the demand–capacity mismatch is associated with instability (0.20 and 0.39 log-odds per SD) but adds nothing to scale or capacity alone, so the universal theory is not supported.",
112 + "pdf": "/papers/wp35_uqo.pdf",
113 + "repo": "https://git.spboucher.ai/wp35_uqo"
114 + },
115 + {
116 + "slug": "wp34",
117 + "kind": "wp",
118 + "num": "UQO Working Paper No. 34",
119 + "number": 34,
120 + "field": "math",
121 + "title": "Beyond the Square-Root Barrier: Automated Discovery of Algebraic and Algorithmic Shortcuts in Bitcoin Cryptography",
122 + "subtitle": "A hypothesis-driven, reproducible study of secp256k1, ECDSA, BIP340 Schnorr and SHA-256 with machine-checked identities and controlled negative results",
123 + "version": "1.0",
124 + "date": "November 2026",
125 + "year": 2026,
126 + "pages": "74 pages",
127 + "pageCount": 74,
128 + "authors": [
129 + "Simon-Pierre Boucher"
130 + ],
131 + "description": "Runs a six-stage autonomous discovery pipeline (25 hypotheses, exact symbolic checks, 6,240 synthetic ECDLP instances, SAT cryptanalysis of reduced SHA-256, Lean 4 proofs) against Bitcoin's primitives and finds no shortcut: the secp256k1 structure is fully explained by complex multiplication by Z[ω], Pollard rho matches 1.254√n, batch Schnorr verification saturates at 3.1×, and sixteen reproductions, six observations and three rejected conjectures are released with full provenance.",
132 + "pdf": "/papers/wp34_uqo.pdf",
133 + "repo": "https://git.spboucher.ai/wp34_uqo"
134 + },
135 + {
136 + "slug": "wp33",
137 + "kind": "wp",
138 + "num": "UQO Working Paper No. 33",
139 + "number": 33,
140 + "field": "ai",
141 + "title": "TITAN-MoE: Topology-Aware and Predictive Expert Orchestration for Distributed Mixture-of-Experts Inference on Apple Silicon",
142 + "subtitle": "A Measured Systems Study on a Heterogeneous Mac Cluster",
143 + "version": "0.1",
144 + "date": "November 2026",
145 + "year": 2026,
146 + "pages": "30 pages",
147 + "pageCount": 30,
148 + "authors": [
149 + "Simon-Pierre Boucher"
150 + ],
151 + "description": "Builds and measures TITAN-MoE, an MLX-native mixture-of-experts runtime with bounded expert residency, NVMe loading, causal prefetch and remote execution on a 21-node Mac cluster; bit-identical to mlx-lm at 489 vs 494 tok/s, but prefetch, kernel fusion and the gigabit remote tier do not pay (preliminary draft)",
152 + "status": "Preliminary draft",
153 + "pdf": "/papers/wp33_uqo.pdf"
154 + },
155 + {
156 + "slug": "wp32",
157 + "kind": "wp",
158 + "num": "UQO Working Paper No. 32",
159 + "number": 32,
160 + "field": "math",
161 + "title": "Zero-Forcing Bounds for Generalized Petersen Graphs: Computational Discovery and Certified Proofs for P(n,5)",
162 + "version": "1.0",
163 + "date": "October 2026",
164 + "year": 2026,
165 + "pages": "36 pages",
166 + "pageCount": 36,
167 + "authors": [
168 + "Simon-Pierre Boucher"
169 + ],
170 + "description": "Determines the zero forcing number of the generalized Petersen graph P(n,5) for every n ≥ 11 and proves that Z(P(n,5)) = 12 for all n ≥ 23, using the first-force boundary method, 129 drat-trim-verified DRAT certificates and exhaustive enumeration of 432 billion subsets.",
171 + "pdf": "/papers/wp32_uqo.pdf",
172 + "repo": "https://git.spboucher.ai/wp32_uqo"
173 + },
174 + {
175 + "slug": "wp31",
176 + "kind": "wp",
177 + "num": "UQO Working Paper No. 31",
178 + "number": 31,
179 + "field": "ai",
180 + "title": "PRISME: Intervention-Supervised Sparse Workspace Editing for Small Language Models",
181 + "subtitle": "Design, controlled evaluation, and negative and positive evidence at the 100M-parameter scale",
182 + "version": "1.0",
183 + "date": "October 2026",
184 + "year": 2026,
185 + "pages": "49 pages",
186 + "pageCount": 49,
187 + "authors": [
188 + "Simon-Pierre Boucher"
189 + ],
190 + "description": "Implements, trains and tests PRISME, a sparse 16-slot latent workspace with a counterfactually supervised editing controller, against matched baselines; a preregistered negative result: sparse edits cut a frozen backbone's loss by 2.8%, a global-recurrence adapter does as well, and the learned controller is worse than no routing.",
191 + "pdf": "/papers/wp31_uqo.pdf",
192 + "repo": "https://git.spboucher.ai/wp31_uqo"
193 + },
194 + {
195 + "slug": "wp30",
196 + "kind": "wp",
197 + "num": "UQO Working Paper No. 30",
198 + "number": 30,
199 + "field": "ai",
200 + "title": "Beyond Text: Cache Compression and Knowledge Transfer in Multi-Agent Systems",
201 + "subtitle": "A Reproducible Study of Fidelity, Communication Cost, and Repeated Handoffs",
202 + "version": "1.0",
203 + "date": "October 2026",
204 + "year": 2026,
205 + "pages": "49 pages",
206 + "pageCount": 49,
207 + "authors": [
208 + "Simon-Pierre Boucher"
209 + ],
210 + "description": "Measures 22 ways of passing information between language-model agents (text, summaries, structured evidence, quantized and evicted KV caches) on Qwen2.5-3B and Llama-3.2-3B over 13,280 trials with measured bytes and latency: lossless representations tie in quality, and lossy ones are model- and regime-dependent.",
211 + "pdf": "/papers/wp30_uqo.pdf",
212 + "repo": "https://git.spboucher.ai/wp30_uqo"
213 + },
214 + {
215 + "slug": "wp29",
216 + "kind": "wp",
217 + "num": "UQO Working Paper No. 29",
218 + "number": 29,
219 + "field": "methods",
220 + "title": "Protocol-Guided Autonomous Research with Claude Code",
221 + "subtitle": "Scientific Verification, Reproducibility, and the Cost of Delegation: A Controlled Study",
222 + "version": "1.0",
223 + "date": "October 2026",
224 + "year": 2026,
225 + "pages": "110 pages",
226 + "pageCount": 110,
227 + "authors": [
228 + "Simon-Pierre Boucher"
229 + ],
230 + "description": "Four-arm controlled study of 288 live Claude Code runs (claude-opus-5-5) on 24 bounded scientific tasks scored by hidden deterministic oracles: verified completion is 0.903 under an open instruction and 0.917 under a full protocol with executable gates and review, a null +0.014 (95% interval [−0.028, +0.056]) at two to three times the cost.",
231 + "pdf": "/papers/wp29_uqo.pdf",
232 + "repo": "https://git.spboucher.ai/wp29_uqo"
233 + },
234 + {
235 + "slug": "wp28",
236 + "kind": "wp",
237 + "num": "UQO Working Paper No. 28",
238 + "number": 28,
239 + "field": "finance",
240 + "title": "Beyond One-Step Distributed Maximum Likelihood: Local Bias Cancellation, Global Synchronization, and the Price of Inexact Curvature",
241 + "subtitle": "A joint order–round frontier, its sharpness in an exact model, and measured communication costs",
242 + "version": "1.0",
243 + "date": "October 2026",
244 + "year": 2026,
245 + "pages": "56 pages",
246 + "pageCount": 56,
247 + "authors": [
248 + "Simon-Pierre Boucher"
249 + ],
250 + "description": "Distributed maximum likelihood with order-r split-sample jackknife debiasing on K workers followed by q synchronized Newton rounds: first-order equivalence to the centralized estimator survives far more workers than the classical square-root limit (a half-block jackknife plus one step tolerates K up to N to the 7/8), with sharpness in the exponential model, Monte Carlo and cluster benchmarks.",
251 + "pdf": "/papers/wp28_uqo.pdf",
252 + "repo": "https://git.spboucher.ai/wp28_uqo"
253 + },
254 + {
255 + "slug": "wp27",
256 + "kind": "wp",
257 + "num": "UQO Working Paper No. 27",
258 + "number": 27,
259 + "field": "finance",
260 + "title": "The Invisible Hand of Machines: Price Discovery in Markets Populated by LLM Agents",
261 + "version": "1.0",
262 + "date": "October 2026",
263 + "year": 2026,
264 + "pages": "101 pages",
265 + "pageCount": 101,
266 + "authors": [
267 + "Simon-Pierre Boucher"
268 + ],
269 + "description": "Randomized matched-seed experiments in an event-sourced limit-order-book simulator assign 0–100 percent of trader slots to a small reasoning model (gpt-5.4-nano): quoted spreads fall from 19 to 6 basis points and no bubble occurs in 112 live runs, but returns become predictable (lag-one autocorrelation 0.02 to 0.32).",
270 + "pdf": "/papers/wp27_uqo.pdf",
271 + "ssrn": "https://ssrn.com/abstract=7679801",
272 + "repo": "https://git.spboucher.ai/wp27_uqo"
273 + },
274 + {
275 + "slug": "wp26",
276 + "kind": "wp",
277 + "num": "UQO Working Paper No. 26",
278 + "number": 26,
279 + "field": "real-estate",
280 + "title": "Can Autonomous AI Agents Discover the Fundamental Value of Real Estate? Evidence from Quebec Transactions and Multi-Agent Deliberation",
281 + "version": "1.0",
282 + "date": "October 2026",
283 + "year": 2026,
284 + "pages": "129 pages",
285 + "pageCount": 129,
286 + "authors": [
287 + "Simon-Pierre Boucher"
288 + ],
289 + "description": "Preregistered experiment on 631,866 registered Quebec sales (2021–2026) testing whether debate among twelve role-specialised LLM valuation agents adds information: debate reaches a mean absolute log error of 0.158 against 0.159 for the same agents kept silent, and every preregistered contrast includes zero.",
290 + "pdf": "/papers/wp26_uqo.pdf",
291 + "repo": "https://git.spboucher.ai/wp26_uqo"
292 + },
293 + {
294 + "slug": "wp25",
295 + "kind": "wp",
296 + "num": "UQO Working Paper No. 25",
297 + "number": 25,
298 + "field": "finance",
299 + "title": "When AI Agents Become Economic Agents: An Agentic General Equilibrium Theory",
300 + "version": "1.0",
301 + "date": "October 2026",
302 + "year": 2026,
303 + "pages": "71 pages",
304 + "pageCount": 71,
305 + "authors": [
306 + "Simon-Pierre Boucher"
307 + ],
308 + "description": "Delegating decisions to agents built on shared foundation models turns idiosyncratic error into common model noise: closed-form loadings in the delegation share and number of models, a zero private–social wedge under constant returns, and order-flow correlation rising from 0.11 to 0.97 in an agentic market laboratory.",
309 + "pdf": "/papers/wp25_uqo.pdf",
310 + "repo": "https://git.spboucher.ai/wp25_uqo"
311 + },
312 + {
313 + "slug": "wp24",
314 + "kind": "wp",
315 + "num": "UQO Working Paper No. 24",
316 + "number": 24,
317 + "field": "finance",
318 + "title": "From Labor to Capital: Artificial General Intelligence, General Equilibrium, and Asset Valuation",
319 + "version": "1.0",
320 + "date": "October 2026",
321 + "year": 2026,
322 + "pages": "55 pages",
323 + "pageCount": 55,
324 + "authors": [
325 + "Simon-Pierre Boucher"
326 + ],
327 + "description": "A general-equilibrium extension of Restrepo (2025) in which compute is a priced, depreciating, energy-using capital good alongside structures and land; calibrated to the United States and solved to 2300, the labor share falls to 0.283 by 2050 while compute's market value never exceeds one fifth of GDP.",
328 + "pdf": "/papers/wp24_uqo.pdf",
329 + "ssrn": "https://ssrn.com/abstract=7677199",
330 + "repo": "https://git.spboucher.ai/wp24_uqo"
331 + },
332 + {
333 + "slug": "wp23",
334 + "kind": "wp",
335 + "num": "UQO Working Paper No. 23",
336 + "number": 23,
337 + "field": "real-estate",
338 + "title": "Beyond Hedonic Models: Spatiotemporal, Hierarchical and Probabilistic Architectures for Residential Real Estate Price Forecasting",
339 + "subtitle": "A pseudo-real-time evaluation on 632,000 Quebec transactions and 82 local market indices, 2021–2026",
340 + "version": "1.1",
341 + "date": "October 2026",
342 + "year": 2026,
343 + "pages": "56 pages",
344 + "pageCount": 56,
345 + "authors": [
346 + "Simon-Pierre Boucher"
347 + ],
348 + "description": "A strict pseudo-real-time test of five forecasting architectures and an integrated stack against 14 benchmarks on 632,006 Quebec sales (2021–2026): gradient boosting is the most accurate model for future sale prices at every horizon, and no architecture improves on momentum and autoregression for index changes.",
349 + "pdf": "/papers/wp23_uqo.pdf",
350 + "repo": "https://git.spboucher.ai/wp23_uqo"
351 + },
352 + {
353 + "slug": "wp22",
354 + "kind": "wp",
355 + "num": "UQO Working Paper No. 22",
356 + "number": 22,
357 + "field": "real-estate",
358 + "title": "SHARP-HPI: A Spatially Hierarchical, Adaptive and Robust Framework for Quality-Adjusted Housing Price Indices",
359 + "subtitle": "Monthly municipal indices for Quebec from 770,000 registered sales, 2021–2026",
360 + "version": "1.0",
361 + "date": "October 2026",
362 + "year": 2026,
363 + "pages": "49 pages",
364 + "pageCount": 49,
365 + "authors": [
366 + "Simon-Pierre Boucher"
367 + ],
368 + "description": "A two-stage hedonic and state-space index estimator with empirical-Bayes pooling across cells, applied to 769,995 registered Quebec sales (2021–2026); in Monte Carlo it reduces the RMSE of municipal monthly changes to 1.58 log points against 2.91 for per-cell state-space estimation.",
369 + "pdf": "/papers/wp22_uqo.pdf",
370 + "ssrn": "https://ssrn.com/abstract=7651200",
371 + "repo": "https://git.spboucher.ai/wp22_uqo"
372 + },
373 + {
374 + "slug": "wp21",
375 + "kind": "wp",
376 + "num": "UQO Working Paper No. 21",
377 + "number": 21,
378 + "field": "real-estate",
379 + "title": "The Global Evidence on Housing Price Determinants: A Meta-Analysis of Hedonic Pricing Models, 1970–2026",
380 + "version": "1.0",
381 + "date": "October 2026",
382 + "year": 2026,
383 + "pages": "51 pages",
384 + "pageCount": 51,
385 + "authors": [
386 + "Simon-Pierre Boucher"
387 + ],
388 + "description": "A PRISMA-style meta-analysis of the global hedonic housing literature, 1970–2026: 103,011 records screened to 242 eligible studies from 55 countries and 18,831 coefficient rows, giving a pooled living-area elasticity of 0.65 and heterogeneity beyond sampling error for every implicit price.",
389 + "pdf": "/papers/wp21_uqo.pdf",
390 + "ssrn": "https://ssrn.com/abstract=7646338",
391 + "repo": "https://git.spboucher.ai/wp21_uqo"
392 + },
393 + {
394 + "slug": "wp20",
395 + "kind": "wp",
396 + "num": "UQO Working Paper No. 20",
397 + "number": 20,
398 + "field": "real-estate",
399 + "title": "From Listing Images to Property Valuation: A Multimodal AI Framework for Automated Building Characterization and the Cost Approach",
400 + "subtitle": "Evidence from Québec single-family listings matched to the assessment roll",
401 + "version": "1.0",
402 + "date": "October 2026",
403 + "year": 2026,
404 + "pages": "66 pages",
405 + "pageCount": 66,
406 + "authors": [
407 + "Simon-Pierre Boucher"
408 + ],
409 + "description": "Vision–language models read 14,245 photographs and the text of 1,200 Québec house listings matched to the assessment roll to specify the improvements for a cost approach: the indication's ratio to asking price moves from 0.75 to 0.95, a modest gain next to a 29.9% unit-cost-anchor effect.",
410 + "pdf": "/papers/wp20_uqo.pdf",
411 + "ssrn": "https://ssrn.com/abstract=7651178",
412 + "repo": "https://git.spboucher.ai/wp20_uqo"
413 + },
414 + {
415 + "slug": "wp19",
416 + "kind": "wp",
417 + "num": "UQO Working Paper No. 19",
418 + "number": 19,
419 + "field": "real-estate",
420 + "title": "The Endogenous Housing Liquidity Premium: Search Frictions, Seller Urgency, and Price Discovery",
421 + "version": "1.0",
422 + "date": "October 2026",
423 + "year": 2026,
424 + "pages": "60 pages",
425 + "pageCount": 60,
426 + "authors": [
427 + "Simon-Pierre Boucher"
428 + ],
429 + "description": "A continuous-time search model with free entry of buyers, continuously distributed seller carrying costs and Nash bargaining derives the urgency discount, sale hazards, price dispersion and the cash–mortgage gap in eight propositions; in the illustrative baseline the most urgent seller concedes 5.6%. No data are used.",
430 + "pdf": "/papers/wp19_uqo.pdf",
431 + "ssrn": "https://ssrn.com/abstract=7651119",
432 + "repo": "https://git.spboucher.ai/wp19_uqo"
433 + },
434 + {
435 + "slug": "wp18",
436 + "kind": "wp",
437 + "num": "UQO Working Paper No. 18",
438 + "number": 18,
439 + "field": "real-estate",
440 + "title": "Housing, Leverage, and Intergenerational Wealth Inequality: A Dynamic General Equilibrium Theory",
441 + "version": "1.0",
442 + "date": "October 2026",
443 + "year": 2026,
444 + "pages": "65 pages",
445 + "pageCount": 65,
446 + "authors": [
447 + "Simon-Pierre Boucher"
448 + ],
449 + "description": "A closed-form three-stage benchmark and a twelve-period overlapping-generations model with nominal mortgages, gifts and estates: transfers create a 0.95 income-year owner–renter wealth gap among equal-income households, prices absorb 64–66% of the down-payment effect, and a 10% inflation surprise lowers the real house price by 0.45%.",
450 + "pdf": "/papers/wp18_uqo.pdf",
451 + "ssrn": "https://ssrn.com/abstract=7651140",
452 + "repo": "https://git.spboucher.ai/wp18_uqo"
453 + },
454 + {
455 + "slug": "wp16",
456 + "kind": "wp",
457 + "num": "UQO Working Paper No. 16",
458 + "number": 16,
459 + "field": "real-estate",
460 + "title": "Does the Sales Comparison Approach Actually Work? A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and Value Reconciliation",
461 + "subtitle": "Evidence from 391,000 Single-Family Transactions in Quebec, 2021–2026",
462 + "version": "1.1",
463 + "date": "October 2026",
464 + "year": 2026,
465 + "pages": "52 pages",
466 + "pageCount": 52,
467 + "authors": [
468 + "Simon-Pierre Boucher"
469 + ],
470 + "description": "A point-in-time backtest of 2,256 configurations of the sales comparison approach on 390,996 Quebec single-family sales: adjustments cut the five-comparable median error from 18.1% to 13.0%, and the best transparent rule reaches 11.1% out of sample against 9.0% for gradient boosting.",
471 + "pdf": "/papers/wp16_uqo.pdf",
472 + "ssrn": "https://ssrn.com/abstract=7626720",
473 + "repo": "https://git.spboucher.ai/wp16_uqo"
474 + },
475 + {
476 + "slug": "wp15",
477 + "kind": "wp",
478 + "num": "UQO Working Paper No. 15",
479 + "number": 15,
480 + "field": "finance",
481 + "title": "Faster, Wider, More Fragile? The Evolution of Intraday Shock Propagation in U.S. Equity Markets, 1992–2026",
482 + "version": "1.1",
483 + "date": "October 2026",
484 + "year": 2026,
485 + "pages": "43 pages",
486 + "pageCount": 43,
487 + "authors": [
488 + "Simon-Pierre Boucher"
489 + ],
490 + "description": "Using 7.69 billion one-minute bars on every listed and delisted U.S. stock from 1992 to 2026, isolated ±6σ shocks raise market-wide tail activity by at most about 2% of its base rate in every decade, while transmission to the twenty closest correlation neighbours roughly tripled.",
491 + "pdf": "/papers/wp15_uqo.pdf",
492 + "repo": "https://git.spboucher.ai/wp15_uqo"
493 + },
494 + {
495 + "slug": "wp14",
496 + "kind": "wp",
497 + "num": "UQO Working Paper No. 14",
498 + "number": 14,
499 + "field": "real-estate",
500 + "title": "Hedonic House Pricing with Language-Model and Vision Features: Evidence from Quebec Real-Estate Listings",
501 + "version": "2.1",
502 + "date": "October 2026",
503 + "year": 2026,
504 + "pages": "46 pages",
505 + "pageCount": 46,
506 + "authors": [
507 + "Simon-Pierre Boucher"
508 + ],
509 + "description": "A large language model and a vision–language model convert the descriptions and façade photographs of 19,123 Quebec house listings into schema-constrained hedonic regressors, raising in-sample R² from 0.717 to 0.755 (3.0 of the 3.8 points from the photograph) and out-of-sample R² for every estimator.",
510 + "pdf": "/papers/wp14_uqo.pdf",
511 + "ssrn": "https://ssrn.com/abstract=7651058",
512 + "repo": "https://git.spboucher.ai/wp14_uqo"
513 + },
514 + {
515 + "slug": "wp13",
516 + "kind": "wp",
517 + "num": "UQO Working Paper No. 13",
518 + "number": 13,
519 + "field": "finance",
520 + "title": "VQuant: An Agentic Large Language Model Architecture for Automated Quantitative Financial Analysis and Document Generation",
521 + "subtitle": "Reasoning-Augmented Tool Orchestration, In-Loop Numerical Computation, and Multi-Format Reporting",
522 + "version": "1.1",
523 + "date": "October 2026",
524 + "year": 2026,
525 + "pages": "41 pages",
526 + "pageCount": 41,
527 + "authors": [
528 + "Simon-Pierre Boucher"
529 + ],
530 + "description": "A design-science description of VQuant, a deployed platform in which a reasoning LLM orchestrates 245 schema-defined tools and a Python sandbox so that every number in a financial report comes from live data or auditable code; audited against the source code, with no empirical evaluation yet.",
531 + "pdf": "/papers/wp13_uqo.pdf",
532 + "ssrn": "https://ssrn.com/abstract=7650999",
533 + "repo": "https://git.spboucher.ai/wp13_uqo"
534 + },
535 + {
536 + "slug": "wp12",
537 + "kind": "wp",
538 + "num": "UQO Working Paper No. 12",
539 + "number": 12,
540 + "field": "finance",
541 + "title": "Forecasting Realized Volatility Across Asset Classes: HAR, GARCH, and Machine Learning in a Unified Multi-Asset Horse Race, 2013–2026",
542 + "version": "0.2",
543 + "date": "October 2026",
544 + "year": 2026,
545 + "pages": "50 pages",
546 + "pageCount": 50,
547 + "authors": [
548 + "Simon-Pierre Boucher"
549 + ],
550 + "description": "Twenty-seven realized-variance forecasts (HAR, GARCH, penalized regressions, tree ensembles, LSTM, Transformer, combinations) raced on 25 instruments from four asset classes over 2013–2026 under one protocol: log-linear models win, and ridge on an extended predictor set beats HAR on 24 of 25 assets (working draft).",
551 + "status": "Working draft",
552 + "pdf": "/papers/wp12_uqo.pdf",
553 + "repo": "https://git.spboucher.ai/wp12_uqo"
554 + },
555 + {
556 + "slug": "wp11",
557 + "kind": "wp",
558 + "num": "UQO Working Paper No. 11",
559 + "number": 11,
560 + "field": "real-estate",
561 + "title": "Half a Million Prices, Twenty Models",
562 + "subtitle": "A Systematic Assessment of Hedonic Specifications and Estimation Methods for the Quebec Housing Market, 2021–2026",
563 + "version": "1.1",
564 + "date": "October 2026",
565 + "year": 2026,
566 + "pages": "40 pages",
567 + "pageCount": 40,
568 + "authors": [
569 + "Simon-Pierre Boucher"
570 + ],
571 + "description": "A controlled horse race of twenty hedonic models on 514,212 roll-matched Quebec sales (2021–2026) under one retransformation rule: spatial controls move median error by ten points, functional form by two, and boosting's 3.2-point lead under random validation shrinks to an insignificant 0.5 point in a forward-in-time holdout.",
572 + "pdf": "/papers/wp11_uqo.pdf",
573 + "ssrn": "https://ssrn.com/abstract=7626698",
574 + "repo": "https://git.spboucher.ai/wp11_uqo"
575 + },
576 + {
577 + "slug": "wp10",
578 + "kind": "wp",
579 + "num": "UQO Working Paper No. 10",
580 + "number": 10,
581 + "field": "real-estate",
582 + "title": "The Assessment Gap in Quebec",
583 + "subtitle": "Vertical and Horizontal Inequity in Municipal Property Valuation: Evidence from 522,769 Sales Matched to the Assessment Roll",
584 + "version": "1.1",
585 + "date": "October 2026",
586 + "year": 2026,
587 + "pages": "45 pages",
588 + "pageCount": 45,
589 + "authors": [
590 + "Simon-Pierre Boucher"
591 + ],
592 + "description": "The first province-wide ratio study for a Canadian province matches 522,769 Quebec sales (2021–2026) to 625 municipal assessment rolls; the measured regressivity depends on outlier treatment, with a within-market elasticity of −0.34 under a lenient trim but −0.09 under the IAAO rule.",
593 + "pdf": "/papers/wp10_uqo.pdf",
594 + "ssrn": "https://ssrn.com/abstract=7651201",
595 + "repo": "https://git.spboucher.ai/wp10_uqo"
596 + },
597 + {
598 + "slug": "wp9",
599 + "kind": "wp",
600 + "num": "UQO Working Paper No. 9",
601 + "number": 9,
602 + "field": "real-estate",
603 + "title": "A Grand Hedonic Model of the Canadian Housing Market",
604 + "subtitle": "Decomposing the Value of Structure and Location across 142,121 MLS Listings with High-Dimensional Neighbourhood Fixed Effects",
605 + "version": "1.3",
606 + "date": "October 2026",
607 + "year": 2026,
608 + "pages": "52 pages",
609 + "pageCount": 52,
610 + "authors": [
611 + "Simon-Pierre Boucher"
612 + ],
613 + "description": "A national semi-log hedonic model on 142,121 Canadian MLS listings with 1,155 absorbed neighbourhood fixed effects raises explained log-price variance from 44% to 78%; a Shapley–Owen decomposition attributes about 34 points to structure and 36 to location, and out-of-sample MdAPE is 15.6%.",
614 + "pdf": "/papers/wp9_uqo.pdf",
615 + "ssrn": "https://ssrn.com/abstract=7626678",
616 + "repo": "https://git.spboucher.ai/wp9_uqo"
617 + },
618 + {
619 + "slug": "wp8",
620 + "kind": "wp",
621 + "num": "UQO Working Paper No. 8",
622 + "number": 8,
623 + "field": "finance",
624 + "title": "Prediction Markets as Arrow–Debreu Securities: Evidence from Polymarket",
625 + "subtitle": "State Prices, No-Arbitrage Bounds and Macroeconomic News in Blockchain-Based Contingent Claims",
626 + "version": "1.1",
627 + "date": "October 2026",
628 + "year": 2026,
629 + "pages": "44 pages",
630 + "pageCount": 44,
631 + "authors": [
632 + "Simon-Pierre Boucher"
633 + ],
634 + "description": "Polymarket negative-risk markets on the June 2026 FOMC decision (146 daily closes) and the 2024 presidential election (304 closes) are read as Arrow–Debreu state prices: the complete set averages 1.017 and 1.015, within executable bounds, with no daily lead–lag against Treasuries, equities, the VIX or Bitcoin.",
635 + "pdf": "/papers/wp8_uqo.pdf",
636 + "ssrn": "https://ssrn.com/abstract=7651118",
637 + "repo": "https://git.spboucher.ai/wp8_uqo"
638 + },
639 + {
640 + "slug": "wp7",
641 + "kind": "wp",
642 + "num": "UQO Working Paper No. 7",
643 + "number": 7,
644 + "field": "finance",
645 + "title": "The Options-Implied Information Content for Cross-Asset Return and Volatility Prediction: Evidence from 3.8 Billion Option Contracts",
646 + "version": "1.2",
647 + "date": "October 2026",
648 + "year": 2026,
649 + "pages": "53 pages",
650 + "pageCount": 53,
651 + "authors": [
652 + "Simon-Pierre Boucher"
653 + ],
654 + "description": "On a 2010–2025 panel of 69 liquid US underlyings built from 3.83 billion option records, the implied volatility surface explains only 0.3% of weekly stock returns but raises the HAR-RV R² for weekly realized variance from 0.421 to 0.515; the earlier weekly return results are withdrawn.",
655 + "pdf": "/papers/wp7_uqo.pdf",
656 + "ssrn": "https://ssrn.com/abstract=7626643",
657 + "repo": "https://git.spboucher.ai/wp7_uqo"
658 + },
659 + {
660 + "slug": "wp6",
661 + "kind": "wp",
662 + "num": "UQO Working Paper No. 6",
663 + "number": 6,
664 + "field": "finance",
665 + "title": "Intraday Volatility Transmission Across Asset Classes: A Connectedness Analysis of Equities, Foreign Exchange, Commodities, and Cryptocurrencies",
666 + "version": "1.1",
667 + "date": "October 2026",
668 + "year": 2026,
669 + "pages": "48 pages",
670 + "pageCount": 48,
671 + "authors": [
672 + "Simon-Pierre Boucher"
673 + ],
674 + "description": "Diebold–Yilmaz connectedness of daily realized volatility built from 5-minute bars for ten instruments on 2,514 common trading days (2016–2026): cross-asset shocks explain 54.8% of forecast-error variance, U.S. equity futures are the only robust net transmitters, and bitcoin becomes a net receiver in the spot-ETF era.",
675 + "pdf": "/papers/wp6_uqo.pdf",
676 + "ssrn": "https://ssrn.com/abstract=7651038",
677 + "repo": "https://git.spboucher.ai/wp6_uqo"
678 + },
679 + {
680 + "slug": "wp5",
681 + "kind": "wp",
682 + "num": "UQO Working Paper No. 5",
683 + "number": 5,
684 + "field": "real-estate",
685 + "title": "Airbnb, Residential Rents, and Housing Market Pressure: A Hedonic and Spatial Econometric Analysis",
686 + "version": "1.1",
687 + "date": "October 2026",
688 + "year": 2026,
689 + "pages": "59 pages",
690 + "pageCount": 59,
691 + "authors": [
692 + "Simon-Pierre Boucher"
693 + ],
694 + "description": "Hedonic rent equations on 8,303 Quebec rental listings and 3,456 Airbnb listings (2026) show each Airbnb within 500 m associated with 0.4% higher rent under city fixed effects, but only 0.08% and insignificant under spatial inference once postal-code neighbourhood fixed effects are added.",
695 + "pdf": "/papers/wp5_uqo.pdf",
696 + "ssrn": "https://ssrn.com/abstract=7626658",
697 + "repo": "https://git.spboucher.ai/wp5_uqo"
698 + },
699 + {
700 + "slug": "wp4",
701 + "kind": "wp",
702 + "num": "UQO Working Paper No. 4",
703 + "number": 4,
704 + "field": "real-estate",
705 + "title": "Hedonic Pricing of Short-Term Rental Properties: A Multi-Method Analysis of Airbnb Listing Prices in Quebec",
706 + "version": "1.1",
707 + "date": "October 2026",
708 + "year": 2026,
709 + "pages": "39 pages",
710 + "pageCount": 39,
711 + "authors": [
712 + "Simon-Pierre Boucher"
713 + ],
714 + "description": "Hedonic, quantile, spatial and machine-learning models of Quebec Airbnb nightly prices from a scrape of 5,000 listing cards, after a two-regime mixture removes the 28% of displayed prices that are multi-night stay totals; a city-fixed-effects model explains 51% of log-price variance.",
715 + "pdf": "/papers/wp4_uqo.pdf",
716 + "ssrn": "https://ssrn.com/abstract=7651018",
717 + "repo": "https://git.spboucher.ai/wp4_uqo"
718 + },
719 + {
720 + "slug": "wp3",
721 + "kind": "wp",
722 + "num": "UQO Working Paper No. 3",
723 + "number": 3,
724 + "field": "real-estate",
725 + "title": "Hedonic Housing Price Models for the United States: A Multi-Method Comparison of Parametric, Quantile, and Machine Learning Approaches",
726 + "version": "1.2",
727 + "date": "October 2026",
728 + "year": 2026,
729 + "pages": "82 pages",
730 + "pageCount": 82,
731 + "authors": [
732 + "Simon-Pierre Boucher"
733 + ],
734 + "description": "OLS, quantile regression and gradient boosting are compared on 788,842 Zillow listings across all 50 states; XGBoost reaches R² = 0.833 under random validation but 0.425 when ten states are held out, and OLS is as accurate out of state (0.539) once corrupt records are removed.",
735 + "pdf": "/papers/wp3_uqo.pdf",
736 + "ssrn": "https://ssrn.com/abstract=7639718",
737 + "repo": "https://git.spboucher.ai/wp3_uqo"
738 + },
739 + {
740 + "slug": "wp2",
741 + "kind": "wp",
742 + "num": "UQO Working Paper No. 2",
743 + "number": 2,
744 + "field": "real-estate",
745 + "title": "Decoding Real Estate Descriptions: Semantic Embeddings and Hedonic Pricing of Residential Properties in Quebec",
746 + "version": "1.2",
747 + "date": "October 2026",
748 + "year": 2026,
749 + "pages": "61 pages",
750 + "pageCount": 61,
751 + "authors": [
752 + "Simon-Pierre Boucher"
753 + ],
754 + "description": "Sentence embeddings of 17,087 Quebec single-family listings are projected onto 20 named reference descriptions and added to a hedonic model; adjusted R² rises from 0.452 to 0.511, but net of listing language and 415 area fixed effects the semantic block adds only 2.0 percentage points.",
755 + "pdf": "/papers/wp2_uqo.pdf",
756 + "ssrn": "https://ssrn.com/abstract=7626640",
757 + "repo": "https://git.spboucher.ai/wp2_uqo"
758 + },
759 + {
760 + "slug": "wp1",
761 + "kind": "wp",
762 + "num": "UQO Working Paper No. 1",
763 + "number": 1,
764 + "field": "real-estate",
765 + "title": "Heterogeneous Amenity Capitalization in the Quebec Residential Market: A Spatial Quantile Regression Approach",
766 + "version": "1.1",
767 + "date": "October 2026",
768 + "year": 2026,
769 + "pages": "58 pages",
770 + "pageCount": 58,
771 + "authors": [
772 + "Simon-Pierre Boucher"
773 + ],
774 + "description": "Quantile and two-stage spatial quantile regressions on 42,481 Centris listings (January 2026) show that, for houses, the highway capitalisation differential falls from 0.205 at the 10th percentile to 0.034 at the 90th, and a spatial lag attenuates median differentials by a third to a half.",
775 + "pdf": "/papers/wp1_uqo.pdf",
776 + "ssrn": "https://ssrn.com/abstract=7639660",
777 + "repo": "https://git.spboucher.ai/wp1_uqo"
778 + },
779 + {
780 + "slug": "firc2",
781 + "kind": "firc",
782 + "num": "UQO Working Paper No. 0002 (FIRC)",
783 + "number": 2,
784 + "field": "finance",
785 + "title": "Does the Language of Artificial Intelligence Move Markets? A Semantic Analysis of Russell 3000 SEC Filings and the Cross-Section of Stock Returns, 2010–2026",
786 + "version": "2.0",
787 + "date": "October 2026",
788 + "year": 2026,
789 + "pages": "48 pages",
790 + "pageCount": 48,
791 + "authors": [
792 + "Manel Kammoun",
793 + "Djerrycharli Tandjambianda",
794 + "Simon-Pierre Boucher"
795 + ],
796 + "description": "Scores AI language in 527,548 SEC filings of about 2,500 Russell 3000 firms with sentence embeddings and links it to forward returns over 27,468 firm-years: a modest persistent premium (+5.5 pp per year, t = 2.41); the post-ChatGPT jump seen in the S&P 500 is a large-cap re-rating, not a market-wide regime change.",
797 + "pdf": "/papers/firc2_uqo.pdf",
798 + "repo": "https://git.spboucher.ai/wp17_uqo",
799 + "funding": "Fonds institutionnel de recherche et de création (FIRC), UQO"
800 + },
801 + {
802 + "slug": "firc1",
803 + "kind": "firc",
804 + "num": "UQO Working Paper No. 0001 (FIRC)",
805 + "number": 1,
806 + "field": "finance",
807 + "title": "From Corporate Disclosures to a Russell 3000 Strategic Project Database: Identification, Measurement, Validation, and Research Applications",
808 + "version": "1.0",
809 + "date": "October 2026",
810 + "year": 2026,
811 + "pages": "39 pages",
812 + "pageCount": 39,
813 + "authors": [
814 + "Manel Kammoun",
815 + "Djerrycharli Tandjambianda",
816 + "Simon-Pierre Boucher"
817 + ],
818 + "description": "Documents SPID, a project-level database built from 527,548 SEC filings (10-K, 10-Q, 8-K, 2010–2026) of 2,564 Russell 3000 issuers: a schema-constrained LLM extractor yields 308,065 evidence-linked project mentions resolved into 121,099 candidate projects, with a validation framework stated but not yet executed.",
819 + "pdf": "/papers/firc1_uqo.pdf",
820 + "funding": "Fonds institutionnel de recherche et de création (FIRC), UQO"
821 + }
822 + ],
823 + "details": [
824 + {
825 + "slug": "wp38",
826 + "repo": "https://git.spboucher.ai/wp38_uqo",
827 + "hero": {
828 + "headline": "Regional specialization in Quebec's online retail is a property of merchants, not of listing volumes",
829 + "subheadline": "On 390,108 listings of 1,569 stores, the region–category dependence beats 2,000 merchant reshuffles under merchant-equal weights (Cramér's V 0.112 vs 0.086) but is indistinguishable from chance under listing weights; catalogues are so concentrated (Gini 0.85) that the preregistered joint criterion fails and the complexity index is withheld."
830 + },
831 + "abstract": [
832 + "Can product-level online catalogues reveal the spatial organisation, diversity and specialization of a regional retail economy beyond what business counts show? This paper studies one read-only snapshot (1 November 2026) of Fabri-Ka, an aggregator of the public catalogues of 1,569 Quebec-based online stores (390,108 published listings, 17 administrative regions), with a preregistered design in which every regional statistic is confronted with a null that reallocates merchants rather than listings, and with three adversarial referee rounds whose corrections are reported.",
833 + "After cleaning (370,054 listings, 1,557 stores) and recovering merchant locations from official geography, regional specialization turns out to be a property of merchants, not of listing volumes: under merchant-equal weights the region–category dependence exceeds 2,000 merchant reshuffles (Cramér's V 0.112 vs. 0.086, p < 0.001) in every sample, labelling and discovery-source variant, at the level of regions and of MRCs, whereas under listing weights it is indistinguishable from chance (p = 0.20; 0.43 after trimming the largest one per cent of stores), a low-power comparison because catalogue sizes are extremely skewed (Gini 0.85; the top decile of stores holds 79% of listings). By the preregistered joint criterion H1 is nevertheless not supported. Total regional variety rises with regional size, but the breadth of a typical set of fifteen merchants shows no detectable association with population, business counts, income or density across sixteen regions.",
834 + "On 1,064 matched products sold by several merchants (matching precision 0.98–1.00), posted prices differ by a median of 4.7 log points; resellers post prices 11.6 log points below producers' own shops (p = 0.004), while the joint test of regional price levels does not reject equality once the wild cluster bootstrap is implemented correctly (p = 0.098) and rests on a single region. An ECI-type complexity indicator fails its preregistered bootstrap validation (median rank correlation 0.68 on a 17 × 20 matrix) and is withheld; the index family is published as separately validated components only. Measurement is central: the platform's keyword category labels reach 46% precision against annotator consensus and are replaced, by a rule fixed in advance, with a classifier validated at 88% on a held-out audit; a fifth of listings have no recoverable location; coverage per thousand business locations varies six-fold across regions. A local-origin price premium is untestable in this source. Code, audits, decision log, amendments and referee reports are released."
835 + ],
836 + "findings": [
837 + {
838 + "value": "390,108",
839 + "label": "listings of 1,569 Quebec online stores, 17 regions (370,054 / 1,557 after cleaning)"
840 + },
841 + {
842 + "value": "0.112 vs 0.086",
843 + "label": "Cramér's V of the region–category dependence against 2,000 merchant reshuffles (p < 0.001)"
844 + },
845 + {
846 + "value": "p = 0.20",
847 + "label": "the same dependence under listing weights: indistinguishable from chance"
848 + },
849 + {
850 + "value": "Gini 0.85",
851 + "label": "concentration of catalogue sizes; the top decile of stores holds 79% of listings"
852 + },
853 + {
854 + "value": "−11.6 log pts",
855 + "label": "reseller prices below producers' own shops on 1,064 matched products (p = 0.004)"
856 + },
857 + {
858 + "value": "46% → 88%",
859 + "label": "category-label precision: platform keywords vs the pre-specified replacement classifier"
860 + }
861 + ],
862 + "contributions": [
863 + {
864 + "title": "A merchant-preserving null for regional statistics",
865 + "description": "Every regional indicator is compared with a reallocation of merchants rather than listings, which separates the location of large catalogues from genuine specialization; under this null the merchant-level signal survives every variant and the listing-level signal vanishes."
866 + },
867 + {
868 + "title": "Preregistered verdicts, hypothesis by hypothesis",
869 + "description": "H1 (specialization) not supported by its joint criterion, H2 (diversity and scale) inconclusive, H3 (concentration) supported, H4 (related diversification) exploratory, H5 split (channel supported, geography not), H6 untestable, H7 (complexity) not validated."
870 + },
871 + {
872 + "title": "Price dispersion across channels, not regions",
873 + "description": "Resellers post 11.6 log points below producers on identical items while the joint test of regional price levels fails to reject equality once the wild cluster bootstrap is corrected and the influence of a single region is exposed."
874 + },
875 + {
876 + "title": "Measurement defects quantified and corrected",
877 + "description": "Platform category labels (46% precision) replaced by a validated classifier (88%), a fifth of listings without recoverable location, six-fold variation in coverage per thousand business locations, and regional labels carrying a pre-2021 map, each measured and carried into the conclusions."
878 + },
879 + {
880 + "title": "Withdrawn first-draft findings reported as such",
881 + "description": "A geographically structured price level and a power-law tail did not survive adversarial refereeing of the code and are reported as withdrawn, which is what the preregistered, auditable pipeline exists for."
882 + }
883 + ],
884 + "data": [
885 + {
886 + "title": "Fabri-Ka catalogue snapshot",
887 + "description": "Owner-authorised, read-only copy of the aggregator database taken with sqlite3 backup on 2026-11-01 (SHA-256 recorded): 390,108 published listings of 1,569 Quebec-based online stores; cleaned to 370,054 listings and 1,557 stores; not redistributed because catalogues belong to the merchants."
888 + },
889 + {
890 + "title": "Official geography and business counts",
891 + "description": "Merchant locations recovered from official administrative geography (17 regions, MRCs); population, business counts, income and density per region from public statistics; discovery runs through regional directories, which drives the six-fold variation in coverage."
892 + },
893 + {
894 + "title": "Matched-product price sample",
895 + "description": "1,064 products sold by several merchants, matched with precision 0.98–1.00, used for the channel and regional price tests; an annotator-consensus audit of category labels and a held-out validation of the replacement classifier."
896 + }
897 + ],
898 + "methodology": [
899 + {
900 + "title": "Preregistration with amendments",
901 + "description": "Seven hypotheses with joint decision criteria fixed before analysis; a decision log records amendments; three adversarial referee rounds over the code with corrections reported."
902 + },
903 + {
904 + "title": "Merchant-reshuffling permutation tests",
905 + "description": "Region–category dependence (Cramér's V) under merchant-equal and listing weights against 2,000 reallocations of merchants, at region and MRC level, across sample, labelling and discovery-source variants."
906 + },
907 + {
908 + "title": "Diversity, concentration and relatedness",
909 + "description": "Total regional variety versus rarefied breadth of a typical fifteen-merchant set; Gini and tail analysis of catalogue sizes; store-level relatedness density and category entry over ten weeks."
910 + },
911 + {
912 + "title": "Price tests with wild cluster bootstrap",
913 + "description": "Channel (reseller vs producer) and regional price-level tests on matched products with correctly implemented wild cluster bootstrap and single-region influence checks."
914 + },
915 + {
916 + "title": "Complexity indicator validation",
917 + "description": "ECI-type ranking subjected to a preregistered bootstrap over merchants on a 17 × 20 matrix; withheld when the median rank correlation (0.68) fails the threshold."
918 + }
919 + ],
920 + "reproducibility": [
921 + "Deterministic pipeline (seed 20261101) from a single Makefile: ingest, clean, match, validate, features, models, robust, tables, figures, paper; make test runs invariant tests on the cleaned panel",
922 + "Raw snapshot not redistributed; data dictionary, sample-flow table, every experiment summary, decision log, amendments and referee reports released",
923 + "Category classifier, annotation audit and matching precision reported against held-out samples",
924 + "Full LaTeX source of the 51-page paper"
925 + ],
926 + "keywords": [
927 + "economic geography",
928 + "online retail",
929 + "product assortment",
930 + "revealed comparative advantage",
931 + "diversity",
932 + "economic complexity",
933 + "price dispersion",
934 + "web-scraped data",
935 + "Quebec"
936 + ],
937 + "jel": "R12, L81, O18, C81, D22, R32"
938 + },
939 + {
940 + "slug": "wp37",
941 + "repo": "https://git.spboucher.ai/wp37_uqo",
942 + "hero": {
943 + "headline": "Grocery prices are not one inflation process: banners differ by 10–25 log points on identical products, and promotions are half of all price changes",
944 + "subheadline": "From 300,613 posted-price events in 54 Quebec feeds, the paper separates persistent banner gaps, deep but rare promotions that return exactly to their reference price, store-mixture artefacts in scraped storefronts, and matched-model from naive indices; a five-component Quebec Grocery Price and Affordability Index is published with uncertainty bands."
945 + },
946 + "abstract": [
947 + "Grocery prices are not a single inflation process. Using a read-only research copy of Food-Ka, a Quebec grocery price aggregator, the paper assembles an auditable panel of 124,579 listings from 54 active source feeds (26 retail companies, two channels: online storefronts and weekly circulars) observed every seven hours between 12 August and 1 November 2026 (300,613 logged price events, 723,140 listing-weeks). A first, measurement finding applies to any study of scraped prices: storefronts that serve store-specific prices to anonymous sessions produce intra-day oscillation among three to five price levels, so sub-weekly volatility in such feeds is a mixture over unidentified stores, not repricing.",
948 + "A calibrated matching procedure (exact UPC within Metro Inc.; brand, package and token-identical names elsewhere; precision 0.93 and recall 0.98 on 4,413 UPC-identified pairs, plus a 240-pair reviewed sample) yields 5,909 cross-banner groups. On identical products in the same week, the median max–min log price gap across banners is 0.154 (79% of group-weeks exceed 5%), banner-pair gaps are persistent (AR(1) 0.66 against a 0.00 permutation placebo), and banner fixed effects range from −0.20 (Walmart, Giant Tiger, Maxi) to +0.12 (specialty shops) relative to Metro. Inferred promotions cover 4.6% of storefront listing-weeks with a median depth of 22%; half of all weekly price changes are promotion entries or exits, doubling measured volatility (ratio 1.92, CI 1.84–2.02). Temporary sales return exactly to their pre-sale reference (mean change −0.0003 log points): no reference-price drift within the window; a pre-sale price elevation is mostly a store-mixture artefact and shrinks to about two log points in single-store shops.",
949 + "Lead–lag price associations between rivals are small and fail a lead placebo for cross-company pairs, so no causal response is claimed. Matched-model and naive indices agree within one point at the market level over eleven weeks, but naive averages are off by 10–30 points for banners with assortment turnover. A fixed-basket mixed-integer program shows that unconstrained cheapest-banner shopping saves 15% relative to the best single banner, 85% of which two banners capture; store geography and travel costs are unobserved. The paper publishes a five-component Quebec Grocery Price and Affordability Index with uncertainty bands and a prospective collection plan, and reports null and failed tests alongside positive ones."
950 + ],
951 + "findings": [
952 + {
953 + "value": "300,613",
954 + "label": "posted-price events on 124,579 listings from 54 feeds, 26 companies, 12 Aug–1 Nov 2026"
955 + },
956 + {
957 + "value": "0.154",
958 + "label": "median max–min log price gap across banners on identical products (79% of group-weeks above 5%)"
959 + },
960 + {
961 + "value": "−0.20 to +0.12",
962 + "label": "banner fixed effects relative to Metro (Walmart, Giant Tiger, Maxi vs specialty shops)"
963 + },
964 + {
965 + "value": "×1.92",
966 + "label": "measured volatility with promotions vs without (CI 1.84–2.02); promotions are half of weekly price changes"
967 + },
968 + {
969 + "value": "−0.0003",
970 + "label": "mean log change after a temporary sale: prices return exactly to their reference"
971 + },
972 + {
973 + "value": "15%",
974 + "label": "saving from unconstrained cheapest-banner shopping, 85% captured by two banners"
975 + }
976 + ],
977 + "contributions": [
978 + {
979 + "title": "A measurement warning for scraped storefront prices",
980 + "description": "Storefronts that serve store-specific prices to anonymous sessions oscillate among three to five levels within a day; sub-weekly volatility in such feeds measures cross-store dispersion, not repricing, and the pre-sale price elevation seen in mixed feeds shrinks to about two log points in single-store shops."
981 + },
982 + {
983 + "title": "Persistent banner architecture on identical products",
984 + "description": "5,909 cross-banner groups matched at precision 0.93 / recall 0.98 show a median 0.154 log gap, banner-pair gaps with AR(1) 0.66 against a 0.00 placebo, and fixed effects from −0.20 to +0.12 relative to Metro, mostly inside chains whose discount and conventional formats carry the same UPCs."
985 + },
986 + {
987 + "title": "Promotions quantified",
988 + "description": "Rare in catalogues (4.6% of storefront listing-weeks) but deep (median 22%), recurring every few weeks, half of all posted-price changes, doubling measured volatility and returning exactly to the reference price within the window."
989 + },
990 + {
991 + "title": "No evidence of rival response",
992 + "description": "Lead–lag associations between rivals are small and fail a lead placebo for cross-company pairs; visible co-movement is the calendar of a chain, not a reaction."
993 + },
994 + {
995 + "title": "Index design and a shopping-cost bound",
996 + "description": "Matched-model and naive indices agree at market level but diverge by 10–30 points for banners with assortment turnover; a fixed-basket mixed-integer program bounds the saving from multi-banner shopping; a five-component Quebec Grocery Price and Affordability Index with uncertainty bands is published with a prospective, store-pinned collection design."
997 + }
998 + ],
999 + "data": [
1000 + {
1001 + "title": "Food-Ka research copy",
1002 + "description": "Read-only export of the author's grocery price aggregator: 124,579 listings from 54 active feeds (26 companies; online storefronts and weekly circulars) observed every seven hours, 300,613 price events and 723,140 listing-weeks; not redistributed, available to referees under a data-use agreement with the file's SHA-256."
1003 + },
1004 + {
1005 + "title": "Cross-banner product matches",
1006 + "description": "Tiered matching (exact UPC within Metro Inc.; brand, package and token-identical names elsewhere) calibrated on 4,413 UPC-identified pairs and a 240-pair reviewed sample, with an auxiliary LLM second annotation (four runs, 1.58 USD) whose labels are stored so the pipeline runs without API access."
1007 + },
1008 + {
1009 + "title": "Statistics Canada reference series",
1010 + "description": "CPI tables 18-10-0004-01 and 18-10-0245-01 from the public Web Data Service, raw JSON kept with download timestamps, used to compare matched-model and naive indices."
1011 + }
1012 + ],
1013 + "methodology": [
1014 + {
1015 + "title": "Panel reconstruction and unit normalisation",
1016 + "description": "Ingestion to Parquet with checksum verification, unit normalisation, retailer hierarchy, daily and weekly panel reconstruction in DuckDB, with fifteen tests on parsing, panel invariants and matching rules."
1017 + },
1018 + {
1019 + "title": "Dispersion and persistence models",
1020 + "description": "Within-group max–min log gaps, banner-pair gap persistence (AR(1)) against a permutation placebo, and banner fixed effects with Metro as reference."
1021 + },
1022 + {
1023 + "title": "Promotion inference and event analysis",
1024 + "description": "Promotion entries and exits inferred from listing histories, depth and recurrence measured, post-sale return to reference tested, and the store-mixture artefact isolated on single-store shops."
1025 + },
1026 + {
1027 + "title": "Lead–lag tests with placebos",
1028 + "description": "Cross-banner lead–lag price associations with a lead placebo for cross-company pairs; no causal claim where the placebo is not passed."
1029 + },
1030 + {
1031 + "title": "Indices and basket optimisation",
1032 + "description": "Matched-model vs naive indices over eleven weeks; a fixed-basket mixed-integer program for cheapest-banner shopping; the QGPAI index with bootstrap uncertainty bands."
1033 + }
1034 + ],
1035 + "reproducibility": [
1036 + "Python 3.12 pipeline with pinned dependencies runs from the raw snapshot to the compiled paper with make; every stochastic step uses seed 20261101",
1037 + "Each stage writes a manifest with SHA-256 hashes and a line in a structured run ledger; all numbers in text and tables are generated from JSON model summaries",
1038 + "Preregistration and basket hashed before estimation; the decision log records three amendments",
1039 + "Prospective collection design (store pinning, regular-price logging, circulars by postal code) specified but not implemented"
1040 + ],
1041 + "keywords": [
1042 + "grocery prices",
1043 + "price dispersion",
1044 + "promotions",
1045 + "high–low pricing",
1046 + "web-scraped prices",
1047 + "price indices",
1048 + "record linkage",
1049 + "retail competition",
1050 + "Quebec"
1051 + ],
1052 + "jel": "D22, D40, E31, L11, L81, C81"
1053 + },
1054 + {
1055 + "slug": "wp36",
1056 + "repo": "https://git.spboucher.ai/wp36_uqo",
1057 + "hero": {
1058 + "headline": "One used-car listing in five is marked down, the hazard doubles over three weeks, and neighbours reprice on the same day without reacting to each other",
1059 + "subheadline": "A daily panel of 273,265 Quebec dealer listings observed seven times a day for eleven weeks: nonlinear depreciation, median 2.8% markdowns on overpriced cars, regional premia within 4% of Montréal, exits that overstate sales by at least a tenth, and a five-component Automobile Market Liquidity Index that captures 56% of the common variation in regional turnover."
1060 + },
1061 + "abstract": [
1062 + "The paper studies price discovery and liquidity in the Quebec used-car market with a daily panel of 273,265 dealer listings observed by the aggregator Auto-Ka between 12 August and 1 November 2026, with every asking-price change and listing disappearance recorded about seven times a day. Hedonic models with flexible age and mileage splines reject linear depreciation and improve dealer-held-out fit by three percent; profiles differ by powertrain. One listing in five receives a sustained markdown (median 2.8 percent of the entry price); the markdown hazard doubles over the first three weeks on the lot, and vehicles that entered above their quality-adjusted value or above their local competitors are cut sooner and more deeply.",
1063 + "A competitor event study with market-by-week and model-year-by-week fixed effects finds a same-day co-movement of price cuts among close local competitors (0.9 percentage points against a daily base rate of 1.4 percent) but no lagged response, and a within-make–model placebo yields nothing: this is interpreted as common shocks and synchronised repricing calendars, not strategic reaction. Quality-adjusted regional premia across Quebec's seventeen regions are economically small (all within four percent of Montréal) and jointly insignificant once trim is held fixed (p = 0.53), although coarser quality controls detect small but significant differences.",
1064 + "Listing exit is strongly predicted by mileage, price and listing age; cross-source matching by vehicle identification number shows that 94 percent of exits coincide within a week on two independent sources while at least 10 percent are followed by a relisting, so exit overstates sales. A transparent five-component Automobile Market Liquidity Index captures 56 percent of the common variation in regional turnover and, built without the exit component, predicts next-week exit intensity within regions under z-score standardisation, a result that weakens under rank standardisation; deep markdowns load as a symptom of illiquidity. No external forecasting claim is possible with an eleven-week window, and the paper documents what listing data can and cannot say about transactions."
1065 + ],
1066 + "findings": [
1067 + {
1068 + "value": "273,265",
1069 + "label": "dealer listings observed about seven times a day, 12 Aug–1 Nov 2026"
1070 + },
1071 + {
1072 + "value": "1 in 5",
1073 + "label": "listings receive a sustained markdown, median 2.8% of the entry price"
1074 + },
1075 + {
1076 + "value": "×2",
1077 + "label": "markdown hazard over the first three weeks on the lot"
1078 + },
1079 + {
1080 + "value": "0.9 pts",
1081 + "label": "same-day co-movement of cuts among close competitors (base rate 1.4%), no lagged response"
1082 + },
1083 + {
1084 + "value": "< 4%",
1085 + "label": "quality-adjusted regional premia relative to Montréal, jointly insignificant with trim fixed (p = 0.53)"
1086 + },
1087 + {
1088 + "value": "56%",
1089 + "label": "of the common variation in regional turnover captured by the five-component liquidity index"
1090 + }
1091 + ],
1092 + "contributions": [
1093 + {
1094 + "title": "Nonlinear depreciation by powertrain",
1095 + "description": "Hedonic models with flexible age and mileage splines reject linear depreciation, improve dealer-held-out fit by three percent, and show profiles that differ across powertrains."
1096 + },
1097 + {
1098 + "title": "Markdown dynamics on the lot",
1099 + "description": "A discrete-time hazard that doubles over the first three weeks; vehicles priced above their quality-adjusted value or above local competitors are cut sooner and more deeply, with the thirty-day predictability of repricing mostly reflecting dealer practice."
1100 + },
1101 + {
1102 + "title": "Co-movement without strategic reaction",
1103 + "description": "A competitor event study with market-by-week and model-year-by-week fixed effects finds same-day co-movement and weekly harmonics but no lagged response, and a within-make–model placebo yields nothing."
1104 + },
1105 + {
1106 + "title": "What exits do and do not measure",
1107 + "description": "VIN matching across two independent sources shows 94% of exits coincide within a week while at least 10% are followed by a relisting, so exit overstates retail sales; the median listing disappears after nine days."
1108 + },
1109 + {
1110 + "title": "A transparent liquidity index",
1111 + "description": "Five components summarise more than half of the common variation in regional turnover; the non-exit components anticipate next week's exits under the preregistered z-score standardisation, less clearly under rank standardisation; deep markdowns load as a symptom of illiquidity."
1112 + }
1113 + ],
1114 + "data": [
1115 + {
1116 + "title": "Auto-Ka daily listing panel",
1117 + "description": "Every listing published by 131 dealer websites, two full-inventory portals and five dealer groups in Quebec, observed about seven times a day between 12 August and 1 November 2026, with each asking-price change and disappearance logged; immutable snapshot with an exclusion log."
1118 + },
1119 + {
1120 + "title": "Cross-source VIN matches",
1121 + "description": "Vehicle identification numbers matched across two independent sources to date exits and detect relistings; 94% of exits coincide within a week, at least 10% are relisted."
1122 + },
1123 + {
1124 + "title": "Regional and trim controls",
1125 + "description": "Seventeen administrative regions, make–model–trim and model-year identifiers used for quality adjustment, competitor sets and placebo tests."
1126 + }
1127 + ],
1128 + "methodology": [
1129 + {
1130 + "title": "Hedonic splines",
1131 + "description": "Flexible age and mileage splines by powertrain, evaluated on dealer-held-out samples against linear depreciation."
1132 + },
1133 + {
1134 + "title": "Discrete-time markdown and exit hazards",
1135 + "description": "Hazards of sustained markdown and of listing exit as functions of lot age, mileage, price and mispricing relative to quality-adjusted value and local competitors."
1136 + },
1137 + {
1138 + "title": "Competitor event study with placebos",
1139 + "description": "Market-by-week and model-year-by-week fixed effects, same-day and lagged responses, and a within-make–model placebo."
1140 + },
1141 + {
1142 + "title": "Regional price tests",
1143 + "description": "Quality-adjusted regional premia with and without trim held fixed; joint significance tests across seventeen regions."
1144 + },
1145 + {
1146 + "title": "Preregistered liquidity index",
1147 + "description": "Five-component Automobile Market Liquidity Index, built with and without the exit component, tested as a within-region predictor of next-week exit intensity under z-score and rank standardisations."
1148 + }
1149 + ],
1150 + "reproducibility": [
1151 + "Pipeline with immutable snapshot, exclusion log, preregistered plan and a claims ledger (CLAIMS_LEDGER.csv) that ties every statement to an experiment",
1152 + "Decision log (DECISIONS.md) and experiment log (EXPERIMENTS.md) released with configs and figures",
1153 + "Listing data are asking prices, not transactions; extension to registration transfers is specified as the next step",
1154 + "Full LaTeX source of the 44-page paper"
1155 + ],
1156 + "keywords": [
1157 + "used cars",
1158 + "asking prices",
1159 + "hedonic pricing",
1160 + "markdown hazard",
1161 + "listing survival",
1162 + "dealer competition",
1163 + "market liquidity index",
1164 + "Quebec"
1165 + ],
1166 + "jel": "D22, D83, L11, L62, R32, C41"
1167 + },
1168 + {
1169 + "slug": "wp35",
1170 + "repo": "https://git.spboucher.ai/wp35_uqo",
1171 + "hero": {
1172 + "headline": "The compression reading of institutions is coherent and simulable; its empirical corollary fails a frozen pre-registration",
1173 + "subheadline": "Five theorems and 6,048 agent-based runs make the framework precise, but on 626 Seshat polities, 1,291 Ethnographic Atlas societies and 14,637 country-years the demand–capacity mismatch, though associated with instability (0.20 and 0.39 log-odds per SD), adds nothing identifiable to scale alone or capacity alone."
1174 + },
1175 + "abstract": [
1176 + "Institutions, standards, records and administrative hierarchies can be read as devices that lower the communication cost of coordinating many people: a form of compression. This paper states that reading formally, derives what it does and does not imply, and tests its empirical corollary, that a mismatch between a society's coordination demand (population, territory, settlement structure) and its coordination capacity (administration, standardized rules and money, communication infrastructure, record-keeping) predicts subsequent instability, under a frozen pre-registration.",
1177 + "Demand, capacity and mismatch indices are built from the Seshat Polaris-2026 databank (626 polities; 127 CrisisDB crisis records with polity-level predictors, dated facts preceding the crisis where they exist), the Ethnographic Atlas (1,291 societies) and a V-Dem–Maddison country-year panel (14,637 country-years, 901 distinct event-years, 1820–2025), with a transparent block index and a latent-variable alternative, uncertainty propagation and out-of-sample validation. Capacity and scale are positively associated in all three datasets. The mismatch index is positively associated with crisis severity in Seshat (0.20 log-odds per SD, stable across regions, eras and cultural spheres) and with instability onsets in modern states (0.39 per SD), but by the pre-registered criterion it fails: it does not beat the scale-only baseline out of sample in Seshat, nor the capacity-only baseline in the modern panel; the restriction it imposes on scale and capacity is rejected in the modern panel; a placebo index is not distinguishable from it in Seshat; the modern association is nearly as strong with past as with future events; and it is not robust to an Oster-type adjustment. No threshold is found in Seshat and only a small nonlinearity in the modern panel. Among Seshat polities with a crisis record, higher capacity is also associated with a shorter polity duration, against the hypothesis.",
1178 + "Five theorems make the compression reading precise (coding gain, break-even scale, stale-code penalty, hierarchical routing, backlog dynamics), and factorial agent-based experiments (6,048 runs, five imposed institutional regimes, equal administrative budgets) instantiate them in silico on the cost side, bits, messages, queues, while locating the regimes where institutions hurt: unpaid set-up costs, saturated hierarchies, and weakly structured task environments. The conclusion is that the compression framework is coherent and simulable, that scale and capacity each matter, but that their mismatch, as measurable in existing historical data, carries no identifiable predictive content of its own. The universal theory is not supported; the narrower results are stated with their scope."
1179 + ],
1180 + "findings": [
1181 + {
1182 + "value": "626 / 1,291 / 14,637",
1183 + "label": "Seshat polities, Ethnographic Atlas societies, V-Dem–Maddison country-years (1820–2025)"
1184 + },
1185 + {
1186 + "value": "0.20 log-odds / SD",
1187 + "label": "mismatch index on crisis severity in Seshat, stable across regions, eras and cultural spheres"
1188 + },
1189 + {
1190 + "value": "0.39 / SD",
1191 + "label": "mismatch index on instability onsets in modern states"
1192 + },
1193 + {
1194 + "value": "Fails",
1195 + "label": "pre-registered criterion: no out-of-sample gain over scale-only (Seshat) or capacity-only (modern) baselines"
1196 + },
1197 + {
1198 + "value": "5",
1199 + "label": "theorems: coding gain, break-even scale, stale-code penalty, hierarchical routing, backlog dynamics"
1200 + },
1201 + {
1202 + "value": "6,048",
1203 + "label": "factorial agent-based runs under five institutional regimes with equal administrative budgets"
1204 + }
1205 + ],
1206 + "contributions": [
1207 + {
1208 + "title": "A formal statement of institutions as compression",
1209 + "description": "Shared codes, records and routing structures that lower the communication cost per coordinated task, with five provable properties and an explicit account of what the reading does not imply."
1210 + },
1211 + {
1212 + "title": "A pre-registered empirical test that fails honestly",
1213 + "description": "The mismatch index is associated with instability in the predicted direction but does not beat scale-only or capacity-only baselines out of sample, is matched by a placebo, loads as strongly on past as on future events and does not survive an Oster-type adjustment; the universal framing is rejected under the rule committed to in advance."
1214 + },
1215 + {
1216 + "title": "A measurement model for Seshat",
1217 + "description": "Presence codes treated as missing where missing, with ranges, dates and latent-score uncertainty carried through to inference, and a transparent block index beside a latent-variable alternative."
1218 + },
1219 + {
1220 + "title": "Agent-based testbed of the mechanisms",
1221 + "description": "Factorial experiments measure bits, messages and queues under five regimes and locate where institutions hurt: unpaid set-up costs, saturated hierarchies and weakly structured task environments."
1222 + },
1223 + {
1224 + "title": "What the data cannot answer",
1225 + "description": "Scale and capacity are so tightly coupled that their difference carries one sixth of their joint variance; utilization, reach and administrative load are not coded in any existing databank, which defines the next credible test."
1226 + }
1227 + ],
1228 + "data": [
1229 + {
1230 + "title": "Seshat Polaris-2026 databank",
1231 + "description": "626 polities and 127 CrisisDB crisis records with polity-level predictors; dated facts preceding each crisis used where they exist; presence codes handled as missing when missing."
1232 + },
1233 + {
1234 + "title": "Ethnographic Atlas",
1235 + "description": "1,291 societies used to test the association between coordination capacity and scale in a cross-section of ethnographically described societies."
1236 + },
1237 + {
1238 + "title": "V-Dem–Maddison country-year panel",
1239 + "description": "14,637 country-years with 901 distinct event-years over 1820–2025, combining institutional capacity measures with population and income to test instability onsets."
1240 + }
1241 + ],
1242 + "methodology": [
1243 + {
1244 + "title": "Frozen pre-registration",
1245 + "description": "Hypotheses, indices, baselines and decision criteria committed before estimation; results reported against those criteria, with the universal and the narrower readings separated."
1246 + },
1247 + {
1248 + "title": "Index construction with uncertainty",
1249 + "description": "Transparent block indices for demand and capacity and a latent-variable alternative, with uncertainty propagation and out-of-sample validation."
1250 + },
1251 + {
1252 + "title": "Baselines, placebos and robustness",
1253 + "description": "Scale-only and capacity-only baselines, placebo indices, lead–lag symmetry checks on past versus future events, Oster-type adjustment, threshold and nonlinearity tests, and polity-duration analysis."
1254 + },
1255 + {
1256 + "title": "Formal framework",
1257 + "description": "Five theorems on coding gain, break-even scale, stale-code penalty, hierarchical routing and backlog dynamics, each with proof."
1258 + },
1259 + {
1260 + "title": "Factorial agent-based experiments",
1261 + "description": "6,048 runs across five imposed institutional regimes with equal administrative budgets, measuring the cost side (bits, messages, queues) of coordination."
1262 + }
1263 + ],
1264 + "reproducibility": [
1265 + "Replication package reproduces every number from the raw downloads with one command (Makefile, pyproject, configs)",
1266 + "Pre-registration and measurement model for Seshat released, with a validated bibliography and novelty matrix for the converging literatures",
1267 + "Agent-based testbed released with the factorial design and regime definitions",
1268 + "Full LaTeX source of the 80-page paper"
1269 + ],
1270 + "keywords": [
1271 + "institutions",
1272 + "coordination",
1273 + "state capacity",
1274 + "social complexity",
1275 + "Seshat",
1276 + "pre-registration",
1277 + "agent-based simulation",
1278 + "information theory"
1279 + ],
1280 + "jel": "D02, D83, H11, N40, P48, C63"
1281 + },
1282 + {
1283 + "slug": "wp34",
1284 + "repo": "https://git.spboucher.ai/wp34_uqo",
1285 + "hero": {
1286 + "headline": "No shortcut past the square-root barrier: an autonomous pipeline checked 25 hypotheses against Bitcoin's primitives and reports controlled negatives",
1287 + "subheadline": "secp256k1's structure is fully explained by complex multiplication by Z[ω] (the libsecp256k1 GLV constants are the coordinates of π̄ − 1), Pollard rho matches 1.254√n on 6,240 instances, batch Schnorr verification saturates at 3.1×, reduced SHA-256 yields to SAT only up to 18 steps, and the word-level identities are machine-checked in Lean 4."
1288 + },
1289 + "abstract": [
1290 + "Bitcoin's security rests on the elliptic-curve discrete-logarithm problem on secp256k1, on SHA-256, and on the ECDSA and BIP340 Schnorr signature schemes. The generic cost of the discrete logarithm, Θ(√n), is a theorem in the generic-group model; the curve's unusual structure, a special prime, j-invariant 0 and complex multiplication by the Eisenstein integers, has long invited speculation that an algebraic shortcut might exist. The paper tests this speculation with an autonomous, six-stage discovery pipeline (verified literature base, 25 structured hypotheses, exact symbolic checks, seeded experiments with full provenance, Lean 4 machine-checked statements, six-way classification) executed by an AI research agent under a written mandate that forbids assuming shortcuts exist.",
1291 + "Results. Every secp256k1 parameter and invariant is verified twice with exact arithmetic; End(E) = Z[ω] leaves nothing beyond the six automorphisms; the GLV lattice is the ideal (π̄ − 1), which gives the hard-coded libsecp256k1 constants in closed form; the automorphism orbits admit no compatible group law; prime-field index calculus and all transfer attacks are excluded quantitatively. A native engine solved 6,240 synthetic ECDLP instances of 20–56 bits: empirical exponents 0.48–0.50, rho constant 1.254√n against the random-mapping prediction 1.2533, order-6 classes 2.42× fewer iterations (√6) but no wall-clock gain in this implementation, 22× parallel speed-up on 28 threads, the predicted time–memory and preprocessing trade-offs, and no iteration-function design below the random-mapping floor. BIP340 batch verification reaches 3.1× and saturates for an identified reason; ECDSA and Schnorr verify at the same cost and differ in proof structure.",
1292 + "Reduced-step SHA-256 diffuses fully in six steps, shows no first-order regularity beyond five, and yields to generic SAT search only up to 18-step collisions and 2^16-bit preimages, far below the 31-step records; the 9-step local collision is reproduced and its linear part proved. Lean 4 proofs cover the word-level identities, the bijectivity of the step and of all four Σ/σ functions (refuting the pipeline's own conjecture that σ0, σ1 are singular), the schedule inversion, the linearised local collision, ECDSA malleability and Schnorr correctness. The pipeline produced sixteen reproductions with measured constants, six empirical observations, three rejected conjectures and no claim of a weakness; the paper argues that this is the correct output for automated mathematical research on cryptographic standards, and releases everything needed to reproduce it."
1293 + ],
1294 + "findings": [
1295 + {
1296 + "value": "25",
1297 + "label": "structured, falsifiable hypotheses tested through a six-stage pipeline under a no-assumed-shortcut mandate"
1298 + },
1299 + {
1300 + "value": "6,240",
1301 + "label": "synthetic ECDLP instances (20–56 bits) solved by a native engine; exponents 0.48–0.50"
1302 + },
1303 + {
1304 + "value": "1.254√n",
1305 + "label": "measured Pollard rho iterations vs the random-mapping prediction 1.2533√n"
1306 + },
1307 + {
1308 + "value": "(π̄ − 1)",
1309 + "label": "the GLV lattice ideal: libsecp256k1's hard-coded constants in closed form"
1310 + },
1311 + {
1312 + "value": "3.1×",
1313 + "label": "BIP340 batch verification speed-up, saturating for an identified reason"
1314 + },
1315 + {
1316 + "value": "18 steps",
1317 + "label": "largest reduced-round SHA-256 collision reached by generic SAT search (records: 31 steps)"
1318 + }
1319 + ],
1320 + "contributions": [
1321 + {
1322 + "title": "Algebraic structure of secp256k1 fully accounted for",
1323 + "description": "Complex multiplication by Z[ω] yields the √6 reduction in rho iterations and the halving of doublings in scalar multiplication, determines the GLV constants of libsecp256k1 in closed form, and leaves no room for a further endomorphism, a compatible group law on automorphism orbits, or index-calculus and transfer attacks."
1324 + },
1325 + {
1326 + "title": "Generic algorithms measured, not assumed",
1327 + "description": "A native engine with operation counts and provenance reproduces the random-mapping theory on 6,240 instances, including the order-6 iteration saving (2.42×) without wall-clock gain, 22× parallel scaling on 28 threads, and the 1.5√(n/T) preprocessing trade-off; an automated search over iteration functions cannot beat the floor."
1328 + },
1329 + {
1330 + "title": "Signature schemes compared on cost and proof structure",
1331 + "description": "ECDSA and BIP340 Schnorr verify at the same cost and differ in proof structure; batch verification is the one real, consensus-neutral speed-up and saturates near 3×."
1332 + },
1333 + {
1334 + "title": "Reduced-step SHA-256 under SAT",
1335 + "description": "Full diffusion in six steps, no first-order regularity beyond five, 18-step collisions and 2^16-bit preimages as the reach of generic solvers, and the 9-step local collision reproduced with its linear part proved."
1336 + },
1337 + {
1338 + "title": "Machine-checked identities and a methodology for automated discovery",
1339 + "description": "Lean 4 proofs of word-level identities, Σ/σ bijectivity (refuting the pipeline's own conjecture), schedule inversion, ECDSA malleability and Schnorr correctness; the controls that prevented over-interpretation (operation counts beside timings, two implementations per component, negative controls, exact arithmetic, provenance, literature match before the word new) are documented as the pattern for automated research on cryptographic standards."
1340 + }
1341 + ],
1342 + "data": [
1343 + {
1344 + "title": "No external data: generated instances with provenance",
1345 + "description": "6,240 synthetic ECDLP instances of 20–56 bits generated from seeds, with operation counts, timings and provenance recorded for every run; secp256k1 parameters and invariants verified twice with exact arithmetic."
1346 + },
1347 + {
1348 + "title": "Verified literature base",
1349 + "description": "A curated, checked bibliography of the discrete-logarithm, endomorphism, signature and SHA-256 cryptanalysis literature against which every candidate result is classified before the word new can be used."
1350 + },
1351 + {
1352 + "title": "Formal artefacts",
1353 + "description": "Lean 4 statements and proofs for the word-level identities, Σ/σ functions, schedule inversion, linearised local collision, ECDSA malleability and Schnorr correctness; SAT instances for reduced-step collisions and preimages."
1354 + }
1355 + ],
1356 + "methodology": [
1357 + {
1358 + "title": "Six-stage hypothesis-driven pipeline",
1359 + "description": "Verified literature base, 25 structured hypotheses, exact symbolic checks, seeded experiments with full provenance, machine-checked statements and a six-way classification rubric executed by an AI research agent under a written mandate."
1360 + },
1361 + {
1362 + "title": "Exact algebra of the curve",
1363 + "description": "Endomorphism ring, automorphism orbits, GLV lattice as the ideal (π̄ − 1), and quantitative exclusion of index calculus and transfer attacks."
1364 + },
1365 + {
1366 + "title": "Native experimental engine",
1367 + "description": "Pollard rho with order-6 classes, parallel, time–memory and preprocessing variants, with an automated search over iteration-function designs against the random-mapping floor."
1368 + },
1369 + {
1370 + "title": "SAT-based cryptanalysis of reduced SHA-256",
1371 + "description": "Diffusion and regularity tests, collision and preimage search with generic solvers, reproduction of the 9-step local collision."
1372 + },
1373 + {
1374 + "title": "Formal verification and classification",
1375 + "description": "Lean 4 proofs separated from informal statements; every candidate result classified as reproduction, observation, rejected conjecture or new only with a literature match."
1376 + }
1377 + ],
1378 + "reproducibility": [
1379 + "Everything needed to reproduce the sixteen reproductions, six observations and three rejected conjectures is released: hypotheses, experiments with seeds and provenance, formal proofs, figures and logs",
1380 + "Two implementations per component, negative controls per family and exact arithmetic throughout",
1381 + "Lean 4 formalisation of the word-level and signature-equation facts",
1382 + "Full LaTeX source of the 74-page paper"
1383 + ],
1384 + "keywords": [
1385 + "elliptic curve discrete logarithm",
1386 + "secp256k1",
1387 + "Pollard rho",
1388 + "endomorphisms",
1389 + "ECDSA",
1390 + "Schnorr signatures",
1391 + "BIP340",
1392 + "SHA-256",
1393 + "SAT-based cryptanalysis",
1394 + "automated mathematical discovery",
1395 + "Lean 4",
1396 + "reproducibility"
1397 + ],
1398 + "jel": "C63, C88, D83 (MSC 2020: 94A60, 11T71, 11G20, 68V15, 68W30)"
1399 + },
1400 + {
1401 + "slug": "firc1",
1402 + "hero": {
1403 + "headline": "121,099 disclosed strategic projects, each traceable to an SEC filing passage",
1404 + "subheadline": "SPID reads 527,548 filings of 2,564 issuers since 2010 and extracts 308,065 project mentions into a 21-category taxonomy; the paper documents the instrument and its audit, not yet a validated measurement."
1405 + },
1406 + "abstract": [
1407 + "This paper documents the construction of SPID, a project-level research database that reconstructs the strategic initiatives disclosed by U.S.-listed companies in their SEC filings. Starting from an issuer universe that merges a current S&P 500 snapshot with the holdings of the iShares Russell 3000 ETF (2,564 distinct SEC registrants), the pipeline collects every 10-K, 10-Q and 8-K filed since January 2010 (527,548 documents), segments them into 1.34 million sections, and submits 1.12 million eligible sections to a schema-constrained large-language-model extractor.",
1408 + "The result is 308,065 evidence-linked project mentions, resolved into 121,099 candidate projects for 2,492 issuers, each tied to its SEC accession number, section and source passage, together with a controlled 21-category taxonomy, status, cited years, locations, technologies, partners and disclosed amounts. The paper describes the universe definition and its reconciliation, the distributed acquisition and parsing architecture, the extraction contract and its confidence semantics, the mention-to-project resolution rule and its known failure modes, and the point-in-time structure of the timelines.",
1409 + "A validation framework (stratified human annotation, precision and recall on an independent gold standard, linkage audits and coverage gates) is set out, stating explicitly which components are implemented and which remain to be executed. Descriptive patterns are reported for the full corpus (M&A and integration account for 34.5% of projects; a USD amount is attached to 46.7% of projects, median 128.5 M USD), with the caveat that a disclosed project is neither a verified investment nor a realized expenditure. SPID should be read as a research instrument under validation rather than as an exhaustive census of Russell 3000 projects."
1410 + ],
1411 + "findings": [
1412 + {
1413 + "value": "527,548",
1414 + "label": "SEC filings (10-K, 10-Q, 8-K) of 2,564 issuers, 2010–2026"
1415 + },
1416 + {
1417 + "value": "1.12 million",
1418 + "label": "filing sections submitted to the LLM extractor"
1419 + },
1420 + {
1421 + "value": "308,065",
1422 + "label": "evidence-linked project mentions"
1423 + },
1424 + {
1425 + "value": "121,099",
1426 + "label": "resolved candidate projects for 2,492 issuers"
1427 + },
1428 + {
1429 + "value": "34.5%",
1430 + "label": "of projects are M&A and post-merger integration"
1431 + },
1432 + {
1433 + "value": "128.5 M USD",
1434 + "label": "median largest disclosed amount per project"
1435 + }
1436 + ],
1437 + "contributions": [
1438 + {
1439 + "title": "A precise measurement target",
1440 + "description": "A project is defined as a distinct, identifiable, issuer-associated strategic initiative, separated from a project mention, a generic strategy, a historical or third-party reference and an uncertain candidate, and tied to the existing 21-category SPID taxonomy."
1441 + },
1442 + {
1443 + "title": "A scalable, reproducible architecture",
1444 + "description": "A frozen issuer universe, distributed acquisition across ten Apple-silicon cluster nodes and one external server under a shared SEC rate budget, section-level parsing of 527,548 filings and a resumable, idempotent extraction step costing roughly 775 USD."
1445 + },
1446 + {
1447 + "title": "An extraction contract exposed to criticism",
1448 + "description": "The prompt, the JSON schema, the confidence threshold (mentions below 0.45 dropped) and the deterministic resolution rule are documented in enough detail to be audited; 35.2% of mentions carry confidence 1.0 and 32.7% carry 0.9, a calibration question left for the annotation study."
1449 + },
1450 + {
1451 + "title": "Descriptive patterns with explicit caveats",
1452 + "description": "M&A and integration dominate (34.5% of projects), followed by capex programs (8.9%) and partnerships (8.2%); technology categories are small in count (264–449 projects each) but carry high amounts (median 128–166 M USD for data centers and AI initiatives)."
1453 + },
1454 + {
1455 + "title": "A validation framework with stated status",
1456 + "description": "Stratified human annotation by two independent coders, precision/recall/F1 on a gold standard, confidence calibration, linkage audits and coverage gates are specified; the paper states which gates are met and which are not, and asks that SPID be cited as a database under validation."
1457 + }
1458 + ],
1459 + "data": [
1460 + {
1461 + "title": "SPID (SEC Project Intelligence Database), Russell 3000 release",
1462 + "description": "527,548 Form 10-K, 10-Q and 8-K filings of 2,564 issuers (current S&P 500 snapshot merged with iShares IWV holdings) filed January 2010–October 2026, parsed into 1.34 million sections; 308,065 mentions and 121,099 projects stored in DuckDB (spid.duckdb, 3.2 GB) on top of the 68 GB UQO EDGAR research corpus. Research funded by the FIRC (Fonds institutionnel de recherche et de création, UQO)."
1463 + },
1464 + {
1465 + "title": "Project attributes and graph",
1466 + "description": "Each project carries category, status, cited years, locations, technologies, partners and disclosed amounts (USD amount on 120,240 mentions and 56,500 projects; median 128.5 M USD, P99 12.5 B USD) and is projected into a graph of 183,472 nodes; health care (23,679 projects) and industrials (17,890) contribute the most."
1467 + }
1468 + ],
1469 + "methodology": [
1470 + {
1471 + "title": "Universe definition and reconciliation",
1472 + "description": "S&P 500 snapshot and IWV holdings are matched to SEC CIKs (2,564 registrants, 2,499 filing the three forms); the fund-holdings proxy at one date is acknowledged as a substitute for licensed Russell membership."
1473 + },
1474 + {
1475 + "title": "Distributed acquisition and parsing",
1476 + "description": "Eleven machines (ten MacLustr Apple-silicon nodes and one OVHcloud server) download under SEC fair-access limits with an interleaved CIK allocation, parse filings into sections and export tables to Parquet."
1477 + },
1478 + {
1479 + "title": "Schema-constrained LLM extraction",
1480 + "description": "1.12 million eligible sections (focus windows around lexical cues) are submitted to an extractor returning structured mentions with category, status, amounts, years, locations, technologies, partners and a self-reported confidence."
1481 + },
1482 + {
1483 + "title": "Mention-to-project resolution and timelines",
1484 + "description": "A deterministic key resolves mentions into projects with point-in-time timelines; its known failure modes (merging distinct phases, splitting renamed initiatives) are documented, with explainable linkage and human adjudication proposed."
1485 + },
1486 + {
1487 + "title": "Validation protocol",
1488 + "description": "Stratified gold-standard annotation, inter-annotator agreement (Cohen's κ), mention-level precision/recall/F1, category macro/micro-F1, field accuracy, abstention rate and confidence calibration with bootstrap intervals, to be executed before derived data release."
1489 + }
1490 + ],
1491 + "reproducibility": [
1492 + "Corpus and project layers reproducible from the uqo-research-db repository (revision 2324b58), the frozen universe artifact, download logs and the extraction ledger",
1493 + "Every number in the paper regenerated by code/export_stats.py (DuckDB) and code/make_tables.py (CSV to LaTeX tables and figures)",
1494 + "Claim register (Appendix D) separating implemented, documented-but-unverified, proposed and empirically verified claims; raw filings not redistributed under SEC fair-access policy",
1495 + "Full LaTeX source of the 39-page paper; research funded by the FIRC (Fonds institutionnel de recherche et de création, UQO)"
1496 + ],
1497 + "keywords": [
1498 + "SEC filings",
1499 + "text as data",
1500 + "corporate investment",
1501 + "strategic projects",
1502 + "information extraction",
1503 + "large language models",
1504 + "entity resolution",
1505 + "Russell 3000",
1506 + "measurement validation"
1507 + ],
1508 + "repo": "",
1509 + "jel": "C55, C81, G31, G38, M41, O33"
1510 + },
1511 + {
1512 + "slug": "firc2",
1513 + "hero": {
1514 + "headline": "The post-ChatGPT AI premium is a large-cap episode, not a market-wide regime change",
1515 + "subheadline": "In the Russell 3000, AI language earns a modest, persistent +5.5 pp per year quintile spread (t = 2.41); the jump replicated in the S&P 500 (+5.0 to +15.0 pp) is confined to the largest thousand firms (+2.9 to +12.8 pp), and disclosed AI projects carry no premium."
1516 + },
1517 + "abstract": [
1518 + "This paper measures artificial-intelligence (AI) engagement in the SEC filings of some 2,500 U.S. listed firms (the current S&P 500 constituents and the holdings of the iShares Russell 3000 ETF) by embedding every passage of their Form 10-K, 10-Q and 8-K filings of 2010–2026 with a sentence transformer and scoring it against curated AI anchor concepts (internal adoption, infrastructure exposure, product-disruption risk). The signal of filing year Y is linked to the forward twelve-month return from July of Y+1, so that it predates the holding period by at least six months. The panel of 27,468 firm-years (2,379 firms) extends the authors' earlier S&P 500 study, which rested on current reports only, to annual and quarterly reports and to the mid- and small-capitalization firms that make up most of the Russell 3000, and adds a second, independent measure of AI engagement: the AI, data-center, cloud and automation projects that a large-language-model reader extracts from the same filings (the SPID database).",
1519 + "Three results stand out. First, in the broad universe AI language carries a modest but persistent cross-sectional premium over the whole sample: the equal-weighted top-minus-bottom quintile spread is +5.5 percentage points per year (t = 2.41) and the Fama–MacBeth slope +3.07 points per standard deviation (t = 2.80), with a significant within-sector component. Second, the sharp post-ChatGPT increase documented for the S&P 500, replicated within the new panel (+5.0 to +15.0 points), does not generalize: in the Russell 3000 the post-launch spread is +7.9 points against +4.9 before, the 2022 cohort is negative, a placebo over ten candidate break years ranks 2022 eighth, and the monthly long–short earns no post-launch alpha. The increase is confined to the largest thousand firms (+2.9 to +12.8 points), whereas smaller firms earned about +9.6 points before the launch and +8.4 after; the ChatGPT event study shows the same size gradient (+19.5 versus +9.9 points of one-year abnormal return).",
1520 + "Third, disclosed AI projects carry no return premium, while the language of annual reports carries the strongest one. The evidence is consistent with a persistent, partly within-industry premium on AI exposure in the broad market and with a large-cap re-rating after 2022, not with a market-wide regime change; an index sample alone would have read a large-cap episode as a change in how the market prices disclosure."
1521 + ],
1522 + "findings": [
1523 + {
1524 + "value": "27,468",
1525 + "label": "firm-years, 2,379 firms, formation years 2010–2024"
1526 + },
1527 + {
1528 + "value": "+5.5 pp/yr",
1529 + "label": "equal-weighted Q5–Q1 spread, full sample (t = 2.41)"
1530 + },
1531 + {
1532 + "value": "+3.07 pp/SD",
1533 + "label": "Fama–MacBeth slope (t = 2.80)"
1534 + },
1535 + {
1536 + "value": "+2.9 → +12.8 pp",
1537 + "label": "post-ChatGPT spread among the largest 1,000 firms"
1538 + },
1539 + {
1540 + "value": "8th of 10",
1541 + "label": "rank of 2022 among placebo break years"
1542 + },
1543 + {
1544 + "value": "−0.22",
1545 + "label": "Fama–MacBeth slope of disclosed AI projects (t = −0.24)"
1546 + }
1547 + ],
1548 + "contributions": [
1549 + {
1550 + "title": "A broad-market test of AI-language pricing",
1551 + "description": "The laboratory is enlarged from 500 index constituents to about 2,500 firms across the Russell 3000 capitalization spectrum, from current reports alone to the full text of 10-K, 10-Q and 8-K filings, and to 27,468 look-ahead-free firm-years."
1552 + },
1553 + {
1554 + "title": "A modest, persistent premium in the broad market",
1555 + "description": "The equal-weighted Q5–Q1 spread is +5.5 pp per year (t = 2.41), the value-weighted spread +13.9 (t = 2.59), and the Fama–MacBeth slope +3.07 pp per SD (t = 2.80), with a within-sector slope of +1.46 (t = 4.38)."
1556 + },
1557 + {
1558 + "title": "The post-ChatGPT jump does not generalize",
1559 + "description": "Replicated in the S&P 500 members (+5.0 to +15.0 pp, t = 2.21), the increase is +4.9 to +7.9 pp in the Russell 3000 (post t = 1.34), with a negative 2022 cohort, a placebo rank of 8th out of 10 break years and a post-launch FF6 alpha of −0.3% per year."
1560 + },
1561 + {
1562 + "title": "A size gradient, not a regime",
1563 + "description": "The largest 1,000 firms rise from +2.9 to +12.8 pp (t = 2.09) while the remaining firms move from +9.6 to +8.4; the ChatGPT event study gives +19.5 pp one-year abnormal return for the largest 1,000 versus +9.9 for the rest."
1564 + },
1565 + {
1566 + "title": "Words versus projects",
1567 + "description": "Disclosed AI projects from the SPID database carry no premium (tech-project quintile spread −1.9 pre and −1.3 post; FM −0.22, t = −0.24), whereas 10-K language carries the strongest signal (+6.9 pre to +16.7 pp post, t = 2.61)."
1568 + },
1569 + {
1570 + "title": "A methodological warning about index samples",
1571 + "description": "The S&P 500 is the one part of the market where the premium was absent before 2022 and large afterward; a study confined to it reads a large-cap episode as a change in how the market prices disclosure."
1572 + }
1573 + ],
1574 + "data": [
1575 + {
1576 + "title": "UQO EDGAR Research Database, Russell 3000 extension",
1577 + "description": "527,548 Form 10-K, 10-Q and 8-K filings of 2,499 issuers filed since January 2010, parsed into sections and 512-word chunks; 10,720,890 chunks embedded with all-MiniLM-L6-v2 (384 dimensions) and scored by cosine similarity to anchor passages for eight concepts. Research funded by the FIRC (Fonds institutionnel de recherche et de création, UQO)."
1578 + },
1579 + {
1580 + "title": "Return and factor panel",
1581 + "description": "Monthly adjusted-close prices for 2,499 tickers from Yahoo Finance (June 2009–October 2026), SPY and IWV benchmarks, SEC share counts, Fama–French five factors plus momentum and a TECH factor; 27,468 firm-years with valid forward returns (6,793 for S&P 500 members)."
1582 + },
1583 + {
1584 + "title": "SPID AI-project counts",
1585 + "description": "Firm-year counts of AI, data-center, cloud and automation projects extracted by a large-language-model reader from the same filings (UQO Working Paper No. 0001), used as an independent, content-based measure of AI engagement."
1586 + }
1587 + ],
1588 + "methodology": [
1589 + {
1590 + "title": "Semantic scoring of filings",
1591 + "description": "Each 512-word chunk is embedded with a sentence transformer and scored against written anchor passages for AI adoption, infrastructure exposure and product-disruption risk; chunk scores are aggregated to a standardized firm-year composite."
1592 + },
1593 + {
1594 + "title": "Look-ahead-free portfolio sorts",
1595 + "description": "The signal of filing year Y is linked to the twelve-month return from July Y+1 to June Y+2; equal- and value-weighted quintile spreads are computed for fifteen formation cohorts, with S&P 500 members and the full universe reported side by side."
1596 + },
1597 + {
1598 + "title": "Fama–MacBeth and panel regressions",
1599 + "description": "Slopes per standard deviation with and without sector dummies and sector-by-year fixed effects, firm-clustered pooled tests, winsorized returns and size-neutral sorts."
1600 + },
1601 + {
1602 + "title": "Break-date placebo and factor spanning",
1603 + "description": "Pre/post ChatGPT comparisons are tested against ten candidate break years; the monthly long–short is spanned on FF6 plus a TECH factor to check for post-launch alpha."
1604 + },
1605 + {
1606 + "title": "ChatGPT event study",
1607 + "description": "Abnormal returns against the IWV market index over 60 days to one year (N = 2,226), with Welch and Brown–Warner crude-dependence-adjustment t-statistics, split by firm size."
1608 + }
1609 + ],
1610 + "reproducibility": [
1611 + "Eight numbered scripts (scripts/r3k_00–07) from factor download and Yahoo prices to panel, core and extended analyses, event study, 14 figures and tables",
1612 + "Every number in the text is a LaTeX macro generated in tables/macros.tex; results stored in results/r3k/*.json",
1613 + "Previous S&P 500 version (v1.1) archived with its review report and revision changelog in docs/",
1614 + "Full LaTeX source of the 48-page paper plus a LaTeX-to-Word conversion pipeline; research funded by the FIRC (Fonds institutionnel de recherche et de création, UQO)"
1615 + ],
1616 + "keywords": [
1617 + "textual analysis",
1618 + "sentence embeddings",
1619 + "artificial intelligence",
1620 + "disclosure",
1621 + "cross-section of returns",
1622 + "Russell 3000",
1623 + "firm size",
1624 + "event study",
1625 + "generative AI"
1626 + ],
1627 + "repo": "https://git.spboucher.ai/wp17_uqo",
1628 + "jel": "G12, G14, G17, M41, O33"
1629 + },
1630 + {
1631 + "slug": "wp1",
1632 + "hero": {
1633 + "headline": "Amenity capitalisation is not one number: it declines along the price distribution",
1634 + "subheadline": "For Quebec single-family houses the highway, transit and park differentials fall monotonically from the 10th to the 90th conditional percentile, the hospital indicator changes sign, and residual spatial autocorrelation (Moran's I 0.51–0.58) is strong in every segment."
1635 + },
1636 + "abstract": [
1637 + "Hedonic regressions estimated at the conditional mean impose a single implicit price for each housing attribute across the price distribution. This paper asks whether the capitalisation of neighbourhood amenities into Quebec residential asking prices varies along the conditional price distribution, and whether it survives an adjustment for spatial dependence. Using a January 2026 snapshot of 42,481 Centris listings in four segments (single-family houses, condominiums, multiplexes and vacant land; estimation samples of 6,250, 1,031, 3,755 and 9,239), it estimates quantile regressions at τ ∈ {0.10, 0.25, 0.50, 0.75, 0.90} and a two-stage spatial quantile regression in which a k-nearest-neighbour spatial lag of neighbouring log prices is instrumented by spatially lagged covariates.",
1638 + "For houses, the coefficients of the highway, transit and park indicators decline monotonically along the distribution (highway from 0.205 at τ = 0.10 to 0.034 at τ = 0.90; transit from 0.178 to 0.053; park from 0.122 to 0.014) and the hospital indicator changes sign (from −0.085 to +0.041); the equality of the tail coefficients is rejected for all four, while the school discount of about 11 log points is constant. Condominiums show no detectable heterogeneity; multiplexes and vacant land show it for two amenities each. Moran's I on OLS residuals lies between 0.51 and 0.58 in every segment; the median spatial-lag coefficient ranges from 0.13 (multiplexes) to 0.32 (land), and its introduction attenuates the median highway, transit and school differentials for houses by a third to a half.",
1639 + "The estimates are quantile-specific capitalisation gradients in asking prices, not willingness-to-pay measures; inference relies on a paired bootstrap that ignores spatial dependence, and the spatial-lag coefficients carry no standard errors. All reported figures are drawn from the preserved log of the original estimation run, and the robustness analyses that a re-estimation should provide are stated explicitly."
1640 + ],
1641 + "findings": [
1642 + {
1643 + "value": "42,481",
1644 + "label": "Centris listings, Quebec, 8 January 2026 snapshot"
1645 + },
1646 + {
1647 + "value": "0.205 → 0.034",
1648 + "label": "house highway coefficient from τ = 0.10 to 0.90"
1649 + },
1650 + {
1651 + "value": "−0.085 → +0.041",
1652 + "label": "hospital coefficient changes sign across quantiles"
1653 + },
1654 + {
1655 + "value": "≈ −11 log pts",
1656 + "label": "school discount, constant at every quantile"
1657 + },
1658 + {
1659 + "value": "0.51–0.58",
1660 + "label": "Moran's I on OLS residuals, all four segments"
1661 + },
1662 + {
1663 + "value": "0.13–0.32",
1664 + "label": "median spatial-lag coefficient across segments"
1665 + }
1666 + ],
1667 + "contributions": [
1668 + {
1669 + "title": "Capitalisation gradients instead of one implicit price",
1670 + "description": "For single-family houses the highway, transit and park differentials fall monotonically from the 10th to the 90th conditional percentile (highway 0.205 → 0.034, transit 0.178 → 0.053, park 0.122 → 0.014); tail equality is rejected for all three."
1671 + },
1672 + {
1673 + "title": "A sign-changing hospital differential",
1674 + "description": "The hospital indicator moves from −0.085 at τ = 0.10 to +0.041 at τ = 0.90 for houses (Wald 12.1), whereas the school discount of 0.10–0.14 log points is the same at every quantile (Wald 0.05)."
1675 + },
1676 + {
1677 + "title": "Heterogeneity is segment-specific",
1678 + "description": "Significant tail heterogeneity appears for 4 of 6 amenities among houses, none among condominiums, and 2 of 6 each for multiplexes (highway, hospital) and vacant land (school, golf)."
1679 + },
1680 + {
1681 + "title": "Spatial dependence is strong and attenuates the gradients",
1682 + "description": "Moran's I on OLS residuals is 0.506–0.580; the median spatial lag (0.291 for houses) reduces the median highway, transit and school differentials by a third to a half (highway 0.113 → 0.057)."
1683 + },
1684 + {
1685 + "title": "An explicit audit trail",
1686 + "description": "Every figure in version 1.1 is parsed from the preserved console log of the original run; the v1.0 tables that could not be traced to that log, and the robustness checks without surviving output, were withdrawn and listed as analyses to execute."
1687 + }
1688 + ],
1689 + "data": [
1690 + {
1691 + "title": "Centris active-listing snapshot",
1692 + "description": "42,481 active listings for the province of Quebec on 8 January 2026 in four product types, with asking price, structural attributes, coordinates, municipality, parking type and a broker text field from which six binary amenity indicators (highway, hospital, park, school, transit, golf) are derived; after filtering and trimming, estimation samples of 6,250 houses, 1,031 condominiums, 3,755 multiplexes and 9,239 parcels (20,275 listings). The snapshot is not redistributed."
1693 + }
1694 + ],
1695 + "methodology": [
1696 + {
1697 + "title": "OLS hedonic benchmark and Moran's I",
1698 + "description": "Semi-log hedonic regression with HC3 standard errors per segment, with Moran's I computed on OLS, quantile and second-stage residuals using a row-standardised inverse-distance 10-nearest-neighbour weight matrix."
1699 + },
1700 + {
1701 + "title": "Quantile regression with paired bootstrap",
1702 + "description": "Koenker–Bassett quantile regressions at five quantiles with paired-bootstrap standard errors (B = 200) and Koenker–Machado pseudo-R²; inter-quantile Wald tests compare τ = 0.10 and τ = 0.90 (and adjacent pairs) using bootstrap variances."
1703 + },
1704 + {
1705 + "title": "Two-stage spatial quantile regression",
1706 + "description": "A least-squares first stage regresses the spatial lag of log price on the covariates and their spatial lags (Kelejian–Prucha instruments); the quantile second stage uses the fitted lag, recorded at the median only and without bootstrap standard errors."
1707 + },
1708 + {
1709 + "title": "Dollar-equivalent implicit prices and impacts",
1710 + "description": "Implicit prices P(τ)[exp(β̂(τ)) − 1] at reconstructed quantile price levels, and a scalar LeSage–Pace direct/indirect/total decomposition at the median spatial-lag estimate."
1711 + }
1712 + ],
1713 + "reproducibility": [
1714 + "Six numbered Python scripts from the OLS baseline to the multi-segment 2SQR with bootstrap; the preserved console log results/sqr_multi_segment_output.txt is the single source of every reported number",
1715 + "scripts/06_tables_figures_from_log.py parses the log in about 5 seconds into 17 LaTeX tables, 5 figures and results/log_parsed.json, with completeness assertions",
1716 + "Full LaTeX source of the 58-page paper with 125 Crossref-verified BibTeX entries; the pre-revision v1.0 PDF and the review report are kept in docs/",
1717 + "Re-estimation on the raw Centris snapshot (about 15 minutes) is documented but was not re-run; the analyses it should add are listed explicitly in the paper"
1718 + ],
1719 + "keywords": [
1720 + "hedonic price function",
1721 + "quantile regression",
1722 + "spatial lag",
1723 + "amenity capitalisation",
1724 + "implicit prices",
1725 + "housing submarkets",
1726 + "asking prices",
1727 + "Quebec"
1728 + ],
1729 + "repo": "https://git.spboucher.ai/wp1_uqo",
1730 + "jel": "R31, C21, C31, Q51"
1731 + },
1732 + {
1733 + "slug": "wp10",
1734 + "hero": {
1735 + "headline": "Whether Quebec assessments are regressive depends on how outliers are treated",
1736 + "subheadline": "Across 522,769 roll-matched sales, the within-market elasticity of the assessment ratio is −0.34 under a lenient 1/99 trim but −0.09 under the IAAO outlier rule, and proportionality holds across the upper 80% of every market."
1737 + },
1738 + "abstract": [
1739 + "Municipal property taxes are levied on assessed values, so any systematic relationship between assessment ratios and market value redistributes the tax burden within a municipality. This paper matches 522,769 residential sales recorded in Quebec between 2021 and 2026 to the triennial assessment rolls of 625 municipalities at the parcel level and provides the first province-wide ratio study of vertical and horizontal assessment equity for a Canadian province. Because the Loi sur la fiscalité municipale pins every roll to a single statutory market-condition date, municipality × roll × sale-year fixed effects separate the mechanical drift of ratios between reference dates from within-market inequity by design.",
1740 + "The central finding is that the measured degree of regressivity hinges on the treatment of extreme assessment ratios. Under a lenient 1/99 trim, the within-market elasticity of the assessment ratio with respect to price is −0.34 (−0.08 under a rank instrument in the spirit of Clapp, 1990) and 95% of municipalities fall below the IAAO price-related-bias band. Under the IAAO outlier rule, which removes 9% of sales concentrated in the bottom price decile, the elasticity is −0.09 (−0.01 and insignificant under the instrument), the province-wide PRB is +0.02, the COD is 17, and three quarters of municipalities meet the vertical-equity standard. A piecewise specification shows that proportionality holds across the upper 80% of each market's price distribution and fails only in the bottom quintile.",
1741 + "Two patterns are invariant to the trimming rule: condominiums are assessed proportionally while single-family homes, plexes, older buildings, land-heavy properties and small municipalities remain mildly regressive, and within-municipality price-related bias widened in 2021–2022 when the post-pandemic boom pulled prices away from the rolls. The implied overpayment by the bottom within-market price decile ranges from about 6% to 65% depending on whether extreme-ratio sales are treated as market transactions, and the top decile underpays by 3–5%; narrowing that range requires sale-validation data that Quebec's registries do not currently provide."
1742 + ],
1743 + "findings": [
1744 + {
1745 + "value": "522,769",
1746 + "label": "Quebec sales matched to assessment rolls, 2021–2026"
1747 + },
1748 + {
1749 + "value": "625",
1750 + "label": "municipalities with parcel-level roll matching"
1751 + },
1752 + {
1753 + "value": "−0.34 / −0.09",
1754 + "label": "assessment-ratio elasticity, lenient trim versus IAAO rule"
1755 + },
1756 + {
1757 + "value": "17",
1758 + "label": "province-wide COD under the IAAO outlier rule"
1759 + },
1760 + {
1761 + "value": "upper 80%",
1762 + "label": "share of each market's price distribution assessed proportionally"
1763 + },
1764 + {
1765 + "value": "6–65%",
1766 + "label": "overpayment by the bottom decile depending on outlier treatment"
1767 + }
1768 + ],
1769 + "contributions": [
1770 + {
1771 + "title": "First province-wide ratio study for a Canadian province",
1772 + "description": "522,769 sales are matched at the parcel level to the rolls of 625 municipalities, with municipality × roll × sale-year cells (2,884) that separate statutory reference-date drift from within-market inequity by design."
1773 + },
1774 + {
1775 + "title": "Regressivity is bounded, not pinned down",
1776 + "description": "The Cheng fixed-effects elasticity is −0.344 under a 1/99 trim and −0.088 under the IAAO 1.5 × IQR rule; the Clapp rank instrument gives −0.083 and −0.013 (insignificant); the Kochin–Parks bracket [0.656, 1.137] contains one."
1777 + },
1778 + {
1779 + "title": "Failure is concentrated in the bottom quintile",
1780 + "description": "Piecewise slopes by within-market price quintile are 0.07, 0.89, 0.71, 1.00 and 0.95 on the baseline sample; 53% of bottom-decile sales are IAAO outliers, and the registry carries no arm's-length flag to settle their status."
1781 + },
1782 + {
1783 + "title": "IAAO standards are met by most markets once outliers are removed",
1784 + "description": "Province-wide COD falls from 28.5 to 17.0 and PRB moves from −0.029 to +0.025; the ten largest cities post CODs of 9.2–12.3; municipalities with PRB below −0.05 drop from 95% to 24%."
1785 + },
1786 + {
1787 + "title": "Robust anatomy of inequity",
1788 + "description": "Condominiums are assessed proportionally (trimmed elasticity 0.00) while single-family homes (−0.12), plexes (−0.12), land-heavy (−0.14) and old buildings (−0.14) stay mildly regressive; Montréal is the least regressive large city, not an exception."
1789 + },
1790 + {
1791 + "title": "Staleness widens bias at the margin, not structurally",
1792 + "description": "The within-municipality PRB of the trimmed sample moves from −0.04 in 2021–2022 to zero from 2024; the province-wide swing is compositional; the modest redistribution implied is 5–7% overpayment by the cheapest decile and 3–5% underpayment by the most expensive."
1793 + }
1794 + ],
1795 + "data": [
1796 + {
1797 + "title": "Matched sales and assessment rolls",
1798 + "description": "745,119 Quebec residential transactions (2021–2026) matched to the triennial assessment rolls, filtered to 522,769 usable sales (471,148 under the IAAO rule) across 625 municipalities with roll vintage, land and building values, class, age and land share at the parcel level."
1799 + }
1800 + ],
1801 + "methodology": [
1802 + {
1803 + "title": "Cheng fixed-effects ratio regression",
1804 + "description": "ln AV on ln SP within municipality × roll × sale-year cells by absorbing least squares; γ = β − 1 is the elasticity of the assessment ratio, with municipality-clustered standard errors (625 clusters)."
1805 + },
1806 + {
1807 + "title": "Clapp rank instrument and measurement-error bounds",
1808 + "description": "A rank instrument purges attenuation from transaction noise; the Kochin–Parks reverse-regression bracket and the implied noise shares make the errors-in-variables problem explicit (Appendix A)."
1809 + },
1810 + {
1811 + "title": "IAAO ratio-study battery under five trimming rules",
1812 + "description": "Median ratio, COD, PRD and PRB with bootstrap intervals for every municipality with at least 100 sales, recomputed under baseline 1/99, 3 × IQR, 1.5 × IQR (IAAO), 2.5/97.5 and 5/95 rules."
1813 + },
1814 + {
1815 + "title": "Piecewise, quantile and subgroup profiles",
1816 + "description": "Piecewise-linear slopes by within-market price quintile, conditional quantiles of ln AV, subgroup elasticities by class, age, land share and market size, and horizontal-dispersion regressions."
1817 + },
1818 + {
1819 + "title": "Intra-municipal tax-shift calculation",
1820 + "description": "Excess tax by within-market price decile under three sorting variables (price, assessed value, IAAO blended proxy) and two benchmarks, bracketing the redistribution at stake."
1821 + }
1822 + ],
1823 + "reproducibility": [
1824 + "Six numbered Python scripts (sample build, IAAO statistics, regressions, figures, tables, revision analyses); the revision script runs in 38 seconds on Apple silicon",
1825 + "Every CSV in results/reproduced/, including 16 rev_*.csv files feeding the trimming grid, spline, tax-shift, city and bounds tables",
1826 + "Ten figures and all LaTeX tables generated from results; anonymised PDF (main_anon.pdf) for journal submission",
1827 + "Full LaTeX source of the 45-page paper; pre-revision PDF, review report and changelog archived in docs/"
1828 + ],
1829 + "keywords": [
1830 + "property tax",
1831 + "assessment ratio",
1832 + "vertical equity",
1833 + "regressivity",
1834 + "ratio study",
1835 + "measurement error",
1836 + "IAAO standards",
1837 + "Quebec"
1838 + ],
1839 + "repo": "https://git.spboucher.ai/wp10_uqo",
1840 + "jel": "H71, H22, R51, R31"
1841 + },
1842 + {
1843 + "slug": "wp11",
1844 + "hero": {
1845 + "headline": "Controls and the holdout matter more than functional form",
1846 + "subheadline": "Twenty hedonic models on 514,212 Quebec sales: dropping spatial fixed effects costs ten error points, splines beat linear by under two, and boosting's 3.2-point lead under random validation shrinks to an insignificant 0.5 point when the test set lies in the future."
1847 + },
1848 + "abstract": [
1849 + "Fifty years after Rosen (1974), applied hedonic practice still rests on specification choices (functional form, temporal controls, spatial controls, estimation method) that are made by convention and rarely stress-tested jointly. Using 514,212 Quebec residential sales (2021–2026) whose structural descriptors come from the provincial assessment roll, this paper runs a controlled horse race of twenty hedonic models organised along those four axes, scoring every model on identical holdouts and on the same price-level scoreboard with a single retransformation rule, with a municipality-clustered bootstrap interval and a Diebold–Mariano-type test on every comparison.",
1850 + "Four lessons emerge. Functional form is second-order: from linear to splines the median absolute error spans 17.6–19.2%, and the data-driven Box–Cox exponent gains half a point. Controls are first-order: dropping spatial fixed effects costs ten error points; a ~1 km grid used alone overfits, but nested within municipalities it becomes the most accurate linear specification, so the bias–variance trade-off in absorbing location is resolved by hierarchy, not coarseness. Under random validation the estimation method dominates: gradient boosting reaches 14.4%, an 18% improvement over the best linear model (95% interval 1.9–4.7 points) that survives holdouts grouped by parcel and by building and comes almost entirely from a flexible surface over space.",
1851 + "Under a forward-in-time holdout (train before 2025, test 2025–2026) every model degrades and the gap shrinks to half a point and to insignificance. Decomposing the error shows that relative pricing skill transfers forward intact (re-levelled, boosting leads by 2.9 points) while every frozen model under-prices the typical future sale by 10–20%, the ensembles slightly more. Random cross-validation hides this staleness cost; it does not, as a previous version of this paper claimed on the basis of an asymmetric retransformation, reverse the ranking. Constant-quality indices agree across functional forms to within 2.5 points but a boosted imputation index rises nine points less, and the boosted model's implicit age and area profiles share the shape of the quadratic OLS gradients with a milder depreciation path."
1852 + ],
1853 + "findings": [
1854 + {
1855 + "value": "514,212",
1856 + "label": "roll-matched Quebec residential sales, 2021–2026"
1857 + },
1858 + {
1859 + "value": "20",
1860 + "label": "hedonic models scored on one price-level scoreboard"
1861 + },
1862 + {
1863 + "value": "~10 pp",
1864 + "label": "MdAPE cost of dropping spatial fixed effects"
1865 + },
1866 + {
1867 + "value": "17.6–19.2%",
1868 + "label": "MdAPE range across six functional forms"
1869 + },
1870 + {
1871 + "value": "14.4% vs 17.6%",
1872 + "label": "boosting versus best linear MdAPE, random holdout"
1873 + },
1874 + {
1875 + "value": "0.5 pp",
1876 + "label": "insignificant boosting lead under the forward-in-time holdout"
1877 + }
1878 + ],
1879 + "contributions": [
1880 + {
1881 + "title": "A controlled experiment with inference on every comparison",
1882 + "description": "Six functional forms, a time-FE ladder, a spatial-FE ladder and six estimation methods face the same sales, attributes, retransformation rule and holdouts, with municipality-clustered bootstrap intervals (B = 500) and Diebold–Mariano-type tests."
1883 + },
1884 + {
1885 + "title": "Spatial controls are first-order, hierarchy resolves the trade-off",
1886 + "description": "Removing spatial fixed effects raises MdAPE from 19.2% to 28.8%; a ~1.1 km grid alone overfits (21.0%), but nested within municipalities it is the best linear specification at 17.3%."
1887 + },
1888 + {
1889 + "title": "Functional form is second-order",
1890 + "description": "Linear to splines spans 17.6–19.2% under random validation; splines beat semi-log by 1.4 points and the profile-likelihood Box–Cox exponent (λ = 0.25–0.35) gains half a point."
1891 + },
1892 + {
1893 + "title": "Boosting's gain is real and spatial",
1894 + "description": "Gradient boosting reaches 14.4% against 17.6% for splines (−3.2 pp, interval −4.7 to −1.9, t = −3.6); the gap is unchanged in parcel- and building-grouped holdouts, and boosting without coordinates falls to 23.2%."
1895 + },
1896 + {
1897 + "title": "Forward validation reveals a staleness cost, not a reversal",
1898 + "description": "Trained before 2025 and tested on 2025–2026, boosting scores 21.2% versus 21.7% for splines; its staleness cost is 7.5 points against 4.7 for splines, but re-levelled it leads by 2.9 points, and its lead erodes from 1.5 points in 2025Q1 to zero by 2026Q3."
1899 + },
1900 + {
1901 + "title": "Economic objects are stable across forms",
1902 + "description": "Constant-quality monthly indices from four linear forms agree within 2.5 points, while the boosted imputation index ends 9 points lower (160.5 versus 168.6–171.0); boosted age profiles are 0.10–0.13 log points flatter after age 20."
1903 + }
1904 + ],
1905 + "data": [
1906 + {
1907 + "title": "Roll-matched Quebec sales",
1908 + "description": "514,212 residential transactions (2021–2026) filtered from 745,119 raw sales and matched to the provincial assessment roll for assessor-grade attributes (floor area, lot, age, storeys, units, class, physical link); mean price $440,207, median $397,375; 20.6% of sales are resales of the same parcel and 30.3% share exact coordinates with another sale."
1909 + }
1910 + ],
1911 + "methodology": [
1912 + {
1913 + "title": "Four design axes, twenty models",
1914 + "description": "Linear, semi-log, log-log, Box–Cox, quadratic and cubic-spline forms; none/year/quarter/month time effects; none/municipality/~5.5 km/~1.1 km spatial effects plus nested grids; OLS, cross-validated ridge, random forest, gradient boosting with and without coordinates, and spatial k-NN comparables."
1915 + },
1916 + {
1917 + "title": "One price-level scoreboard",
1918 + "description": "Every transformed prediction is inverted with the same median rule (Duan smearing reported as robustness) and scored on MdAPE, MAPE, RMSE_ln and R²_ln."
1919 + },
1920 + {
1921 + "title": "Two holdouts plus grouped and same-period splits",
1922 + "description": "A random 80/20 split and a forward-in-time split (train before 2025, test 2025–2026 with carry-forward time effects), complemented by parcel- and building-grouped holdouts, a same-period comparison, re-levelling and quarterly tracking."
1923 + },
1924 + {
1925 + "title": "Clustered bootstrap and predictive-accuracy tests",
1926 + "description": "Municipality-clustered bootstrap intervals for each model's MdAPE and for differences, and Diebold–Mariano/Giacomini–White tests with clustered standard errors on absolute-percentage and squared-log losses."
1927 + },
1928 + {
1929 + "title": "Extensions",
1930 + "description": "Constant-quality indices, implicit age and area profiles, learning curves, accuracy by market segment, a fine Box–Cox grid and four boosting hyper-parameter configurations."
1931 + }
1932 + ],
1933 + "reproducibility": [
1934 + "Six numbered Python scripts from the raw parquet to the 20-model horse race, extensions, robustness, 10 figures and 16 LaTeX tables; full run about 25 minutes on Apple silicon",
1935 + "The harness in src/wp11/models.py and inference in src/wp11/inference.py apply one retransformation rule to every model; no table is typed by hand",
1936 + "v1.0 results archived under the pre-revision-20261017 tag with the changed MdAPE values listed in docs/CHANGELOG-REVISION.md",
1937 + "Full LaTeX source of the 40-page paper with an anonymised version; pre-revision PDF and review report in docs/"
1938 + ],
1939 + "keywords": [
1940 + "hedonic price function",
1941 + "functional form",
1942 + "spatial fixed effects",
1943 + "machine learning",
1944 + "automated valuation model",
1945 + "forward-in-time holdout",
1946 + "house price index",
1947 + "Quebec"
1948 + ],
1949 + "repo": "https://git.spboucher.ai/wp11_uqo",
1950 + "jel": "R31, C21, C52, C53"
1951 + },
1952 + {
1953 + "slug": "wp12",
1954 + "hero": {
1955 + "headline": "Most of the machine-learning gain over HAR is just taking logs",
1956 + "subheadline": "On a common sample of 59,323 asset-days, log-HAR lowers HAR's one-day QLIKE by 25% on every instrument and ridge on an extended predictor set is the best forecaster (ratio 0.70); trees and deep learners trail, and the most accurate forecasters lose money net of costs."
1957 + },
1958 + "abstract": [
1959 + "The paper compares 27 forecasts of daily realized variance (six HAR specifications, GARCH, GJR, EGARCH and Realized GARCH, penalized regressions, random forests, XGBoost and LightGBM on two information sets, an LSTM, a Transformer, a LightGBM pooled across assets, and two combinations) on 25 liquid instruments from four asset classes (equities and ETFs, FX, cryptocurrencies, futures) over 2013–2026, under one protocol: the same subsampled 5-minute realized-variance target, a 1,000-day rolling window, horizons of 1, 5 and 22 days, QLIKE and MSE losses, Diebold–Mariano tests, model confidence sets, and a common evaluation sample of 59,323 asset-days.",
1960 + "Log-linear models dominate: log-HAR lowers the QLIKE of HAR in levels by 25% at one day on every instrument, and ridge regression on an extended predictor set is the best individual forecaster (ratio 0.70), significantly better than HAR on 24 of 25 instruments. Tree ensembles and deep learners trail the linear learners and fall behind HAR at one month; pooling across assets helps the trees but does not transfer to unseen assets outside a homogeneous class. Gains are largest for cryptocurrencies (0.62) and smallest for FX (0.85), grow with the noise of the realized measure, and shrink in the calm 2024–2026 period.",
1961 + "In volatility timing net of 5 basis points, smooth forecasters (LSTM, GARCH) earn higher Sharpe ratios and positive utility gains over HAR than the most accurate ones, which trade too much; at the 5% VaR level all forecasters are calibrated, at 1% none is. Version 0.2 is a complete working draft: window-length sensitivity and a tick-level cleaning of the cryptocurrency bars remain to be run."
1962 + ],
1963 + "findings": [
1964 + {
1965 + "value": "27",
1966 + "label": "forecasting models raced under one protocol"
1967 + },
1968 + {
1969 + "value": "25",
1970 + "label": "instruments from four asset classes, 2013–2026"
1971 + },
1972 + {
1973 + "value": "59,323",
1974 + "label": "asset-days in the common evaluation sample"
1975 + },
1976 + {
1977 + "value": "−25%",
1978 + "label": "one-day QLIKE of log-HAR relative to HAR, all instruments"
1979 + },
1980 + {
1981 + "value": "0.70",
1982 + "label": "QLIKE ratio of ridge (extended set) to HAR"
1983 + },
1984 + {
1985 + "value": "24 / 25",
1986 + "label": "instruments where ridge beats HAR significantly"
1987 + }
1988 + ],
1989 + "contributions": [
1990 + {
1991 + "title": "One design across four asset classes",
1992 + "description": "The same 5-minute subsampled target, 1,000-day rolling window, three horizons, losses, inference and common sample are applied to equities and ETFs, FX, cryptocurrencies and futures, so the HAR-versus-machine-learning disagreement in the literature is tested on comparable terms."
1993 + },
1994 + {
1995 + "title": "The log transformation explains most of the gain",
1996 + "description": "Log-HAR achieves QLIKE ratios of 0.75, 0.63 and 0.79 to HAR at 1, 5 and 22 days and lower loss on 25 of 25 instruments at one day; ridge on 18 predictors adds about five points more (0.70) and is in the 90% model confidence set for every instrument."
1997 + },
1998 + {
1999 + "title": "Nonlinear learners do not pay",
2000 + "description": "Random forests, XGBoost and LightGBM (ratios 0.79–0.85 at one day, 0.95–1.16 at one month), the LSTM (1.33) and the Transformer (1.89) are worse than the linear learners everywhere; pooling across assets rescues the trees but transfer to an unseen asset works only within FX."
2001 + },
2002 + {
2003 + "title": "Gains track the noise of volatility",
2004 + "description": "The ridge-to-HAR ratio is 0.62 for cryptocurrencies, 0.72 for equities, 0.78 for futures and 0.85 for FX; it rises from 0.51 before 2020 to 0.88 in 2024–2026 and from 0.43 with 1-minute RV to 0.86 with a realized kernel."
2005 + },
2006 + {
2007 + "title": "Accuracy and economic value diverge",
2008 + "description": "Net of 5 basis points the LSTM (Sharpe 0.38) and daily GARCH (0.35) beat HAR (0.34) in volatility timing while ridge (0.30) and LightGBM (0.26) trade too much, losing 47 to 121 basis points a year relative to HAR; at 5% VaR every forecaster is calibrated, at 1% all fail."
2009 + },
2010 + {
2011 + "title": "Code corrections that changed the GARCH results",
2012 + "description": "The revision found and fixed frozen GARCH forecasts between re-estimations, a one-day conditioning lag, a mis-studentised model confidence set, a missing insanity filter on the GARCH family and a collapsed common sample; the econometric family was re-run three times."
2013 + }
2014 + ],
2015 + "data": [
2016 + {
2017 + "title": "HF Market Data 1-minute bars, 25 instruments",
2018 + "description": "93.8 million 1-minute bars plus daily bars for 25 assets and the VIX, 1 January 2013 to 31 July 2026, from the HF Market Data API (1.6 GB parquet cache, not committed); equities use regular trading hours, futures, FX and crypto the Eastern calendar day. The headline target is 5-minute subsampled realized variance (five offset grids); 1-minute RV, realized kernel, bipower variation, quarticities, jumps and semivariances are alternatives. Panel of 84,898 asset-days before the common-sample restriction."
2019 + }
2020 + ],
2021 + "methodology": [
2022 + {
2023 + "title": "Realized measures and target",
2024 + "description": "Subsampled 5-minute realized variance as the common target, with realized kernel (Parzen), bipower variation, tripower and realized quarticity, BNS jump tests and semivariances computed in src/wp12/realized_vol.py."
2025 + },
2026 + {
2027 + "title": "Econometric and machine-learning forecasters",
2028 + "description": "HAR, HAR-J, HAR-CJ, SHAR, HARQ and log-HAR as direct projections; GARCH, GJR and EGARCH refiltered daily with an RV-unit mapping; Realized GARCH by QML; ridge, LASSO, elastic net, random forest, XGBoost and LightGBM on the HAR set and an extended 18-predictor set with walk-forward tuning; LSTM and Transformer with quarterly refits; a pooled LightGBM with asset identifiers; mean and inverse-MSE combinations."
2029 + },
2030 + {
2031 + "title": "Rolling out-of-sample evaluation",
2032 + "description": "1,000-day rolling windows refit every 22 days, horizons of 1, 5 and 22 days, QLIKE and MSE on a common sample per asset and horizon, Diebold–Mariano tests with Bartlett bandwidth, model confidence sets (T_max, stationary bootstrap, 5,000 draws), Mincer–Zarnowitz and encompassing regressions."
2033 + },
2034 + {
2035 + "title": "Robustness and economic value",
2036 + "description": "Sub-periods, alternative RV measures, implausible crypto days, leave-one-asset-out transfer and Giacomini–Rossi fluctuation tests; volatility timing with a 10% target and 0–20 basis-point costs, Fleming–Kirby–Ostdiek utility gains, and 1% and 5% VaR backtests with Kupiec and Christoffersen tests."
2037 + }
2038 + ],
2039 + "reproducibility": [
2040 + "Nine numbered scripts from data download to paper build (01_download_data … 09_build_paper), driven by config.yaml; 91_key_numbers.py prints every number quoted in the text",
2041 + "18 pytest tests on the realized-measure and evaluation code; 25 result CSVs, 12 generated LaTeX tables and 9 figures",
2042 + "Pipeline re-executed end to end on 2026-10-17 for the econometric, pooled and evaluation stages; tabular ML and deep forecasts reused from the August 2026 run with unchanged code",
2043 + "Full LaTeX source with 73 Crossref-verified references (docs/bib-verification.csv); open items for version 1.0 listed in the README and conclusion"
2044 + ],
2045 + "keywords": [
2046 + "realized volatility",
2047 + "HAR model",
2048 + "GARCH",
2049 + "machine learning",
2050 + "forecast evaluation",
2051 + "model confidence set",
2052 + "volatility timing",
2053 + "cryptocurrency",
2054 + "high-frequency data"
2055 + ],
2056 + "repo": "https://git.spboucher.ai/wp12_uqo",
2057 + "jel": "C22, C45, C53, G17"
2058 + },
2059 + {
2060 + "slug": "wp13",
2061 + "hero": {
2062 + "headline": "The model plans and writes; auditable code computes every number",
2063 + "subheadline": "VQuant exposes 245 tools, two market-data providers (249 functions over 233 endpoints), three web-research services and a sandbox with 29 scientific packages through a recursive, batched, streaming tool-calling loop; the paper specifies the architecture and reports no accuracy, latency or cost measurements."
2064 + },
2065 + "abstract": [
2066 + "Large language models write fluently about finance, but fluency does not make a number correct, current, or reproducible. This paper is a design-science description of VQuant, a deployed web platform that couples a reasoning-capable LLM with a deterministic execution substrate so that a natural-language request (\"analyse NVDA's fundamentals and run a Monte Carlo simulation\") is turned into a report in which every figure is produced by live data retrieval or auditable code rather than by the model.",
2067 + "Four mechanisms are documented as implemented and audited line by line against the platform's source code (vquant.git, commit e2a148a): (i) an agentic orchestration loop that exposes 245 schema-defined tools, wrapping two market-data providers (249 typed functions over 233 endpoint paths), three web-research services, and a Python sandbox pre-loading 29 scientific packages, through a recursive, batched, streaming tool-calling protocol; (ii) a three-stage context-management procedure (per-result truncation at execution, a re-send cap, and adaptive summarisation under a token budget) that keeps long sessions within the model's window; (iii) a single-call computation convention that consolidates data acquisition, estimation, and figure rendering into one sandboxed execution; and (iv) a document pipeline that renders one Markdown artifact into PDF, DOCX, LaTeX Beamer, and spreadsheets with live formulas.",
2068 + "The paper reports no empirical evaluation: it contains no measurements of numerical accuracy, hallucination rates, latency, or cost, and makes no comparative claims. Its contribution is a reproducible specification of the architecture, an account of where the implementation departs from its own documentation, and a pre-registered evaluation protocol (benchmark, metrics, ablations, and baselines) that has not yet been executed."
2069 + ],
2070 + "findings": [
2071 + {
2072 + "value": "245",
2073 + "label": "schema-defined tools exposed to the model, 17 families"
2074 + },
2075 + {
2076 + "value": "249 / 233",
2077 + "label": "typed market-data functions over distinct endpoint paths"
2078 + },
2079 + {
2080 + "value": "29",
2081 + "label": "scientific packages pre-loaded by the Python sandbox"
2082 + },
2083 + {
2084 + "value": "3 stages",
2085 + "label": "context management: truncation, re-send cap, summarisation"
2086 + },
2087 + {
2088 + "value": "4",
2089 + "label": "output formats from one Markdown artifact"
2090 + },
2091 + {
2092 + "value": "0",
2093 + "label": "empirical measurements reported; evaluation protocol pending"
2094 + }
2095 + ],
2096 + "contributions": [
2097 + {
2098 + "title": "An implementation-level specification of an agentic loop",
2099 + "description": "A recursive, batched (5 calls, 500 ms spacing), streaming tool-calling protocol over 245 tools with tool-use/tool-result pairing and prompt-level routing by request complexity, documented with its actual constants (Algorithm 1)."
2100 + },
2101 + {
2102 + "title": "Context management with real constants",
2103 + "description": "Per-result truncation at 3,000/15,000 characters, a 5,000-character re-send cap and adaptive summarisation at a 10^5-token budget keeping the last 5 messages; the local token count is a BPE approximation, not an exact tokenizer, as the audit established."
2104 + },
2105 + {
2106 + "title": "A single-call computation convention",
2107 + "description": "Data acquisition, estimation and figure rendering are consolidated into one execution in a process-isolated Python child with a 90-second timeout and a 150-dpi figure sweep, so that no numeric value originates from the model."
2108 + },
2109 + {
2110 + "title": "One artifact, four documents",
2111 + "description": "A single Markdown artifact is rendered to PDF (Puppeteer and KaTeX), DOCX (Pandoc), LaTeX Beamer (a second model call and two pdflatex passes) and XLSX with live formulas."
2112 + },
2113 + {
2114 + "title": "An audit that corrects the system's own description",
2115 + "description": "Line-by-line comparison with commit e2a148a corrected the tool count, the tokenizer claim, the reachability of pre-built numerical routines (only the option pricer is in the catalogue), the multi-provider model layer, and removed three features (SSO, daily caps, bring-your-own-keys) absent from any committed version; all unmeasured latency claims were withdrawn."
2116 + },
2117 + {
2118 + "title": "A pre-registered evaluation protocol",
2119 + "description": "Task suite, metrics including an unsupported-number rate, ablations of the single-call convention and summarisation, baselines and controls are specified in Section 6.6 and have not been executed; any claim of superiority waits on them."
2120 + }
2121 + ],
2122 + "data": [
2123 + {
2124 + "title": "The VQuant source code (no empirical data)",
2125 + "description": "The paper describes a software system rather than a dataset: the TypeScript/Python platform maintained in the spbgit repository vquant.git at commit e2a148a (2 October 2026), deployed on the MacLustr cluster. Audit facts are extracted by docs/verify/audit_vquant_code.py into code_facts.json and tool_families.json."
2126 + }
2127 + ],
2128 + "methodology": [
2129 + {
2130 + "title": "Design-science framing",
2131 + "description": "Following Hevner et al. (2004) and Gregor and Hevner (2013), the paper contributes a situated implementation and a demonstration; the evaluation step is specified but explicitly not performed."
2132 + },
2133 + {
2134 + "title": "Mechanism specification",
2135 + "description": "Transport, recursive loop (Algorithm 1), batching, routing, three-stage context management (Algorithm 2), token-accounting correction and the single-call computation convention are stated at implementation level with their constants."
2136 + },
2137 + {
2138 + "title": "Claim-by-claim code audit",
2139 + "description": "Every architectural statement is mapped to a source file and line in an appendix table generated from a read-only clone of the platform repository; statements contradicted by the code were corrected or removed."
2140 + },
2141 + {
2142 + "title": "Proposed evaluation protocol",
2143 + "description": "A benchmark of financial-analysis tasks, metrics (numerical accuracy, unsupported-number rate, latency, cost), ablations and baselines against simpler agents, written as a pre-registration for future execution."
2144 + }
2145 + ],
2146 + "reproducibility": [
2147 + "docs/verify/audit_vquant_code.py regenerates paper/tables/tools.tex and the code facts quoted in the text from a clone of vquant.git at commit e2a148a",
2148 + "No empirical pipeline, data or measurements: the paper has no src/, scripts/ or data/ directories by design",
2149 + "Full LaTeX source of the 41-page paper with TikZ architecture diagrams compiled inline; pre-revision v1.0 PDF, review report and changelog in docs/",
2150 + "Appendix E maps every claim to a source file; the evaluation protocol of Section 6.6 is specified for future execution"
2151 + ],
2152 + "keywords": [
2153 + "large language models",
2154 + "LLM agents",
2155 + "tool use",
2156 + "design science",
2157 + "quantitative finance",
2158 + "automated report generation",
2159 + "reproducibility",
2160 + "code sandbox",
2161 + "software architecture"
2162 + ],
2163 + "repo": "https://git.spboucher.ai/wp13_uqo",
2164 + "jel": "C45, C63, C88, G17, O33"
2165 + },
2166 + {
2167 + "slug": "wp14",
2168 + "hero": {
2169 + "headline": "Façade photographs add more to a hedonic model than text does",
2170 + "subheadline": "On 19,123 Quebec listings two schema-constrained AI layers lift R² from 0.717 to 0.755, 3.0 of the 3.8 points coming from the façade image; the best ensemble reaches an out-of-sample R² of 0.779 with a 14.0% median absolute error."
2171 + },
2172 + "abstract": [
2173 + "Hedonic house-price models are bounded by the characteristics an analyst can measure, yet much of a home's value is recorded only in the free-text description and photographs of its listing. This paper treats general-purpose AI models as a feature-engineering technology: a large language model converts each description into structured property specifications and a vision–language model converts the façade photograph into visual attributes, both under strict JSON schemas that make the outputs auditable econometric variables.",
2174 + "On 19,123 Quebec house listings, the two AI layers raise the R² of an already rich log-linear hedonic specification with municipality fixed effects from 0.717 to 0.755, 3.0 of the 3.8 points coming from the façade photograph, and they raise the out-of-sample R² of every estimator from OLS to gradient boosting and neural networks (best ensemble: R² 0.779, median absolute error 14.0%). The vision layer adds more to the linear model than switching from OLS to a random forest does. The extracted attributes carry economically sensible implicit prices — 2.3% per point of curb appeal, 3–4% for brick or stone façades — that survive municipality-clustered inference, a cluster bootstrap, postal-code fixed effects, and a control for the language of the description, which itself carries a 7% conditional premium for English listings.",
2175 + "The paper also documents that for-sale-by-owner descriptions embed the asking price, so that the text layer's gain is credible only on the brokered platform, and that spatially grouped cross-validation lowers all accuracy levels by three to five points while preserving the AI increments."
2176 + ],
2177 + "findings": [
2178 + {
2179 + "value": "19,123",
2180 + "label": "Quebec house listings, Realtor.ca and DuProprio, July 2026"
2181 + },
2182 + {
2183 + "value": "0.717 → 0.755",
2184 + "label": "in-sample R² with the text and vision layers"
2185 + },
2186 + {
2187 + "value": "3.0 of 3.8 pt",
2188 + "label": "R² gain attributable to the façade photograph"
2189 + },
2190 + {
2191 + "value": "0.779",
2192 + "label": "out-of-sample R² of the best ensemble, MdAPE 14.0%"
2193 + },
2194 + {
2195 + "value": "+2.3%",
2196 + "label": "implicit price per point of curb appeal (1–10 scale)"
2197 + },
2198 + {
2199 + "value": "19.7%",
2200 + "label": "listings whose description embedded the asking price"
2201 + }
2202 + ],
2203 + "contributions": [
2204 + {
2205 + "title": "AI models as auditable feature engineering",
2206 + "description": "Schema-constrained decoding turns LLM and vision outputs into ordinary regressors — 18 text-derived and 21 façade-derived variables — at a marginal cost of a fraction of a cent per listing."
2207 + },
2208 + {
2209 + "title": "The vision layer beats a change of estimator",
2210 + "description": "Adding façade attributes raises the out-of-sample R² of OLS by 5.8 points (0.628 to 0.686), more than the 4.9 points gained by switching from OLS to a random forest; gradient boosting moves from 0.743 to 0.770."
2211 + },
2212 + {
2213 + "title": "Economically sensible implicit prices",
2214 + "description": "Curb appeal is worth 2.3% per point (cluster-bootstrap 95% CI [0.1%, 3.6%]), brick and stone façades 3.4% and 3.7%, vinyl −4.4%; the estimates hold under municipality-clustered SEs, HC3, a 299-replication cluster bootstrap and 250 postal-area fixed effects."
2215 + },
2216 + {
2217 + "title": "Asking-price leakage quantified, not papered over",
2218 + "description": "All 3,767 DuProprio descriptions embed the platform's structured sheet including the asking price; the text layer's gain is therefore reported by platform and is credible only on Realtor.ca (+0.6 point in-sample versus +3.3 on DuProprio). The vision layer is not exposed."
2219 + },
2220 + {
2221 + "title": "Spatially grouped validation preserves the AI increments",
2222 + "description": "Holding out entire postal areas lowers every accuracy level by three to five points (GBM 0.690 → 0.703 → 0.734) but leaves the gains of both layers intact."
2223 + },
2224 + {
2225 + "title": "Language of the description priced",
2226 + "description": "44.6% of descriptions are French, 43.1% English and 12.3% mixed; an English listing carries a +7.2% conditional premium within municipality, and the AI coefficients are unchanged when it is controlled for."
2227 + }
2228 + ],
2229 + "data": [
2230 + {
2231 + "title": "Quebec house listings",
2232 + "description": "27,302 Realtor.ca and 7,693 DuProprio listings crawled on 3 July 2026; single-family houses and townhouses with a valid price, outliers removed, giving 19,123 observations (15,356 Realtor.ca, 3,767 DuProprio) with living area, lot, rooms, year built, type, municipality, postal code and 15 bilingual keyword amenities."
2233 + },
2234 + {
2235 + "title": "AI feature layers",
2236 + "description": "Each description parsed by a large language model into a fixed JSON schema of specifications (renovations, finishes, systems, extras; 18 regressors) and each façade photograph parsed by a vision–language model into 12 visual fields (cladding, storeys, garage, curb appeal; 21 regressors); exteriors detected for 95.8% of listings, raw outputs archived."
2237 + }
2238 + ],
2239 + "methodology": [
2240 + {
2241 + "title": "Schema-constrained extraction",
2242 + "description": "Strict JSON schemas with retry and backoff for both layers; text decoded at temperature zero, vision at the API default; regressors regenerable from archived outputs without API calls."
2243 + },
2244 + {
2245 + "title": "Log-linear hedonic regression with municipality fixed effects",
2246 + "description": "Structural, amenity, room and listing controls plus the two AI blocks; inference by HC3, municipality-clustered standard errors (195 clusters), a pairs cluster bootstrap and postal-area (RTA) fixed effects."
2247 + },
2248 + {
2249 + "title": "Ablation by feature block and estimator",
2250 + "description": "OLS, random forest, histogram gradient boosting and a residual neural network with municipality and type embeddings compared under identical random 5-fold splits and under RTA-grouped spatial folds."
2251 + },
2252 + {
2253 + "title": "Equal-weight blend with weight checks",
2254 + "description": "The best ensemble averages the robust neural network and gradient boosting with a weight fixed a priori at 0.5; a weight grid (0.771–0.780) and a nested-CV weight (0.775) bracket the reported 0.779."
2255 + },
2256 + {
2257 + "title": "Indirect validation of extraction error",
2258 + "description": "Agreement of LLM attributes with keyword rules (κ 0.96 for garage down to 0.50 for waterfront) and of vision attributes with text and structured fields; unreliable fields relegated to the role of controls."
2259 + }
2260 + ],
2261 + "reproducibility": [
2262 + "End-to-end Python pipeline of 19 numbered scripts from API extraction to LaTeX tables; econometrics and ML run in about 25 minutes on an Apple-silicon laptop",
2263 + "Revision analyses (language, leakage, RTA fixed effects, spatial CV, cluster bootstrap, agreement) in scripts/18 with outputs in results/revision_2026-10-17.json and tables from scripts/19",
2264 + "Six figures and eleven LaTeX tables regenerated from results; full LaTeX source in paper/ with sections and bibliography (53 references, DOIs verified)",
2265 + "Pre-revision version 2.0 frozen as docs/main_pre-revision-20261017.pdf with a full audit trail in docs/CHANGELOG-REVISION.md and docs/REVIEW-REPORT.md"
2266 + ],
2267 + "keywords": [
2268 + "hedonic pricing",
2269 + "house prices",
2270 + "large language models",
2271 + "computer vision",
2272 + "text as data",
2273 + "automated valuation",
2274 + "façade photographs",
2275 + "curb appeal",
2276 + "Quebec"
2277 + ],
2278 + "repo": "https://git.spboucher.ai/wp14_uqo",
2279 + "jel": "R31, C21, C55, R21"
2280 + },
2281 + {
2282 + "slug": "wp15",
2283 + "hero": {
2284 + "headline": "Modern markets absorb the idiosyncratic and synchronize on the systemic",
2285 + "subheadline": "Across 1.85 million isolated ±6σ minute events, market-wide spillover stays between zero and about 2% of its base rate; the twenty-neighbour network response rose from +28 to +90 ×10⁻⁴ between the 1990s and the 2020s."
2286 + },
2287 + "abstract": [
2288 + "This paper studies how intraday shocks travel across U.S. equities using 7.69 billion one-minute bars covering the full listed and delisted cross-section from 1992 to 2026. Three findings emerge. First, isolated idiosyncratic shocks barely spread: across 1.85 million ±6σ one-minute events, the abnormal share of other securities with an extreme standardized return, measured against matched within-stock placebos, lies between zero and about 2% of its base rate at horizons of 1 to 120 minutes, in every decade.",
2289 + "Second, transmission is concentrated and has intensified: the shocked stock's twenty closest lagged-correlation neighbours respond (+64.0 ×10⁻⁴ in the probability of an extreme move at five minutes), and this network channel roughly tripled between the 1990s and the 2020s while market-wide spillover stayed bounded near zero — a rise that partly reflects sharper identification of the network as the Epps effect receded.",
2290 + "Third, common information is impounded far faster (one-minute pairwise correlation 0.004 in 1996 versus 0.30 in 2015), ordinary extreme minutes became an order of magnitude rarer after decimalization, and severe co-jump minutes — more than a tenth of the universe moving together — are roughly ten times more frequent in 2008 and 2020 than in calm years. Modern markets absorb the idiosyncratic, transmit it narrowly along statistical linkages, and synchronize on the systemic."
2291 + ],
2292 + "findings": [
2293 + {
2294 + "value": "7.69 bn",
2295 + "label": "one-minute bars on 81,306 tickers, 1992–2026"
2296 + },
2297 + {
2298 + "value": "1.85 M",
2299 + "label": "isolated ±6σ minute events matched to within-stock placebos"
2300 + },
2301 + {
2302 + "value": "≤ 2%",
2303 + "label": "market-wide spillover as a share of its base rate"
2304 + },
2305 + {
2306 + "value": "+64 ×10⁻⁴",
2307 + "label": "twenty-neighbour extreme-move response at five minutes"
2308 + },
2309 + {
2310 + "value": "+28 → +90",
2311 + "label": "network channel (×10⁻⁴), 1990s to 2020s"
2312 + },
2313 + {
2314 + "value": "0.004 → 0.30",
2315 + "label": "one-minute pairwise correlation, 1996 versus 2015"
2316 + }
2317 + ],
2318 + "contributions": [
2319 + {
2320 + "title": "The longest minute-level panel of U.S. equities",
2321 + "description": "7,686,275,524 bars, 81,306 tickers and 221,957 listing spells, including delisted securities, audited month by month and cleaned of splits, OHLC violations and duplicates before any variance-based measure."
2322 + },
2323 + {
2324 + "title": "Idiosyncratic shocks barely spread",
2325 + "description": "The market-wide response to a six-sigma minute is bracketed between zero (difference-in-differences, −1 to −3 ×10⁻⁴) and about 2% of the base rate (+19 ×10⁻⁴ under a pre-level-stratified estimator), at 1–120 minutes and in every decade."
2326 + },
2327 + {
2328 + "title": "Transmission became sharper, not wider",
2329 + "description": "The twenty closest lagged-correlation neighbours respond by +64.0 ×10⁻⁴ at five minutes (+51 in singleton minutes), and the channel rose from +28 in the 1990s to +90 ×10⁻⁴ in the 2020s; part of the rise reflects better identification of neighbours once the Epps effect receded."
2330 + },
2331 + {
2332 + "title": "Common information is impounded within the minute",
2333 + "description": "One-minute pairwise correlation rose from 0.004 in 1996 to 0.30 in 2015 and 0.21 in 2024, whether because incorporation accelerated or because asynchronous-trading attenuation receded — most likely both."
2334 + },
2335 + {
2336 + "title": "Fragility only in a bounded, post-decimalization sense",
2337 + "description": "Extreme minute returns above 1% fell from 518 to 4.8 per 10,000 between the 1990s and the 2010s, while severe co-jump minutes reach 91 and 84 per 1,000 in 2008 and 2020 against 9.5 in 2017–2019; the 1990s are not comparable because price discreteness manufactures synchronization."
2338 + },
2339 + {
2340 + "title": "Diagnostics that qualify the headline numbers",
2341 + "description": "Real events arrive in minutes of elevated activity (pre-event breadth +40.3 ×10⁻⁴ above placebos), only 22% of events are alone in their minute, and ticker-cluster bootstrap standard errors (4.7 versus 1.8 ×10⁻⁴) are reported alongside the baseline inference."
2342 + }
2343 + ],
2344 + "data": [
2345 + {
2346 + "title": "Minute-level OHLCV panel",
2347 + "description": "411 monthly Parquet files (87.7 GB) from the Hugging Face dataset mito0o852/OHLCV-1m, pinned at a fixed revision, January 1992 to March 2026: 7.69 billion bars, 81,306 tickers, 221,957 listing spells; delisted securities present (only 22–26% of 1992 tickers survive); exchange-calendar truncation, 6,944 split screens, OHLC and duplicate filters."
2348 + },
2349 + {
2350 + "title": "Shock-event sample",
2351 + "description": "1,854,384 isolated ±6σ one-minute events in the largest 500 securities, each matched to a within-stock placebo minute (1,766,058 placebos; match rate 95.2% overall, 71% in the 1990s), tracked at 1–120 minutes for breadth, intensity, signed response and network response."
2352 + }
2353 + ],
2354 + "methodology": [
2355 + {
2356 + "title": "Shock detection and matched placebos",
2357 + "description": "Standardized returns with intraday seasonal and robust scale factors; isolated ±6σ events compared with within-stock placebo minutes in a difference-in-differences design, complemented by a pre-level-stratified estimator that bounds the market-wide response from above."
2358 + },
2359 + {
2360 + "title": "Network response along lagged correlations",
2361 + "description": "Each shocked stock's twenty closest lagged-correlation neighbours versus all other receivers, by decade; the network differential is decomposed into its real and placebo components to separate transmission from identification."
2362 + },
2363 + {
2364 + "title": "Epps curves and co-jump counts",
2365 + "description": "Pairwise correlation as a function of sampling interval by year, and counts of securities with extreme standardized returns in the same minute, with a severity scale calibrated on calm, GFC and COVID periods."
2366 + },
2367 + {
2368 + "title": "Inference and robustness",
2369 + "description": "Event-clustered, month-cluster and ticker-cluster bootstrap standard errors; eleven robustness designs (post-decimalization, mid-day, ex-announcement, calm versus crisis, |z| ≥ 8 and 12, ex-non-common, placebo network) plus design-diagnostic and network-component tables."
2370 + }
2371 + ],
2372 + "reproducibility": [
2373 + "Python package and 18 numbered scripts (download, audit, cleaning, shocks, networks, co-jumps, tables, figures), parameters in YAML, 7 pytest tests; download and cleaning run on the MacLustr cluster",
2374 + "All 97 headline numbers generated as LaTeX macros from machine-readable estimate registries (scripts/15_build_results_macros.py)",
2375 + "Figures as CVD-safe vector PDFs with source CSVs; UQO working-paper, IRFA journal and anonymous manuscripts plus an 8-page Internet Appendix built from the same sources",
2376 + "Revision of 2026-10-17 documented in docs/CHANGELOG-REVISION.md with a frozen pre-revision PDF; 124 references verified by DOI on Crossref; analyses not executed listed explicitly"
2377 + ],
2378 + "keywords": [
2379 + "intraday shock propagation",
2380 + "market fragility",
2381 + "high-frequency data",
2382 + "contagion",
2383 + "co-jumps",
2384 + "Epps effect",
2385 + "market microstructure",
2386 + "decimalization",
2387 + "U.S. equities"
2388 + ],
2389 + "repo": "https://git.spboucher.ai/wp15_uqo",
2390 + "jel": "G10, G12, G14"
2391 + },
2392 + {
2393 + "slug": "wp16",
2394 + "hero": {
2395 + "headline": "Adjust, choose similar comparables, and use twenty of them, not three",
2396 + "subheadline": "On 189,354 masked target sales, the best transparent rule frozen in 2024 reaches an 11.1% median absolute error on 2025–2026, within 2.2 points of a quarterly-retrained gradient-boosting model that uses the assessed value."
2397 + },
2398 + "abstract": [
2399 + "The sales comparison approach is the dominant method of residential valuation, yet its operating rules — how many comparables to use, how far and how far back to search, which adjustments to apply, how to weight and reconcile — rest largely on professional convention. This paper subjects these rules to a population-scale, point-in-time backtest on 390,996 arm's-length single-family transactions in Quebec (2021–2026) matched to the municipal assessment roll. For each of 189,354 target sales in 2024–2026 the price is masked and 2,256 configurations of the approach, together with hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks, are applied using only information available on the valuation date; configurations are selected on 2024 and evaluated out of sample on 2025–2026, and paired differences are tested with a cluster bootstrap over municipality–quarter cells.",
2400 + "Adjustments dominate: the median absolute error of the five nearest sales falls from 18.1% with a time adjustment only to 13.0% with a hedonic grid, and a stylised conventional grid captures most of that gain. Selecting physically similar rather than merely nearby comparables delivers the same gain without any adjustment. The optimal number of comparables is about twenty once a grid is applied, with no penalty up to fifty; short windows are dominated and the search radius matters little. Weighting and reconciliation rules matter little.",
2401 + "The best transparent configuration chosen in 2024 reaches 11.1% out of sample, against 9.0% for a quarterly-retrained gradient-boosting model using the assessed value; the cost of transparency, 2.2 points, is precisely estimated and narrows in homogeneous submarkets, and the chosen rule ranks second among 1,370 configurations out of sample. Accuracy is governed primarily by submarket liquidity, and a wide-net rule retaining the most similar sales adapts to it on its own."
2402 + ],
2403 + "findings": [
2404 + {
2405 + "value": "390,996",
2406 + "label": "arm's-length single-family sales, Quebec, 2021–2026"
2407 + },
2408 + {
2409 + "value": "2,256",
2410 + "label": "configurations of the approach backtested point in time"
2411 + },
2412 + {
2413 + "value": "18.1% → 13.0%",
2414 + "label": "five-nearest-sales MdAPE, time-only versus hedonic grid"
2415 + },
2416 + {
2417 + "value": "~20",
2418 + "label": "optimal number of comparables once a grid is applied"
2419 + },
2420 + {
2421 + "value": "11.1%",
2422 + "label": "out-of-sample MdAPE of the best transparent rule"
2423 + },
2424 + {
2425 + "value": "2.2 pt",
2426 + "label": "cost of transparency versus gradient boosting at 9.0%"
2427 + }
2428 + ],
2429 + "contributions": [
2430 + {
2431 + "title": "A population-scale, point-in-time backtest",
2432 + "description": "Every target sale is valued with only the information available on its valuation date — a point-in-time index, quarterly hedonic grids and quarterly-retrained benchmarks — on 124,559 out-of-sample sales of 2025–2026, with 2,256 rule configurations selected on the separate 2024 validation year."
2433 + },
2434 + {
2435 + "title": "Adjustments dominate",
2436 + "description": "With five nearest sales, moving from a time adjustment only to a hedonic grid lowers the median absolute error from 18.1% to 13.0% (paired difference −5.1 points, 95% CI [−5.26; −4.88]); a stylised conventional grid captures most of that gain."
2437 + },
2438 + {
2439 + "title": "Similar beats near",
2440 + "description": "Five physically similar comparables without any physical adjustment reach the same 13.0% as five nearest with a grid, whereas five nearest within ten kilometres and a time adjustment stay at 17.6%."
2441 + },
2442 + {
2443 + "title": "More comparables than convention prescribes",
2444 + "description": "With a grid, the error falls from 16.5% (k = 1) to 13.8% (k = 3), 13.0% (k = 5), 12.4% (k = 10) and 12.1% at k = 20–50, in sparse and dense markets alike; windows should be wide and the radius barely matters."
2445 + },
2446 + {
2447 + "title": "The price of transparency is precisely estimated",
2448 + "description": "The best transparent rule (compromise selection, k = 30, 25 km, 24 months, hedonic grid) reaches 11.1% [10.8; 11.5] versus 9.0% [8.8; 9.2] for gradient boosting with the assessed value — a gap of 2.2 points (t = −36) that narrows to 1.4 points in the most homogeneous tercile. Validation-to-test rank correlation is 0.996 and the chosen rule ranks 2nd of 1,370."
2449 + },
2450 + {
2451 + "title": "Liquidity governs accuracy",
2452 + "description": "The same hedonic-grid rule yields a 10.2% error where at least 200 sales occur within 2 km per year and 25.3% where fewer than five do; weighting and reconciliation rules change the error by tenths of a point."
2453 + }
2454 + ],
2455 + "data": [
2456 + {
2457 + "title": "Vrai Prix transaction and assessment-roll snapshot",
2458 + "description": "Snapshot of 10 October 2026: 766,596 residential transactions and 3.75 million assessment-roll units; single-family sales matched to the roll and cleaned by logged rules R0–R11, from 456,950 to 390,996 sales (January 2021 to September 2026), median price $400,000."
2459 + },
2460 + {
2461 + "title": "Backtest splits",
2462 + "description": "Train 201,642 sales, validation 64,795 (2024) and test 124,559 (2025–2026); 189,354 target sales with masked prices; roll attributes (area, lot, age, storeys, attachment, assessed value) drive selection and adjustment."
2463 + }
2464 + ],
2465 + "methodology": [
2466 + {
2467 + "title": "Point-in-time index and quarterly hedonic grids",
2468 + "description": "A provincial and regional hedonic price index and hedonic adjustment grids re-estimated each quarter from 2024-01 to 2026-07 using only sales recorded before the valuation date."
2469 + },
2470 + {
2471 + "title": "Factorial backtest of the approach",
2472 + "description": "Selection rule (nearest, most similar, compromise, least adjusted) × number of comparables (1–50) × radius × window × adjustment scheme (none, time, conventional grid, hedonic, regional, local ridge, GBM, assessment ratio) × weighting × reconciliation, sharded across five MacLustr nodes."
2473 + },
2474 + {
2475 + "title": "Selection on 2024, evaluation on 2025–2026",
2476 + "description": "Error surfaces and marginal effects of each rule are mapped on the validation year; the winner is frozen and compared out of sample with hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks, then decomposed by liquidity, region, price stratum and property type."
2477 + },
2478 + {
2479 + "title": "Cluster-robust inference and selection audit",
2480 + "description": "Paired differences tested with a 999-replication bootstrap over municipality × quarter cells and Diebold–Mariano-type t statistics; sensitivity to four cluster definitions, to the arm's-length screen R8, and to seasonal bias by calendar quarter; stability of the validation-to-test ranking of all configurations."
2481 + }
2482 + ],
2483 + "reproducibility": [
2484 + "Pipeline 01_dataset → 06_report run on the MacLustr node M2U64 (about 30 minutes, or 15 shards over five nodes with run_distributed.sh), plus 07_revision_analyses.py on the laptop (about 80 s)",
2485 + "Every number in the paper is a LaTeX macro regenerated from out/<version>/ and results/revision-2026-10-17/ by make_paper_tables.py; 33 generated tables and 7 figures",
2486 + "French protocol and report (docs/PROTOCOLE.md, docs/RAPPORT.pdf), full LaTeX source with an anonymous variant for journal submission",
2487 + "Version 1.0 frozen as docs/main_pre-revision-20261017.pdf; the 2026-10-17 revision audit trail in docs/CHANGELOG-REVISION.md leaves every pipeline number unchanged"
2488 + ],
2489 + "keywords": [
2490 + "sales comparison approach",
2491 + "comparable selection",
2492 + "adjustment grid",
2493 + "automated valuation models",
2494 + "hedonic pricing",
2495 + "backtesting",
2496 + "residential real estate",
2497 + "Quebec"
2498 + ],
2499 + "repo": "https://git.spboucher.ai/wp16_uqo",
2500 + "jel": "R31, R32, C52, C53, G12"
2501 + },
2502 + {
2503 + "slug": "wp18",
2504 + "hero": {
2505 + "headline": "Transfers, not income, make housing tenure a sorting device for wealth",
2506 + "subheadline": "Among households with identical income processes, gifts and estates generate a 0.95 income-year wealth gap at mid-life and a rank–rank slope of 0.34 that vanishes without bequests; an unanticipated 10% inflation lowers rather than raises the real house price."
2507 + },
2508 + "abstract": [
2509 + "This paper asks under what conditions inherited wealth, mortgage leverage, nominal debt, housing supply and down-payment constraints generate persistent owner–renter wealth gaps among households with the same labour-income process, and what an unanticipated inflation changes. A three-stage benchmark with two-period generations yields a closed-form ownership-entry threshold that adds a liquidity component to the down-payment bound, a leverage multiplier whose break-even price growth is independent of the loan-to-value ratio, a closed-form intergenerational tenure persistence that is positive only if the house is net wealth at the end of life, existence and a uniqueness condition for the stationary equilibrium, and the accounting of a price-level surprise under three mortgage contracts.",
2510 + "A twelve-period overlapping-generations model with nominal annuity mortgages, gifts and estates, and segmented rental and construction sectors is solved in stationary general equilibrium and along perfect-foresight transitions under an illustrative calibration. Transfers generate a 0.95 income-year wealth gap at mid-life and a rank–rank wealth slope of 0.34 that vanishes without transfers; price adjustment removes 64–66% of the partial-equilibrium ownership response to the down-payment constraint; no multiple equilibria arise under near-fixed supply.",
2511 + "Because households are net nominal creditors, a 10% surprise moves the real house price by −0.45%, a fall where the benchmark predicts a rise; the contract ordering — fixed-rate above renewable term above adjustable — is confirmed. Three results that contradicted the benchmark are reported as results and the responsible assumptions identified."
2512 + ],
2513 + "findings": [
2514 + {
2515 + "value": "0.95 yr",
2516 + "label": "of income: owner–renter wealth gap at 40–44, equal incomes"
2517 + },
2518 + {
2519 + "value": "0.34",
2520 + "label": "rank–rank wealth slope with transfers, −0.01 without"
2521 + },
2522 + {
2523 + "value": "0.54 vs 0.35",
2524 + "label": "child owns at 35–39 given parent owner versus renter"
2525 + },
2526 + {
2527 + "value": "64–66%",
2528 + "label": "of the partial-equilibrium down-payment effect absorbed by prices"
2529 + },
2530 + {
2531 + "value": "−0.45%",
2532 + "label": "real house price after a 10% surprise, fixed-rate contract"
2533 + },
2534 + {
2535 + "value": "9",
2536 + "label": "closed-form propositions from the three-stage benchmark"
2537 + }
2538 + ],
2539 + "contributions": [
2540 + {
2541 + "title": "Ownership entry threshold with a liquidity component",
2542 + "description": "The transfer required to own exceeds the down-payment bound by the stage-one consumption of the marginal buyer; in general equilibrium prices absorb 64% (θ = 0.05) to 66% (θ = 0.40) of what the constraint would do to ownership at fixed prices."
2543 + },
2544 + {
2545 + "title": "Leverage scales exposure, not the break-even",
2546 + "description": "The leverage multiplier raises the sensitivity of terminal wealth to the resale price while the break-even price growth is independent of the loan-to-value ratio."
2547 + },
2548 + {
2549 + "title": "Intergenerational persistence in closed form",
2550 + "description": "Parent–child tenure persistence is the second eigenvalue of the dynasty chain, positive only if the house is net wealth at the end of life — a half-life rather than a trap. Quantitatively, transfers are the whole of the model's persistence: the rank–rank slope of 0.34 falls to −0.01 without bequests."
2551 + },
2552 + {
2553 + "title": "Existence, uniqueness and no multiplicity",
2554 + "description": "The stationary equilibrium exists and is unique when the entry-threshold channel dominates the ladder channel; three price scans, including a supply elasticity of 0.05, find monotone excess demand and a single root."
2555 + },
2556 + {
2557 + "title": "Surprise inflation reverses sign in general equilibrium",
2558 + "description": "With deposits exceeding mortgages, a 10% price-level surprise lowers the real house price by 0.45% (fixed-rate), 0.47% (renewable term) and 0.79% (adjustable) where the benchmark predicted a rise; mortgaged owners gain +1.14%, +0.46% and −1.21% of lifetime consumption, and the Canadian five-year term loses its advantage when the real rate responds."
2559 + },
2560 + {
2561 + "title": "Negative results reported as results",
2562 + "description": "Three simulation outcomes contradicted the benchmark — the inflation sign, price neutrality without segmentation, and monotonicity of the entry threshold in the down payment — and each proposition is restated under the identified premise rather than the simulations discarded."
2563 + }
2564 + ],
2565 + "data": [
2566 + {
2567 + "title": "Benchmark environment",
2568 + "description": "Three stages of life with two-period generations, unit housing demand, a nominal mortgage, an inter-vivos transfer at the child's purchase and a bequest; closed forms for the entry threshold, the leverage multiplier, the dynasty chain and the revaluation of a price-level surprise under fixed-rate, renewable-term and adjustable contracts."
2569 + },
2570 + {
2571 + "title": "Quantitative OLG economy",
2572 + "description": "Twelve five-year periods, four house sizes and continuous rented services, Rouwenhorst income risk, 25-year constant-payment nominal mortgages under three contracts with loan-to-value and payment-to-income constraints, a gift at the child's entry and an estate at the parent's death, segmented rental and construction stocks and a lender balance sheet; simulated on a 96,000-dynasty Monte-Carlo panel."
2573 + },
2574 + {
2575 + "title": "Illustrative calibration",
2576 + "description": "No microdata and no fitted moment; verified Canadian comparison points (Survey of Financial Security 2023 owner–renter net worth and mortgage share, Hou 2010 ownership profile, CIBC gift statistics) are reported next to model moments without being targeted; the housing-supply elasticity, lacking a verified Canadian estimate, is swept over 0.05–2."
2577 + }
2578 + ],
2579 + "methodology": [
2580 + {
2581 + "title": "Closed-form benchmark theory",
2582 + "description": "Nine propositions on entry, leverage, persistence, existence and uniqueness, and surprise-inflation accounting, with proofs in a standalone appendix and a general optimiser used to verify the hand-to-mouth assumption."
2583 + },
2584 + {
2585 + "title": "Household problem by backward value-function iteration",
2586 + "description": "Twelve ages with rent, buy, refinance, sell and stay branches on a cash-on-hand grid; loans on eleven fractions of the binding constraint, gifts on twelve fractions of cash, a five-point estate lottery; numba-accelerated."
2587 + },
2588 + {
2589 + "title": "Stationary general equilibrium",
2590 + "description": "Nested Illinois regula falsi (rental market inside the owner market) with a damped fixed point on the estate lottery and the transfer-tax rebate, tolerance 5e-4; supply scales normalised so that house and rental prices equal one."
2591 + },
2592 + {
2593 + "title": "Perfect-foresight transitions",
2594 + "description": "Sequence-space shooting over 14 periods with durable stocks, a parallel finite-difference Jacobian, Broyden updates and a damped tâtonnement pre-phase; every run reported with its residual (12 of 16 within tolerance)."
2595 + },
2596 + {
2597 + "title": "Welfare",
2598 + "description": "Consumption-equivalent variation net of warm glow for entrants by transfer quintile and for households alive at a shock by age and tenure."
2599 + }
2600 + ],
2601 + "reproducibility": [
2602 + "make all: tests → simulation (45 stationary equilibria, 5 partial-equilibrium runs, 3 price scans, 16 transitions, welfare; about 60 minutes on 18 cores) → 14 figures and 9 tables → LaTeX",
2603 + "69 pytest tests on identities, debt dynamics, feasibility, market clearing and residuals (about 20 s), 44 of them on the closed-form benchmark",
2604 + "Every number in the manuscript is one of about 2,500 macros in numbers.tex read from 358 committed CSV/JSON result files",
2605 + "Research folder with 72 verified references, the model specification, propositions, failed hypotheses and a results-audit integrity gate; proofs compile as a standalone theory appendix"
2606 + ],
2607 + "keywords": [
2608 + "housing tenure",
2609 + "mortgage leverage",
2610 + "intergenerational wealth",
2611 + "bequests",
2612 + "down-payment constraint",
2613 + "surprise inflation",
2614 + "overlapping generations",
2615 + "general equilibrium",
2616 + "Canada"
2617 + ],
2618 + "repo": "https://git.spboucher.ai/wp18_uqo",
2619 + "jel": "R21, R31, D31, D91, E31, G51"
2620 + },
2621 + {
2622 + "slug": "wp19",
2623 + "hero": {
2624 + "headline": "The housing liquidity discount is the price of seller urgency",
2625 + "subheadline": "In the illustrative baseline the most urgent seller concedes 5.6% relative to the most patient, identical houses show 3.5% price dispersion, and halving matching efficiency lowers prices by 4.8% while lengthening time on market from 2.2 to 3.4 months."
2626 + },
2627 + "abstract": [
2628 + "This paper studies how search frictions, heterogeneous seller urgency, financing frictions and bargaining jointly determine the liquidity discount, transaction prices, sale hazards and price dispersion in a local housing market. In a continuous-time matching model with free entry of buyers, sellers differ in a permanent carrying cost, buyers draw match-specific valuations, and prices are set by Nash bargaining. The urgency discount, defined relative to the most patient seller, is nonnegative and increasing in the carrying cost, and the expected price falls by less than the reservation price.",
2629 + "A fall in buyer arrivals or in mortgage qualification lowers prices, lengthens time on market and widens the discount when the valuation distribution has an increasing failure rate; with heavy-tailed valuations a more liquid market can lengthen time on market, under an explicit condition on the carrying cost. Liquidity shocks move prices, volume, inventory and durations with no change in housing-service values, and buyer entry amplifies them. The gap between mortgage and cash prices decomposes into a delay-bargaining term that rises with urgency, an outside-option term, and a selection term that can be negative.",
2630 + "The composition of transactions shifts after a liquidity shock in a direction governed by the shape of the valuation distribution, and its effect on a transaction-price index is an order of magnitude smaller than the within-type liquidity effect, which no quality adjustment removes. The Walrasian free-entry price is shown not to be a liquidity-premium benchmark, since search prices exceed it in thick markets. Results are illustrated numerically with a documented parameterisation and reproducible code; no data are used."
2631 + ],
2632 + "findings": [
2633 + {
2634 + "value": "8",
2635 + "label": "propositions on discounts, hazards, entry, shocks, selection and financing"
2636 + },
2637 + {
2638 + "value": "5.6%",
2639 + "label": "urgency discount of the most urgent seller"
2640 + },
2641 + {
2642 + "value": "3.5%",
2643 + "label": "price dispersion among identical houses in the baseline"
2644 + },
2645 + {
2646 + "value": "−4.8%",
2647 + "label": "price change after halving matching efficiency"
2648 + },
2649 + {
2650 + "value": "≤ 0.15 log-%",
2651 + "label": "sale-selection term in a transaction-price index"
2652 + },
2653 + {
2654 + "value": "1.73%",
2655 + "label": "cash–mortgage gap in the financing extension"
2656 + }
2657 + ],
2658 + "contributions": [
2659 + {
2660 + "title": "Urgency discount characterised relative to the most patient seller",
2661 + "description": "The discount is nonnegative and increasing in the carrying cost, the expected price falls by less than the reservation price, and the discount widens when buyers are scarce whenever the valuation distribution has an increasing failure rate."
2662 + },
2663 + {
2664 + "title": "An exact condition under which liquidity lengthens time on market",
2665 + "description": "With heavy-tailed buyer valuations a more liquid market can lengthen the time on market of urgent sellers, under a condition that depends on the carrying cost but not on the arrival rate — with an explicit Pareto counterexample."
2666 + },
2667 + {
2668 + "title": "Liquidity shocks without fundamentals, amplified by entry",
2669 + "description": "A fall in matching efficiency or in mortgage qualification moves prices, volume, inventory and durations together with no change in housing-service values; free entry amplifies arrival shocks (elasticity 1.10), and a permanent −20% arrival shock cuts sales by 13% on impact while inventory rises 13% with a half-life of 1.9 months."
2670 + },
2671 + {
2672 + "title": "Sign condition for sale-selection bias in price indices",
2673 + "description": "The composition term depends on whether the product of the hazard function and the mean residual valuation rises or falls: exponential and Pareto valuations amplify measured price changes, truncated-normal and Weibull valuations dampen them, and the term stays at or below 0.15 log-% against within-type liquidity effects an order of magnitude larger."
2674 + },
2675 + {
2676 + "title": "Three-term decomposition of the cash–mortgage gap",
2677 + "description": "Mortgage buyers pay 1.73% more than cash buyers in the extension — 0.49 from delay-bargaining, 1.61 from the outside option, −0.37 from selection — ranging from 1.1% for patient to 3.0% for urgent sellers; tighter approval and longer delays widen the gap."
2678 + },
2679 + {
2680 + "title": "The Walrasian price is not a liquidity-premium benchmark",
2681 + "description": "A seller with the option to wait for a high-valuation buyer captures match-specific rents, so search prices exceed the Walrasian free-entry price in thick markets; the discount is therefore measured among sellers in the same market. The welfare-maximising bargaining weight is 0.55 against a Hosios value of 0.50."
2682 + }
2683 + ],
2684 + "data": [
2685 + {
2686 + "title": "Model environment",
2687 + "description": "A continuous-time local housing market with exogenous listing inflow, withdrawals, free entry of buyers at cost k, a Cobb–Douglas contact technology, sellers heterogeneous in a permanent carrying cost drawn from a continuous distribution, buyers drawing match-specific valuations from truncated-normal, exponential, Pareto or Weibull families, Nash bargaining, and extensions with posted prices and a financing contingency (cash versus mortgage buyers)."
2688 + },
2689 + {
2690 + "title": "Illustrative parameterisation",
2691 + "description": "A single configuration file (config/baseline.yaml, hash 0a165f317815d76c, seed 20261020) in which a listing sells in about 2.2 months, a buyer searches about 10.8 months, tightness is 4.39 buyers per listing with contact rates of 8.0 and 1.8 per year, and the most urgent seller concedes 5.6%; no data are used and no empirical claim is made."
2692 + }
2693 + ],
2694 + "methodology": [
2695 + {
2696 + "title": "Analytical results",
2697 + "description": "Eight propositions and supporting lemmas on the urgency discount, the arrival-rate comparative statics, free-entry equilibrium, liquidity shocks, dispersion, sale selection, the cash–mortgage gap and posted prices, with proofs in the appendix."
2698 + },
2699 + {
2700 + "title": "Numerical solution",
2701 + "description": "Seller values by Brent's method on a strictly decreasing fixed-point map, free-entry tightness by sign scan and Brent with multiplicity reported, posted-price and financing variants as root problems, a steady-state planner's problem, and transitions by backward HJB, forward Kolmogorov and pointwise free entry."
2702 + },
2703 + {
2704 + "title": "Twelve pre-registered experiments",
2705 + "description": "Comparative statics of arrivals, qualification, approval delay, bargaining power, entry costs and seller heterogeneity; transitions after permanent and temporary shocks; index decomposition across four valuation families; posted-price protocol; sensitivity of every result to the parameters."
2706 + },
2707 + {
2708 + "title": "Monte Carlo validation",
2709 + "description": "Simulated markets cross-check the analytical moments, residuals and bounds; results are written with the configuration hash, seed, software versions and the pre-registered hypothesis."
2710 + }
2711 + ],
2712 + "reproducibility": [
2713 + "Makefile targets test, simulate (12 experiments, about 6 minutes), diagnostics (Monte Carlo, about 3 minutes), tables, figures and paper, all driven by the single config/baseline.yaml",
2714 + "24 pytest tests on closed forms, propositions, boundary cases, equilibrium and convergence (about 20 s)",
2715 + "14 figures and 23 LaTeX table fragments plus numbers.tex macros regenerated from stored CSVs only; literature matrix of 132 references cross-checked against Crossref",
2716 + "Six falsifiable predictions stated in terms of listing-level data (time on market, withdrawals, financing type, seller circumstances) for the empirical follow-up"
2717 + ],
2718 + "keywords": [
2719 + "housing liquidity",
2720 + "search and matching",
2721 + "seller urgency",
2722 + "time on market",
2723 + "price dispersion",
2724 + "sale selection",
2725 + "financing contingencies",
2726 + "Nash bargaining",
2727 + "free entry"
2728 + ],
2729 + "repo": "https://git.spboucher.ai/wp19_uqo",
2730 + "jel": "R31, R21, D83, C63, G51"
2731 + },
2732 + {
2733 + "slug": "wp2",
2734 + "hero": {
2735 + "headline": "Listing text carries priceable information, but much of it is language",
2736 + "subheadline": "Twenty named semantic scores lift adjusted R² from 0.452 to 0.511; 40% of listings are in English and score lower against French references, and net of language and 415 FSA fixed effects the block adds 2.0 points (3.9 with a multilingual encoder)."
2737 + },
2738 + "abstract": [
2739 + "Broker-written listing descriptions contain information about housing quality that structured hedonic variables omit. This paper proposes an interpretable way to use that information: sentence-transformer embeddings of 17,087 single-family listings from Quebec are projected, by cosine similarity, onto 20 researcher-defined reference descriptions (luxury, needs renovation, waterfront, motivated seller, and so on), and the 20 similarity scores enter a semi-logarithmic hedonic model as named regressors. Adding the semantic block to a structural model raises the adjusted R² from 0.452 to 0.511, a gain confirmed out of sample in random and spatially grouped cross-validation.",
2740 + "Three qualifications follow. First, 40% of the descriptions are written in English and score systematically lower against the French references; the language of the listing alone explains 86% of the variance of the mean similarity and accounts for more than half of the apparent semantic gain. Net of language and of 415 forward-sortation-area fixed effects, the semantic block adds 2.0 percentage points of R² with the baseline encoder and 3.9 points with a multilingual encoder, and remains jointly significant with standard errors clustered by area. Second, the individual implicit-price gradients are sensitive to the phrasing of the references and to the encoder: only a core of dimensions (Luxury, Land & Nature, Family-Friendly, Quiet & Peaceful, Needs Renovation) keeps its sign and significance across specifications, while others, including Motivated Seller, New Construction and Entry-Level, do not.",
2741 + "Third, the coefficients are conditional associations in listing prices, not causal effects of wording. The paper documents what reference-based projection can and cannot deliver for hedonic inference and provides a replication pipeline that reproduces every reported number from the raw database."
2742 + ],
2743 + "findings": [
2744 + {
2745 + "value": "17,087",
2746 + "label": "Quebec single-family listings with price and description"
2747 + },
2748 + {
2749 + "value": "20",
2750 + "label": "named reference descriptions used as semantic regressors"
2751 + },
2752 + {
2753 + "value": "0.452 → 0.511",
2754 + "label": "adjusted R² gain from the semantic block"
2755 + },
2756 + {
2757 + "value": "39.7%",
2758 + "label": "listings written in English, scoring lower on French references"
2759 + },
2760 + {
2761 + "value": "86%",
2762 + "label": "variance of mean similarity explained by language alone"
2763 + },
2764 + {
2765 + "value": "2.0 pp",
2766 + "label": "R² added net of language and 415 FSA fixed effects"
2767 + }
2768 + ],
2769 + "contributions": [
2770 + {
2771 + "title": "Named, auditable text features for hedonic models",
2772 + "description": "Cosine projection of listing embeddings onto 20 reference descriptions turns an opaque embedding into named regressors; the block is jointly significant in every specification (F(21, 17,059) = 99.5 in the baseline)."
2773 + },
2774 + {
2775 + "title": "Language, not content, drives half the gain",
2776 + "description": "57.6% of listings are French, 39.7% English and 2.7% mixed; mean similarity is 0.434 for French versus 0.204 for English texts, and language indicators cut the semantic ΔR² from 0.047 to 0.017."
2777 + },
2778 + {
2779 + "title": "The gain survives location controls and spatial holdouts",
2780 + "description": "With 415 FSA fixed effects and area-clustered errors the block still adds 2.0 points of R² (3.9 with a multilingual encoder); five-fold FSA-grouped cross-validation gives R²_oos of 0.522 versus 0.484 for the structural model."
2781 + },
2782 + {
2783 + "title": "Only five dimensions are robust",
2784 + "description": "Luxury, Land & Nature, Family-Friendly, Quiet & Peaceful and Needs Renovation keep sign and significance across all specifications and both encoders; the previously emphasised Motivated Seller discount (−8.5%) becomes insignificant once language is controlled."
2785 + },
2786 + {
2787 + "title": "Individual gradients are fragile to wording and encoder",
2788 + "description": "Perturbing the reference texts leaves the similarities almost unchanged (correlation 0.93–0.99) and the block ΔR² at 0.042–0.054, yet the vector of 20 coefficients correlates only 0.02–0.81 with the baseline."
2789 + },
2790 + {
2791 + "title": "Inference needs spatial clustering",
2792 + "description": "FSA-clustered standard errors are on average 50% larger (22–107%) than HC3 errors, so dimension-level conclusions change while block-level conclusions do not."
2793 + }
2794 + ],
2795 + "data": [
2796 + {
2797 + "title": "Quebec single-family listings",
2798 + "description": "17,087 listings with a valid price and free-text description, extracted from a 46,479-listing SQLite database (snapshot of 7 January 2026); structural fields, lot size, forward sortation area and language detected by stop-word rule."
2799 + },
2800 + {
2801 + "title": "Reference descriptions and embeddings",
2802 + "description": "20 French reference texts embedded with all-MiniLM-L6-v2 and, in the revision, paraphrase-multilingual-MiniLM-L12-v2; cosine similarities between listing and reference embeddings form the 17,087 × 20 semantic matrix."
2803 + }
2804 + ],
2805 + "methodology": [
2806 + {
2807 + "title": "Reference-based cosine projection",
2808 + "description": "Listing embeddings are projected onto 20 researcher-defined reference descriptions; each similarity score enters the log-price equation as a named regressor whose coefficient is read per standard deviation of similarity."
2809 + },
2810 + {
2811 + "title": "Hedonic models A–E and extended specifications",
2812 + "description": "Structural, length, semantic and full OLS models with HC3 errors, then 14 extended specifications with language indicators, 415 FSA or 1,061 municipality fixed effects, FSA-clustered errors, unit-corrected lot area and language subsamples."
2813 + },
2814 + {
2815 + "title": "Out-of-sample validation",
2816 + "description": "Five-fold random and FSA-grouped cross-validation reporting R²_oos, RMSE and MdAPE for structural, semantic, PCA-embedding and multilingual models."
2817 + },
2818 + {
2819 + "title": "Sensitivity to references and encoder",
2820 + "description": "Three perturbations of the reference phrasing and a full re-encoding with a multilingual model, with block tests by domain and dimension-level stability checks."
2821 + },
2822 + {
2823 + "title": "Robustness battery",
2824 + "description": "Quantile regressions, 1,000-replication bootstrap, 1%/99% trimming, VIF diagnostics, Lasso and elastic-net selection, and PCA of the raw embeddings as a comparison."
2825 + }
2826 + ],
2827 + "reproducibility": [
2828 + "Eight numbered Python scripts from the raw SQLite database to models, robustness, figures and LaTeX tables; the 2026-10-17 revision adds scripts 07–08 (language, fixed effects, clustering, out-of-sample, reference sensitivity, multilingual encoder)",
2829 + "Baseline v1.1 results reproduced exactly and kept as benchmark; all revision tables and figure 12 are generated, never typed",
2830 + "12 figures and 15 tables regenerated from results/*.csv; embedding caches for both encoders",
2831 + "Full LaTeX source of the 61-page paper with pre-revision PDF and review report archived in docs/"
2832 + ],
2833 + "keywords": [
2834 + "hedonic pricing",
2835 + "text as data",
2836 + "sentence embeddings",
2837 + "cosine similarity",
2838 + "listing descriptions",
2839 + "real estate valuation",
2840 + "language effects",
2841 + "Quebec"
2842 + ],
2843 + "repo": "https://git.spboucher.ai/wp2_uqo",
2844 + "jel": "R31, C45, C21, R21"
2845 + },
2846 + {
2847 + "slug": "wp20",
2848 + "hero": {
2849 + "headline": "Listing photographs specify the building; the cost anchor sets the value",
2850 + "subheadline": "Exterior attributes are read with 84.5–95.8% agreement against broker fields, but only 3.1% of listings show a basement within ten photographs; multimodal evidence moves the cost-to-asking-price ratio from 0.75 to 0.95 (MdAPE 30.1% to 27.9%)."
2851 + },
2852 + "abstract": [
2853 + "The cost approach to value requires a specification of the improvements — materials, finish grade, condition, renovations — that assessment rolls do not record and that appraisers obtain by inspection. This paper asks whether that specification can be extracted from the photographs and text of a for-sale listing by general-purpose vision–language models, and what a transparent cost indication built on it is worth. It formalises an extraction-to-valuation framework with five evidence classes (administrative, visual, proxy, text-claimed, latent), deterministic cross-image aggregation, text–image–record reconciliation, a fully sourced cost schedule and uncertainty propagation, and applies it to 1,200 Québec house listings matched within 15 m to their roll unit (14,245 photographs, three extractor tiers, 104 verified references).",
2854 + "Visible exterior attributes are extracted with high agreement against broker fields (roof covering 95.8%, garage type 94.1%, cladding 84.5%) but storeys agree with the roll only loosely (κ = 0.51), and the most cost-relevant interior evidence is simply not shown: within the first ten photographs, 3.1% of listings show a basement and 19.6% a bathroom, a missing-not-at-random structure that no extractor can overcome. Relative to a roll-only baseline, image and text attributes raise the median cost indication and move its ratio to the asking price from 0.75 to 0.95 (median absolute deviation 30.1% to 27.9%), mainly through condition-adjusted depreciation; the gain is real but modest, and the level of the indication depends more on the published unit-cost anchor than on anything seen in the images.",
2855 + "A calibration against assessor building values on five metropolitan areas transfers to a geographic hold-out and implies a hard cost near $300 per square foot, consistent with 2026 Québec industry guides and well above a Montréal hard-cost band. The ontology, prompts, schedule and code are released, and the paper is explicit about what the data cannot establish: no expert labels exist, latent defects are unobservable, and the cost indication is a screening tool, not an appraisal."
2856 + ],
2857 + "findings": [
2858 + {
2859 + "value": "1,200",
2860 + "label": "listings matched to their roll unit, 14,245 photographs"
2861 + },
2862 + {
2863 + "value": "95.8 / 94.1 / 84.5%",
2864 + "label": "roof, garage and cladding agreement with broker fields"
2865 + },
2866 + {
2867 + "value": "κ = 0.51",
2868 + "label": "storeys agreement between image and assessment roll"
2869 + },
2870 + {
2871 + "value": "3.1% / 19.6%",
2872 + "label": "listings showing a basement / bathroom in ten photographs"
2873 + },
2874 + {
2875 + "value": "0.75 → 0.95",
2876 + "label": "cost-to-asking-price ratio, roll-only to multimodal baseline"
2877 + },
2878 + {
2879 + "value": "−29.9%",
2880 + "label": "median indication under a Montréal hard-cost anchor"
2881 + }
2882 + ],
2883 + "contributions": [
2884 + {
2885 + "title": "An extraction-to-valuation framework for the cost approach",
2886 + "description": "Five evidence classes, deterministic cross-image aggregation, text–image–record reconciliation (levels B0–B4), a fully sourced cost schedule and Monte Carlo uncertainty propagation; no price enters the indication."
2887 + },
2888 + {
2889 + "title": "Exterior facts are read reliably and abstention works",
2890 + "description": "Against broker fields the mid-size extractor agrees on roof covering in 95.8% of informative cases, garage type 94.1% and primary cladding 84.5%, with coverage above 85%; storeys agree with the roll at κ = 0.51 because roll, broker and image count half-storeys differently."
2891 + },
2892 + {
2893 + "title": "The binding constraint is what sellers show",
2894 + "description": "Within the first ten photographs 95.0% of listings show an exterior and 67.8% a kitchen, but only 19.6% a bathroom and 3.1% a basement — a missing-not-at-random structure that makes basement finish a text-claimed attribute and leaves condition as the images' unique contribution."
2895 + },
2896 + {
2897 + "title": "Images move the indication through depreciation",
2898 + "description": "From the roll-only to the multimodal baseline the median depreciation falls from 0.62 to 0.38, the median indication rises from $407,545 to $514,919, the ratio to the asking price moves from 0.75 to 0.95 and the median absolute deviation from 30.1% to 27.9%; on 146 recorded sales the ratio to the sale price moves from 0.70 to 0.84."
2899 + },
2900 + {
2901 + "title": "The unit-cost anchor dominates the evidence",
2902 + "description": "Switching from a Québec 2026 industry band to a Montréal hard-cost band moves the median indication by −29.9%, more than any attribute seen in the images; the assumed depreciation schedule is too steep for old houses and too lenient for new ones."
2903 + },
2904 + {
2905 + "title": "Assessor calibration transfers to a geographic hold-out",
2906 + "description": "Calibrating the schedule to assessor building values on five metropolitan areas transfers to the Québec City hold-out and implies a standard hard cost of about $272–306 per square foot, inside the 2026 Québec industry band."
2907 + }
2908 + ],
2909 + "data": [
2910 + {
2911 + "title": "Listings matched to the assessment roll",
2912 + "description": "Read-only copies (17 October 2026) of the Vrai Prix databases: 3.75 million roll units (MAMH, Données Québec), 769,000 transactions, and 77,511 live Immo-Ka house, condo and lot listings from 67 connectors; 39,278 house listings reduced to 7,745 by rules R1–R10, then a stratified sample of 1,200 by metropolitan area, age band and price quartile. Photographs are read by URL and never stored; hosts whose robots.txt disallows retrieval (RE/MAX Québec) were excluded."
2913 + },
2914 + {
2915 + "title": "Photographs and extractions",
2916 + "description": "14,245 photographs processed by three vision–language extractor tiers (Claude 5.5 family) through the Message Batches API under strict JSON schemas, plus text extraction on price-masked descriptions; broker fields and roll attributes serve as reference; 146 listings with a recorded sale after listing."
2917 + },
2918 + {
2919 + "title": "Cost schedule sources",
2920 + "description": "Two published unit-cost anchors (Altus 2026 Montréal hard-cost band; Québec 2026 industry guides at $257–400 per square foot by grade), storey and attachment multipliers, add-ons and mark-ups, a Statistics Canada construction price index (18-10-0289-01) and roll land values indexed to the valuation date; every parameter sourced or flagged."
2921 + }
2922 + ],
2923 + "methodology": [
2924 + {
2925 + "title": "Schema-constrained multimodal extraction",
2926 + "description": "Per-image extraction with versioned JSON schemas on three model tiers, text extraction on masked text, confidence-weighted votes, ordinal medians and means aggregated deterministically across images, then reconciled with the roll and broker fields in five baselines B0–B4."
2927 + },
2928 + {
2929 + "title": "Transparent cost model",
2930 + "description": "Indication = indexed roll land value + (replacement cost new by grade, storeys and add-ons, with soft costs and mark-ups) × (1 − depreciation from condition- and renovation-adjusted effective age); pure functions over the sourced schedule."
2931 + },
2932 + {
2933 + "title": "Evaluation of every link",
2934 + "description": "Agreement, Cohen's κ and Krippendorff's α with municipality cluster bootstrap; channel information content; image-budget truncation; MNAR room-presence analysis and omission simulation; confidence calibration and selective prediction; error taxonomy."
2935 + },
2936 + {
2937 + "title": "Alignment, sensitivity and calibration",
2938 + "description": "Levels B0–B4 compared with indexed assessor building values, asking prices and recorded sales; one-at-a-time sensitivity, labelled scenarios, Monte Carlo intervals (median 10–90% width 26%), and an assessor-based calibration on five metropolitan areas with a Québec City hold-out."
2939 + }
2940 + ],
2941 + "reproducibility": [
2942 + "Makefile pipeline (frame, sample, batches, collect, aggregate, experiments, figures, tables, paper) with a results manifest; about 4 minutes of experiments without API calls, about US$110 of batch extraction",
2943 + "Pre-registered experiment plan (14 of 16 experiments executed, 2 declared infeasible), implementation audit, results-validation integrity gate and a proposed expert annotation protocol in docs/",
2944 + "pytest suite on price masking, schema constraints, aggregation rules, metrics and the cost identity; about 1,180 LaTeX macros in numbers.tex feed every number in the paper",
2945 + "Ontology, prompts, cost schedule and code released; the Vrai Prix databases and photographs are not redistributed (docs/data_rights.md)"
2946 + ],
2947 + "keywords": [
2948 + "cost approach",
2949 + "automated valuation",
2950 + "vision–language models",
2951 + "listing photographs",
2952 + "depreciation",
2953 + "assessment roll",
2954 + "missing not at random",
2955 + "replacement cost",
2956 + "Québec"
2957 + ],
2958 + "repo": "https://git.spboucher.ai/wp20_uqo",
2959 + "jel": "R31, R32, C45, C55, D83"
2960 + },
2961 + {
2962 + "slug": "wp21",
2963 + "hero": {
2964 + "headline": "The hedonic core is global, but no implicit price is homogeneous",
2965 + "subheadline": "Across 242 studies from 55 countries, living area carries an elasticity of 0.65 and a bathroom a 5.91% premium, yet prediction intervals span zero for most characteristics and observable study features explain little of the dispersion."
2966 + },
2967 + "abstract": [
2968 + "This paper synthesises the global hedonic housing literature from 1970 to 2026 with a reproducible pipeline: 103,011 records identified from Semantic Scholar, Crossref, arXiv and OpenAlex, 56,727 unique records screened by rules and by independent agents under a PRISMA-style protocol, 2,249 hedonic studies included at stage one, 501 open-access full texts retrieved, and 242 eligible studies from 55 countries extracted into 18,831 coefficient rows with a validated double-extraction sample. Coefficients are harmonised into percentage premiums, semi-elasticities per common unit and elasticities, and pooled with random-effects, three-level and cluster-robust models, with publication-selection diagnostics and Bayesian and machine-learning cross-checks.",
2969 + "The structural core of the hedonic model is global: the elasticity of price to living area is 0.65, a bathroom adds 5.91 percent, a year of age subtracts 0.24 percent and a kilometre from the centre 3.08 percent, with similar means across continents and decades. Signature amenities carry premiums between four and eleven percent and a waterfront location 32.50 percent. Every implicit price is nonetheless heterogeneous beyond sampling error, prediction intervals span zero for most characteristics, and observable study features, including spatial estimators, explain little of the heterogeneity.",
2970 + "Funnel asymmetry is widespread but cannot be interpreted as selection alone. Machine-learning models predict better than hedonic benchmarks in most of the forty retrievable comparisons, while AI-derived text and image features add modest accuracy and some interpretable characteristics. The synthesis extends Sirmans et al. (2005, 2006) in coverage, harmonisation, inference and scope, and is bounded by the open-access retrieval rate that it documents."
2971 + ],
2972 + "findings": [
2973 + {
2974 + "value": "103,011",
2975 + "label": "bibliographic records identified, 1970–2026"
2976 + },
2977 + {
2978 + "value": "242",
2979 + "label": "eligible hedonic studies from 55 countries"
2980 + },
2981 + {
2982 + "value": "18,831",
2983 + "label": "coefficient rows extracted and harmonised"
2984 + },
2985 + {
2986 + "value": "0.65",
2987 + "label": "pooled elasticity of price to living area"
2988 + },
2989 + {
2990 + "value": "+5.91%",
2991 + "label": "price premium per additional bathroom"
2992 + },
2993 + {
2994 + "value": "+32.50%",
2995 + "label": "waterfront premium, the largest signature amenity"
2996 + }
2997 + ],
2998 + "contributions": [
2999 + {
3000 + "title": "The largest harmonised hedonic synthesis to date",
3001 + "description": "From 103,011 records to 56,727 unique, 2,249 included hedonic studies and 242 extracted from 55 countries, yielding 7,693 usable harmonised estimates from 149 studies across 67 characteristic–family combinations."
3002 + },
3003 + {
3004 + "title": "The structural core is global",
3005 + "description": "Living area (elasticity 0.65, k = 46), bathrooms (+5.91%), age (−0.24% per year) and distance to the centre (−3.08% per km) have precisely estimated means that are similar across continents and decades once specification is controlled."
3006 + },
3007 + {
3008 + "title": "Heterogeneity dominates every implicit price",
3009 + "description": "Median I² is close to 100% and prediction intervals span zero for most characteristics; only 12 of 182 moderator terms are significant under robust variance estimation and machine-learning meta-regressions have negative out-of-study R²."
3010 + },
3011 + {
3012 + "title": "Selection is detectable but not correctable",
3013 + "description": "Egger, PET-PEESE and step selection models find widespread funnel asymmetry, suggestive for amenity premiums, but the asymmetry cannot be read as publication selection alone."
3014 + },
3015 + {
3016 + "title": "Machine learning predicts better, AI features add little",
3017 + "description": "In most of the 40 retrievable same-test-set comparisons tree ensembles beat the hedonic benchmark (median MAPE 9.0 vs 12.4), and nine studies report modest incremental gains from text and image features."
3018 + },
3019 + {
3020 + "title": "Validated agent-based extraction",
3021 + "description": "An independent double extraction of 20 papers reached 100% numerical agreement on 647 matched coefficients, supporting the LLM-assisted extraction of 297 full texts."
3022 + }
3023 + ],
3024 + "data": [
3025 + {
3026 + "title": "Bibliographic corpus",
3027 + "description": "103,011 records retrieved on 2026-10-20 from the Semantic Scholar bulk API, Crossref, arXiv and OpenAlex with 67 multilingual queries; 56,727 unique after DOI and normalised-title deduplication, screened by rules and by independent agents with coded reasons."
3028 + },
3029 + {
3030 + "title": "Harmonised coefficient database",
3031 + "description": "501 open-access full texts (18.8% retrieval rate) via Unpaywall, Semantic Scholar and arXiv; 297 papers extracted, 242 eligible, 2,022 model columns and 18,831 coefficient rows harmonised into Kennedy-corrected premiums, unit-harmonised semi-elasticities and elasticities (7,693 usable estimates from 149 studies)."
3032 + },
3033 + {
3034 + "title": "AI and machine-learning comparison set",
3035 + "description": "40 studies with retrievable same-test-set comparisons between machine-learning models and hedonic benchmarks, plus studies reporting incremental gains from text and image features."
3036 + }
3037 + ],
3038 + "methodology": [
3039 + {
3040 + "title": "PRISMA-style systematic search and screening",
3041 + "description": "Discovery across four bibliographic services, deduplication, rule-based and agent screening with coded reasons, and a PRISMA flow generated directly from the logs."
3042 + },
3043 + {
3044 + "title": "Harmonisation of implicit prices",
3045 + "description": "Coefficients converted to percentage premiums (Kennedy correction), semi-elasticities per common unit, elasticities and partial correlations, with quality flags for each estimate."
3046 + },
3047 + {
3048 + "title": "Multilevel meta-analysis",
3049 + "description": "REML random-effects, three-level and cluster-robust models (metafor rma.uni/rma.mv with clubSandwich CR2) for 67 characteristic–family combinations, with meta-regressions on study moderators."
3050 + },
3051 + {
3052 + "title": "Publication-selection diagnostics",
3053 + "description": "Egger tests, PET-PEESE and step selection models applied to each characteristic, reported as diagnostics rather than corrections."
3054 + },
3055 + {
3056 + "title": "Bayesian and machine-learning cross-checks",
3057 + "description": "Gibbs-sampled Bayesian pooling and grouped random-forest and gradient-boosting meta-regressions tested for out-of-study prediction."
3058 + }
3059 + ],
3060 + "reproducibility": [
3061 + "Makefile pipeline (screen, effects, models, tables, figures, paper) reproduces the paper from stored records and logs without live queries",
3062 + "Preregistered protocol (protocol/preregistration.md, eligibility and search strategy) and a documented extraction protocol with a 20-paper double-extraction validation",
3063 + "14 figures, 23 table fragments and 913 numeric macros regenerated by scripts/visualization/; pytest suite in tests/",
3064 + "Full LaTeX source of the 51-page paper with sections, appendices and bibliography; open-access PDFs not redistributed but re-downloadable from the retrieval log"
3065 + ],
3066 + "keywords": [
3067 + "hedonic pricing",
3068 + "housing characteristics",
3069 + "implicit prices",
3070 + "meta-analysis",
3071 + "meta-regression",
3072 + "publication selection",
3073 + "machine learning valuation",
3074 + "PRISMA",
3075 + "systematic review"
3076 + ],
3077 + "repo": "https://git.spboucher.ai/wp21_uqo",
3078 + "jel": "R31, R21, C13, C80, C45"
3079 + },
3080 + {
3081 + "slug": "wp22",
3082 + "hero": {
3083 + "headline": "An empirical-Bayes hierarchy halves municipal index error in thin markets",
3084 + "subheadline": "On synthetic Quebec-like markets with known latent paths, SHARP-HPI brings the RMSE of municipal monthly changes to 1.58 log points, against 2.91 for per-cell state space and 24.28 for the median, with 98% coverage of 95% intervals."
3085 + },
3086 + "abstract": [
3087 + "This paper develops and evaluates SHARP-HPI, a framework for monthly quality-adjusted housing price indices at the municipal level in thin markets. Stage one is a robust pooled hedonic regression with fine-scale location fixed effects, smoothly time-varying quality prices and provincial month effects; stage two models each geography's deviation from its parent (province, administrative region, regional county municipality, municipality) with heteroskedastic local-level state-space models whose innovation variances are calibrated by empirical Bayes across cells, so that the weight given to local sales adapts to their reliability.",
3088 + "The framework is applied to 769,995 registered sales in Quebec from January 2021 to October 2026 matched to the assessment roll, with an audited population definition (632,006 sales), a measured publication lag and real-time vintages. Against eleven benchmark estimators (median, stratified median, pooled and rolling time dummies, hedonic imputation, GEKS, HMTS, repeat sales, gradient boosting and a production hierarchical index), Monte Carlo experiments on synthetic Quebec-like markets show that the adaptive hierarchy halves the error of municipal monthly changes relative to per-cell state-space estimation and keeps interval coverage near nominal. Two deliberate counter-examples show where it does not: heavy-tailed contamination favours a coarser location partition, and at the regional level region-specific hedonic coefficients beat pooled coefficients plus a smoothed deviation.",
3089 + "The empirical panel covers 2,654 geography–class cells with reliability grades, effective sample sizes and local-data weights; aggregates agree closely with Teranet and CREA indices in Québec City and diverge by an explainable margin in Montréal. Every ingredient of the estimator exists in the literature; the contribution is their integration, the explicit reliability accounting and the documented performance."
3090 + ],
3091 + "findings": [
3092 + {
3093 + "value": "769,995",
3094 + "label": "registered Quebec sales, January 2021 – October 2026"
3095 + },
3096 + {
3097 + "value": "632,006",
3098 + "label": "sales in the audited headline population"
3099 + },
3100 + {
3101 + "value": "2,654",
3102 + "label": "geography–class cells with reliability grades"
3103 + },
3104 + {
3105 + "value": "1.58 vs 2.91",
3106 + "label": "Monte Carlo RMSE of municipal monthly changes, log points"
3107 + },
3108 + {
3109 + "value": "154.9",
3110 + "label": "provincial house index, September 2026 (2021 = 100)"
3111 + },
3112 + {
3113 + "value": "0.16 / 1.72",
3114 + "label": "first-release revisions, province / municipalities, log points"
3115 + }
3116 + ],
3117 + "contributions": [
3118 + {
3119 + "title": "An audited population with real-time vintages",
3120 + "description": "769,995 registered sales matched to the provincial assessment roll, an exclusion ledger leading to 632,006 headline sales, a measured publication lag of 10–13 days and monthly vintages that make revisions a measured quantity."
3121 + },
3122 + {
3123 + "title": "Integration of known components, tested as a whole",
3124 + "description": "A single-sale hedonic observation equation, a four-level administrative hierarchy, empirical-Bayes calibration of pooling strength and reliability accounting for every cell-month, explicitly presented as an integration rather than a new class of model."
3125 + },
3126 + {
3127 + "title": "Half the error of per-cell estimation in simulation",
3128 + "description": "Across nine Monte Carlo scenarios the municipal RMSE of monthly changes is 1.58 log points against 2.91 for per-cell state space, 12.52 for a cell-level time dummy and 24.28 for the median; empirical-Bayes calibration matters for coverage rather than point accuracy."
3129 + },
3130 + {
3131 + "title": "Two counter-examples that the method loses",
3132 + "description": "With heavy-tailed contamination a coarser location partition wins (2.84 vs 4.62), and at the regional level direct region-specific hedonic coefficients beat pooled coefficients plus a smoothed deviation (1.17 vs 1.40)."
3133 + },
3134 + {
3135 + "title": "Robust to composition, consistent with external indices",
3136 + "description": "Dropping the smallest 40% of houses from 2024 in the Montréal region shifts the median by +23.4 log points but SHARP by +0.3; Québec City house prices rise 47.9 log points over 2021–2026 against 45.5 for CREA and 43.7 for Teranet."
3137 + },
3138 + {
3139 + "title": "Reliability grades for every cell-month",
3140 + "description": "The panel of 2,654 geography–class cells over 69 months reports sale counts, effective sample sizes, local-data weights, 95% intervals and A–E grades; municipal house indices in grades A–C range from 133 (P10) to 174 (P90) in September 2026."
3141 + }
3142 + ],
3143 + "data": [
3144 + {
3145 + "title": "Quebec land-registry sales snapshot",
3146 + "description": "769,995 registered sales from 2021-01-04 to 2026-10-07 (Quebec land registry feed, Vrai Prix production database backed up 2026-10-21), geocoded and matched to the MAMH assessment roll (3.75 million units); four earlier extractions measure the publication lag; MRNF SDA V2026-07 boundaries."
3147 + },
3148 + {
3149 + "title": "External indices",
3150 + "description": "Teranet–National Bank, CREA MLS HPI and Statistics Canada NHPI/RPPI series downloaded 2026-10-21 for metropolitan-community comparisons in Québec City and Montréal."
3151 + }
3152 + ],
3153 + "methodology": [
3154 + {
3155 + "title": "Stage one: robust pooled hedonic regression",
3156 + "description": "Huber-IRLS fixed-effects regression with a nested-grid location partition, smoothly drifting quality prices and provincial month effects, estimated on the full sample."
3157 + },
3158 + {
3159 + "title": "Stage two: nested state-space deviations",
3160 + "description": "Each geography's mean residual is a noisy observation of a latent deviation from its parent, filtered and smoothed with a robust heteroskedastic local-level model whose innovation variance follows a grid-EM empirical-Bayes prior on log τ² across cells."
3161 + },
3162 + {
3163 + "title": "Eleven benchmarks on the same data",
3164 + "description": "Median, stratified median, pooled and rolling time dummies, hedonic imputation (Laspeyres, Paasche, Fisher), GEKS, HMTS, repeat sales (BMN and Case–Shiller), static empirical Bayes, out-of-time gradient boosting and a production hierarchical index."
3165 + },
3166 + {
3167 + "title": "Monte Carlo with known latent paths",
3168 + "description": "80 replications of nine synthetic Quebec-like scenarios, including quality drift with flat prices, heavy-tailed noise and two counter-examples, scoring RMSE, bias, lag and interval coverage at every level of the hierarchy."
3169 + },
3170 + {
3171 + "title": "Real-time and validation checks",
3172 + "description": "Monthly vintages with revision statistics and publication policies; placebo, composition-shock, cluster bootstrap, Moran, ablation, aggregation and trimming checks; external comparison with Teranet and CREA."
3173 + }
3174 + ],
3175 + "reproducibility": [
3176 + "Numbered pipeline scripts/01–62 (data, benchmarks, Monte Carlo, panel, vintages, validation, figures, tables) driven by a Makefile; pytest suite for state space, EB prior, estimators and panel invariance",
3177 + "Every number in the manuscript is a generated macro (paper/tables/numbers.tex); index panel, reliability grades, revision and simulation summaries released as CSV/parquet",
3178 + "Pre-specified research protocol with decision log, data dictionary, provenance report and limitations in docs/",
3179 + "Full LaTeX source of the 49-page paper with 115 verified bibliography entries; raw microdata not redistributed"
3180 + ],
3181 + "keywords": [
3182 + "house price index",
3183 + "hedonic regression",
3184 + "state-space model",
3185 + "small-area estimation",
3186 + "empirical Bayes",
3187 + "real-time data",
3188 + "Monte Carlo",
3189 + "thin markets",
3190 + "Quebec"
3191 + ],
3192 + "repo": "https://git.spboucher.ai/wp22_uqo",
3193 + "jel": "R31, C43, C11, C21, C55"
3194 + },
3195 + {
3196 + "slug": "wp23",
3197 + "hero": {
3198 + "headline": "On Quebec data, new forecasting architectures do not beat gradient boosting",
3199 + "subheadline": "LightGBM reaches a 21.21 log-point MAE on future sale prices against 22.36 for the hierarchical network; for index changes a pooled tree model attains MASE 0.75 at three months and nothing beats an autoregression at twelve."
3200 + },
3201 + "abstract": [
3202 + "This paper asks whether new forecasting architectures improve residential price forecasts over strong conventional methods, and under which conditions. Using the audited Quebec land-registry sample of UQO Working Paper 22 (632,006 sales, 2021–2026) and its real-time quality-adjusted index vintages, it defines three tasks in a strict pseudo-real-time design: the price of a property sold one to twelve months after a forecast origin (Task A), the one- to twelve-month change of municipal, regional and provincial quality-adjusted indices (Task B), and calibrated predictive distributions for both (Task C).",
3203 + "Five candidate architectures are implemented (a hierarchical spatiotemporal hedonic network, a dynamic spatial graph forecasting network, a regime-adaptive mixture of experts, a Bayesian hierarchical probabilistic forecaster and a quality-adjusted neural state-space model) together with an integrated stacked architecture, against 14 benchmarks ranging from no-change and momentum rules to gradient boosting, recurrent, convolutional and attention networks. Evaluation uses Diebold–Mariano and Giacomini–White tests, model confidence sets and block bootstraps, with Monte Carlo experiments on synthetic markets whose latent dynamics range from unforecastable random walks to regime switching and regional spillovers.",
3204 + "For the price of a future sale, gradient boosting on hedonic and real-time local market features is the most accurate model at every horizon and the hierarchical network does not improve on it. For index changes, one- to six-month movements are forecastable through the cell's own momentum and seasonality (MASE 0.75 at three months for a pooled tree model); spatial, regime and hierarchical components add nothing that survives ablation, and at twelve months nothing beats an autoregression. The state-space forecaster is the only model with calibrated intervals at every horizon. Every component of the architectures exists in the literature; the contribution is the measured answer, on transaction-level data, to the question of when complexity pays."
3205 + ],
3206 + "findings": [
3207 + {
3208 + "value": "632,006",
3209 + "label": "Quebec registry sales in the audited sample, 2021–2026"
3210 + },
3211 + {
3212 + "value": "6 vs 14",
3213 + "label": "candidate architectures against benchmark forecasters"
3214 + },
3215 + {
3216 + "value": "21.21",
3217 + "label": "LightGBM log-point MAE on sale prices, 1–3 months"
3218 + },
3219 + {
3220 + "value": "0.75",
3221 + "label": "best MASE for three-month index changes, pooled tree model"
3222 + },
3223 + {
3224 + "value": "0.63",
3225 + "label": "AR(2) MASE at twelve months, unbeaten by any architecture"
3226 + },
3227 + {
3228 + "value": "97%",
3229 + "label": "coverage of state-space 90% intervals at three months"
3230 + }
3231 + ],
3232 + "contributions": [
3233 + {
3234 + "title": "Three tasks in a leakage-audited real-time design",
3235 + "description": "For every monthly origin from December 2022 to September 2025 the information set contains only property characteristics, real-time local market features and the WP22 index vintages re-estimated at each publication date; no full-sample index enters a predictor."
3236 + },
3237 + {
3238 + "title": "Gradient boosting dominates the property-level task",
3239 + "description": "At one to three months LightGBM reaches a 21.21 log-point MAE against 22.36 for the hierarchical network, 23.24 for hedonic OLS and 34.96 for the municipal median; at seven to twelve months every model is biased by −4 to −9 log points because the market is not extrapolated."
3240 + },
3241 + {
3242 + "title": "Index changes are forecastable only through momentum",
3243 + "description": "At three months a pooled LightGBM reaches MASE 0.75 versus no change, with the gain concentrated in rising windows (0.63 against 1.00 in flat ones); after Holm adjustment no improvement survives a 10% family-wise error rate."
3244 + },
3245 + {
3246 + "title": "Spatial, regime and hierarchical components add nothing",
3247 + "description": "Removing the graph from the spatial network lowers its municipal six-month error by 0.59 log points, an equal-weight combination beats the learned stack by 0.38, and removing momentum from ridge raises error by 0.53; neural sequence models vary across seeds as much as across architectures."
3248 + },
3249 + {
3250 + "title": "At twelve months, nothing beats an autoregression",
3251 + "description": "AR(2) attains MASE 0.63 while every pooled model is worse than no change, inheriting a +6 to +11 log-point bias from the 2021–2023 boom."
3252 + },
3253 + {
3254 + "title": "Only the state-space forecaster is calibrated",
3255 + "description": "The quality-adjusted neural state-space model is the only architecture with honest intervals at every horizon (97% coverage of 90% intervals at three months, against 73–76% for LightGBM, the graph network and the attention benchmark)."
3256 + }
3257 + ],
3258 + "data": [
3259 + {
3260 + "title": "WP22 Quebec registry sample",
3261 + "description": "632,006 registered sales from January 2021 to 7 October 2026 with assessment-roll characteristics (no listing, bedroom, bathroom, condition or image data), read-only from the SHARP-HPI repository; forecast origins span December 2022 to September 2025 with 31–45 origins per horizon."
3262 + },
3263 + {
3264 + "title": "Real-time quality-adjusted index vintages",
3265 + "description": "SHARP-HPI vintages published from July 2022 to October 2026 (filtered and smoothed modes) for 82 cells (province, 17 regions, 64 municipalities) and 192 municipal cells by property class, with the retrospective final index as the Task B target."
3266 + }
3267 + ],
3268 + "methodology": [
3269 + {
3270 + "title": "Five architectures and an integrated stack",
3271 + "description": "Hierarchical spatiotemporal hedonic network, dynamic spatial graph forecasting network, regime-adaptive mixture of experts, Bayesian hierarchical probabilistic forecaster, quality-adjusted neural state-space model, and a stacked combination, each classified as an application, adaptation or new combination of known components."
3272 + },
3273 + {
3274 + "title": "Fourteen benchmarks with distributional outputs",
3275 + "description": "No-change and momentum rules, ARIMA-type, factor and state-space models, ridge, LightGBM, XGBoost, random forest, SVR, MLP, and LSTM, GRU, TCN and TFT-lite networks, each with a quantile or distributional output."
3276 + },
3277 + {
3278 + "title": "Forward-chaining inference for short samples",
3279 + "description": "Diebold–Mariano tests with horizon-adjusted long-run variances, Giacomini–White conditional tests, model confidence sets, stationary block bootstraps, scale-free MASE and proper scoring rules (CRPS, interval scores, Brier skill)."
3280 + },
3281 + {
3282 + "title": "Simulation with known latent dynamics",
3283 + "description": "The same forecasters run on synthetic markets built with the WP22 simulator: random walks, persistent momentum, Markov-switching regimes, regional spillovers and sparse cells, so the empirical ranking can be read against the generating mechanism."
3284 + },
3285 + {
3286 + "title": "Ablations and robustness",
3287 + "description": "Component ablations, seed variability, ex-post regime splits, directional accuracy at turning points, municipal-cell distributions and equal-weight versus learned combinations."
3288 + }
3289 + ],
3290 + "reproducibility": [
3291 + "Numbered pipeline scripts/10–60 (Task A, Task B, evaluation, interpretation, ablations, robustness, simulation, tables, figures) with src/wp23 modules and a pytest suite; heavy runs executed on a 32-core Mac Studio with the same code",
3292 + "Every number in the manuscript is a generated macro (paper/tables/numbers.tex); results stored per task, evaluation, ablation, robustness and simulation",
3293 + "Leakage audit (docs/leakage_audit.md), decision log, limitations and a verified literature review with novelty scan (152 references)",
3294 + "Full LaTeX source of the 56-page paper; inputs are read-only WP22 artefacts (raw microdata not redistributed)"
3295 + ],
3296 + "keywords": [
3297 + "house price forecasting",
3298 + "machine learning",
3299 + "graph neural networks",
3300 + "state-space models",
3301 + "hierarchical Bayes",
3302 + "regime switching",
3303 + "probabilistic forecasting",
3304 + "pseudo-real-time evaluation",
3305 + "gradient boosting",
3306 + "Quebec"
3307 + ],
3308 + "repo": "https://git.spboucher.ai/wp23_uqo",
3309 + "jel": "R31, C53, C45, C11, C21, C55"
3310 + },
3311 + {
3312 + "slug": "wp24",
3313 + "hero": {
3314 + "headline": "Labor's share vanishes under AGI; its wage level depends on compute scarcity",
3315 + "subheadline": "In the calibrated baseline the labor share falls from 0.56 to 0.283 by 2050, the real return on wealth peaks at 40.5% in 2060, and the wealth-to-income ratio falls to 2.7 before land lifts it to 31.0 by 2100."
3316 + },
3317 + "abstract": [
3318 + "Restrepo (2025) shows that once artificial general intelligence lets compute perform every task, output becomes linear in compute, wages converge to the value of the compute a worker saves, and the labor share goes to zero. This paper extends his task economy by treating compute as a reproducible, priced, depreciating capital good that needs energy and works alongside structures and land, and by adding households that own the claims to income.",
3319 + "The labor-share result survives any constant-returns production structure: it needs only within-task substitutability. The wage-level result does not: when physical complements are gross complements and compute becomes cheap, the price of compute and wages go to zero in level and the income that labor loses goes to the owners of the scarcest complement; with a floor on the compute price, the wage equals the user cost of the replacement compute. Income shares and asset values are different objects: compute is capitalized at the duration of chips, so capital's share can rise while the wealth-to-income ratio falls, and a growth boom raises the real rate and compresses valuation ratios when the elasticity of intertemporal substitution is below one.",
3320 + "In a calibrated model solved along perfect-foresight paths to 2300 under conditional scenarios, the labor share falls to 0.283 by 2050, the real return peaks at 40.5%, the wealth-to-income ratio falls to 2.7 before land captures 0.605 of income and raises it to 31.0 by 2100, and the market value of compute never exceeds one fifth of GDP. A compute tax raises wages one-for-one with the tax-inclusive user cost; a land tax redistributes without distortion; a citizen fund keeps workers' consumption share bounded away from zero."
3321 + ],
3322 + "findings": [
3323 + {
3324 + "value": "0.283",
3325 + "label": "labor share in 2050, down from 0.56 in 2026"
3326 + },
3327 + {
3328 + "value": "40.5%",
3329 + "label": "peak real return on wealth, in 2060"
3330 + },
3331 + {
3332 + "value": "2.7 → 31.0",
3333 + "label": "wealth-to-income ratio, 2042 trough to 2100"
3334 + },
3335 + {
3336 + "value": "0.605",
3337 + "label": "share of income captured by land by 2100"
3338 + },
3339 + {
3340 + "value": "≤ 1/5 of GDP",
3341 + "label": "market value of compute capital along every path"
3342 + },
3343 + {
3344 + "value": "25",
3345 + "label": "conditional scenarios solved over 275 years"
3346 + }
3347 + ],
3348 + "contributions": [
3349 + {
3350 + "title": "The labor-share result is more robust than claimed",
3351 + "description": "The labor share goes to zero under any constant-returns production structure, with or without complements and whether compute is an endowment or a priced capital good, because it needs only perfect substitutability of labor and compute within tasks (Proposition 1)."
3352 + },
3353 + {
3354 + "title": "The wage-level result is less robust",
3355 + "description": "With gross complements (σ < 1) and cheap compute, the price of compute and wages go to zero in level and lost labor income accrues to the owners of the scarcest complement; with a price floor the wage equals the user cost of the replacement compute, (r + δ_Q) p̄ CEU (Propositions 2–4); an energy bottleneck keeps the labor share positive (Proposition 5)."
3356 + },
3357 + {
3358 + "title": "Income shares and asset values diverge",
3359 + "description": "On a balanced path the wealth-to-income ratio is Σ s_k/[(1+r)(1+π_k) − (1−δ_k)] + s_T/(r−g): compute, depreciating at 25% and cheapening at 15% a year, is capitalized at a price–earnings ratio of about two against eight for structures and fifty for land, so capital's share can rise while wealth-to-income falls (Propositions 6–7)."
3360 + },
3361 + {
3362 + "title": "A calibrated transition to 2300",
3363 + "description": "In the baseline the routine task is automated by 2029 and the expert task by 2057; output rises 3.7-fold by 2050 and 233-fold by 2100; the high-skill wage reaches 3.51 times its 2026 level while its task is the last unautomated, then collapses."
3364 + },
3365 + {
3366 + "title": "Policy in general equilibrium",
3367 + "description": "A compute tax raises wages one-for-one with the tax-inclusive user cost at a cost in output, a land tax redistributes without changing allocations, a citizen fund holding one fifth of all claims bounds workers' consumption share away from zero, and an AI monopoly protects wages like a compute tax while rents go to the monopolist."
3368 + },
3369 + {
3370 + "title": "An audited replication of the source",
3371 + "description": "Restrepo (2025) is reproduced exactly in a finite-task CES economy (Propositions 1–6 hold to 1e-6), and the quantitative equilibria satisfy Euler residuals below 1e-9 and income identities below 1e-8."
3372 + }
3373 + ],
3374 + "data": [
3375 + {
3376 + "title": "Model environment",
3377 + "description": "A task economy with four tasks and two skills, four capital stocks with Tobin's q (compute, equipment, structures, housing structures), land, energy drawn from a fixed resource, a two-class economy whose capitalists price every claim, and a heterogeneous-agent economy (2 skills × 3 discount types × 5 productivity states) holding claims through a fund; perfect-foresight paths over 275 years to 2300."
3378 + },
3379 + {
3380 + "title": "Calibration to the United States in 2026",
3381 + "description": "162 public targets retrieved 2026-10-27 with URLs, vintages and SHA-256 hashes: PWT labor share, BEA fixed assets and NIPA via FRED, Fed Z.1 and Distributional Financial Accounts, TIPS and Cleveland Fed real rates, Shiller CAPE and home prices, LBNL data-centre electricity (4.4% in 2023), EIA generation, Epoch AI GPU price–performance (doubling every 2.02 years) and the BEA 2023 use table."
3382 + },
3383 + {
3384 + "title": "Conditional scenarios",
3385 + "description": "25 scenarios on capability speed, Moore's law and its end, energy constraints, elasticities, scarce land, chip depreciation, monopoly, foreign ownership, land, compute and capital taxes, a citizen fund, task reinstatement and risk aversion; the baseline has capability improving 10% a year and compute prices falling 15% a year to one hundredth of today's level."
3386 + }
3387 + ],
3388 + "methodology": [
3389 + {
3390 + "title": "Task block with priced compute",
3391 + "description": "The static allocation of labor and compute across tasks is a concave program solved by SLSQP with a Newton polish on the active set; prices follow from the envelope theorem, with an energy residual and a utilization margin."
3392 + },
3393 + {
3394 + "title": "Propositions in a limit economy",
3395 + "description": "Eight propositions on labor share, wage levels, balanced AGI growth, energy bottlenecks, valuation ratios, the discount-rate channel and tax incidence are proved in a one-complement limit economy (Appendix A)."
3396 + },
3397 + {
3398 + "title": "Perfect-foresight transition by banded Newton",
3399 + "description": "Unknowns per year are four log stocks, capitalists' expenditure and the real rate; residuals are four Tobin's-q Euler equations, goods-market clearing and the household Euler equation, solved directly over 275 years with a colored finite-difference banded Jacobian and a homotopy fallback."
3400 + },
3401 + {
3402 + "title": "Heterogeneous households in general equilibrium",
3403 + "description": "Endogenous grid method with a housing price index and non-stochastic simulation; general equilibrium by damped quasi-Newton on the return path, with welfare and distribution statistics by skill and wealth."
3404 + },
3405 + {
3406 + "title": "Valuation decomposition",
3407 + "description": "Campbell–Shiller decomposition of 2026 valuations into cash-flow and discount-rate news, which are of the same order and opposite sign, plus a falsifiable short-run empirical design on the compute-equivalent wage (Section 11)."
3408 + }
3409 + ],
3410 + "reproducibility": [
3411 + "One-command reproduction (make all: setup, test, benchmark, scenarios, ha, figures, tables, paper, slides) with pinned requirements; pytest covers the Restrepo replication, KKT conditions, accounting identities, balanced growth and equilibria",
3412 + "25 scenarios × 275 years solved with numerical diagnostics reported (Euler residuals < 1e-9, income identity < 1e-8, horizon check < 1e-7); 10 figures, 8 tables and numbers.tex regenerated by scripts",
3413 + "Public data only, with a manifest of URLs, vintages and SHA-256 hashes; source audit of the Restrepo (2025) NBER draft, assumptions register, identification note and an adversarial referee report in docs/",
3414 + "Full LaTeX source of the 55-page paper (118 verified references) and Beamer slides"
3415 + ],
3416 + "keywords": [
3417 + "artificial general intelligence",
3418 + "task-based automation",
3419 + "labor share",
3420 + "compute capital",
3421 + "asset pricing",
3422 + "wealth distribution",
3423 + "land rents",
3424 + "energy constraints",
3425 + "general equilibrium"
3426 + ],
3427 + "repo": "https://git.spboucher.ai/wp24_uqo",
3428 + "jel": "E22, E24, E25, G12, J23, J24, O33, O41"
3429 + },
3430 + {
3431 + "slug": "wp25",
3432 + "hero": {
3433 + "headline": "Shared models turn noise that cancels into noise that does not",
3434 + "subheadline": "Capital, wages and asset prices load on model noise with weight D²/M; the private value of delegation equals its social value under constant returns, and one shared model raises order-flow correlation from 0.11 to 0.97."
3435 + },
3436 + "abstract": [
3437 + "Households and firms are beginning to delegate economic decisions to autonomous agents built on a few shared foundation models. This paper studies what this does in general equilibrium. A human's decision error is idiosyncratic and averages out; an AI agent's error is common to every principal that uses the same model. Delegation therefore converts noise that cancels into noise that does not: capital accumulation, asset demand and credit load on the shared error with weight D (the delegation share) and 1/M (the effective number of independent models), and wages, interest rates and asset prices inherit this model noise.",
3438 + "In a task-based stochastic growth economy with a compute market, the decision problem reduces to a linear-quadratic game with dispersed information in which delegation is the purchase of a correlated signal. In a price-taking economy with constant returns the private value of delegation equals its social marginal value identically: model noise is real but not an inefficiency. Closed-form wedges, Pigouvian delegation taxes and the rate at which model diversity substitutes for them are obtained when the aggregate action has non-pecuniary effects, when price risk is uninsurable, or when prices enter collateral constraints. Delegating the consumption–savings decision has almost no aggregate bite; delegating the allocation of wealth has a large one. In a heterogeneous-agent economy the rich delegate first, better decisions concentrate wealth even with broad ownership of AI, and the distributional effect of automation is decided by who owns the compute.",
3439 + "In an agentic market laboratory with explicit balance sheets, margin rules, defaults, three market designs and live open-weight language-model agents under hard deterministic constraints, order-flow correlation rises with D and falls with M in every design; a shared model makes markets quieter and more correlated at once; and a false shared signal is amplified by collateral in a call market (by a factor of 2.6 at a 20% margin, halved at 50%, eliminated by model diversity) but not in a continuous auction. Language-model agents herd less than Bayesian agents on the same shared signal but load prices on it more, and their proposed actions routinely violate the constraints that deterministic code enforces. Code, data provenance, prompts and audit logs are provided."
3440 + ],
3441 + "findings": [
3442 + {
3443 + "value": "0.11 → 0.97",
3444 + "label": "order-flow correlation, private signals to one shared model"
3445 + },
3446 + {
3447 + "value": "0.43",
3448 + "label": "order-flow correlation with eight independent models"
3449 + },
3450 + {
3451 + "value": "0",
3452 + "label": "private–social wedge of delegation under constant returns"
3453 + },
3454 + {
3455 + "value": "×2.6",
3456 + "label": "collateral amplification of a false shared signal, 20% margin"
3457 + },
3458 + {
3459 + "value": "7.7%",
3460 + "label": "wage model-noise s.d. at full adoption of one model"
3461 + },
3462 + {
3463 + "value": "0.68 → 0.70",
3464 + "label": "wealth Gini from the delegation channel alone"
3465 + }
3466 + ],
3467 + "contributions": [
3468 + {
3469 + "title": "A delegation market sets the error-covariance structure",
3470 + "description": "The delegation share D and the effective number of independent models M are sufficient statistics for how much capital, wages, interest rates and asset prices load on model noise, with closed-form loadings proportional to D²/M (Proposition 2)."
3471 + },
3472 + {
3473 + "title": "Delegation is efficient in the benchmark economy",
3474 + "description": "In a price-taking economy with constant returns and complete markets the private value of delegating equals its social marginal value identically, for every D, M and degree of strategic interaction (Proposition 3), derived symbolically before any simulation and property-tested (maximum wedge 1.9e-7 on the grid)."
3475 + },
3476 + {
3477 + "title": "Closed-form wedges and taxes under three frictions",
3478 + "description": "Non-pecuniary aggregate effects, uninsurable price risk and collateral constraints each yield a signed wedge, a Pigouvian delegation tax (4.5 times the fee at M = 1 and 0.8 at M = 8 with 50% hand-to-mouth households) and the rate 1/M at which model diversity substitutes for it."
3479 + },
3480 + {
3481 + "title": "Which decision is delegated matters most",
3482 + "description": "Delegating consumption–savings has almost no aggregate bite (a one-log-point productivity news moves desired savings by less than 0.002), while delegating portfolio allocation gives the first adopter about 3% of consumption a year and the last 0.4%, and a wage model-noise standard deviation of 7.7% with one model against 3.2% with eight."
3483 + },
3484 + {
3485 + "title": "The rich delegate first and ownership decides distribution",
3486 + "description": "In the Aiyagari economy the information channel alone raises the wealth Gini from 0.68 to 0.70 while adding only 3% to output; AI as a production input lowers the Gini to 0.58 under broad ownership of compute and to 0.66 under concentrated ownership."
3487 + },
3488 + {
3489 + "title": "Agentic markets: quieter, more correlated, conditionally fragile",
3490 + "description": "Across call, continuous-auction and bilateral designs with common random numbers, order-flow correlation moves 0.11 / 0.97 / 0.43 for private signals, one model and eight; in the call market a false shared signal moves prices 3.7% unlevered and 9.8% at a 20% margin, halved at 50% and eliminated by four models, while language-model agents (Qwen2.5-7B, Mistral-7B, Llama-3.2-3B) herd less than Bayesian agents yet load prices on the shared error more."
3491 + }
3492 + ],
3493 + "data": [
3494 + {
3495 + "title": "Three nested models",
3496 + "description": "Model A: a task-based stochastic growth economy with a compute market shared by production automation and inference, whose within-period problem is a linear-quadratic game with dispersed information. Model B: a stationary Aiyagari economy with a discrete delegation choice, a decision-quality return and broad or concentrated ownership of compute. Model C: an agentic market with balance sheets, collateralised credit, margin calls and defaults under Walrasian call, continuous double auction and bilateral bargaining, run over 20 seeds with common random numbers."
3497 + },
3498 + {
3499 + "title": "Calibration targets",
3500 + "description": "Public series fetched 2026-10-27 with provenance and SHA-256 hashes: 23 FRED series, Fed Distributional Financial Accounts, Fernald TFP, Shiller, SEC XBRL capex, EIA, Epoch AI and Census BTOS; the AI-specific behavioural and information parameters are illustrative, recorded in a parameter ledger and swept rather than estimated."
3501 + },
3502 + {
3503 + "title": "LLM laboratory agents",
3504 + "description": "Live open-weight models (Qwen2.5-7B, Mistral-7B-v0.3, Llama-3.2-3B, 4-bit, served locally) and a deterministic surrogate behind the same interface; every action is schema-validated and clipped by deterministic code before touching a ledger, with prompts, audit logs and a replayable cache released."
3505 + }
3506 + ],
3507 + "methodology": [
3508 + {
3509 + "title": "Symbolic derivation of the delegation game",
3510 + "description": "The linear-quadratic game with dispersed information in the Angeletos–Pavan class is solved symbolically (sympy), yielding the equilibrium loadings, adoption threshold, efficiency benchmark and friction wedges of Propositions 1–7 before any numerical work."
3511 + },
3512 + {
3513 + "title": "Mapping the economy to the game",
3514 + "description": "Steady state with a linearised sequence-space household and a mean-variance portfolio mapping translate the production economy into game parameters, separating the savings and portfolio channels of delegation."
3515 + },
3516 + {
3517 + "title": "Heterogeneous-agent computation",
3518 + "description": "Value-function iteration with a discrete delegation choice, Howard acceleration and non-stochastic simulation, with the interest rate cleared by Brent's method and compute ownership assigned broadly or to an owner class."
3519 + },
3520 + {
3521 + "title": "Factorial agentic-market experiments",
3522 + "description": "Market design × delegation share × number of models with common random numbers and bootstrap bands; stress tests with a false shared signal under unlevered, levered (20% and 50% margin) and model-diverse configurations; ledger invariants asserted at every step."
3523 + },
3524 + {
3525 + "title": "Robustness and pre-registration",
3526 + "description": "Sweeps of risk aversion, persistence and the effective number of models (1 to 16); pre-registered hypotheses H1–H6 with the compute crowding-out hypothesis reported as a negative result (labour share moves by at most 0.2 percentage points)."
3527 + }
3528 + ],
3529 + "reproducibility": [
3530 + "End-to-end scripts 00–50 (symbolic derivations, data download, calibration targets, Models A–C, LLM lab, figures, tables) and 30 unit, integration and property tests including the zero-wedge property and Euler's theorem",
3531 + "Pre-registration (experiments/pre_registration.md), parameter ledger, model specification, limitations and an adversarial referee report in docs/",
3532 + "LLM experiments replayable from results/llm_cache with full audit logs; prompts and action schemas in the appendices",
3533 + "Full LaTeX source of the 71-page paper with 144 verified references; raw data re-downloadable from the manifest, not redistributed"
3534 + ],
3535 + "keywords": [
3536 + "artificial intelligence",
3537 + "delegation",
3538 + "foundation models",
3539 + "dispersed information",
3540 + "general equilibrium",
3541 + "compute",
3542 + "wealth inequality",
3543 + "market microstructure",
3544 + "LLM agents",
3545 + "model noise"
3546 + ],
3547 + "repo": "https://git.spboucher.ai/wp25_uqo",
3548 + "jel": "D82, D83, E13, E21, E44, G12, G14, O33"
3549 + },
3550 + {
3551 + "slug": "wp26",
3552 + "hero": {
3553 + "headline": "Deliberation among LLM valuation agents manufactures agreement, not accuracy",
3554 + "subheadline": "In a preregistered experiment on 631,866 Quebec sales, a two-round debate scores 0.158 mean absolute log error against 0.159 for independent agents and 0.157 for a placebo committee; rounds halve disagreement without moving error, and every preregistered contrast includes zero."
3555 + },
3556 + "abstract": [
3557 + "Committees of role-specialised language-model agents that debate before answering are increasingly proposed as valuation tools. This paper asks whether the debate adds information. In a preregistered experiment on 631,866 arm's-length residential sales registered in Quebec between 2021 and 2026, evaluated under a leakage-audited as-of information set, a hedonic regression, a hierarchical repeat-sales model, gradient boosting and a conformal automated valuation model (AVM) are compared with a single language model, twelve independent role-specialised agents, structured debate of one to four rounds, debate with a blinded skeptic and deterministic verification, a placebo committee and a cost-matched single agent, all fed identical evidence packets.",
3558 + "A model of valuation as decentralised information aggregation shows that opinion exchange contracts dispersion without reducing error, that evidence exchange helps only outside the common view, and that no linear deliberation beats the best round-0 pool. The data agree: on the test subsample the debate reaches a mean absolute log error of 0.158 against 0.159 for the silent committee, 0.157 for the placebo and 0.163 for the cost-matched agent; every preregistered paired contrast includes zero (Holm-adjusted p = 1), and rounds halve the agents' disagreement without moving accuracy.",
3559 + "Committees match the AVM (0.171 on the full test partition) but state intervals about 30 percent too narrow, covering 86–89 percent of prices at a nominal 90 percent, and in the 2026 holdout every agentic arm overvalues by ten log points where the AVM is unbiased. A two-round debate costs about 1.5 US dollars and 5.6 minutes per property. Forecasting accuracy is kept distinct from fundamental value, which is triangulated with 92,089 resales, rents and user cost; the human-appraiser arm is pending."
3560 + ],
3561 + "findings": [
3562 + {
3563 + "value": "631,866",
3564 + "label": "registered Quebec residential sales, 2021–2026"
3565 + },
3566 + {
3567 + "value": "0.158 vs 0.159",
3568 + "label": "mean absolute log error, debate vs silent committee"
3569 + },
3570 + {
3571 + "value": "[−0.006, 0.005]",
3572 + "label": "95% interval of the debate contrast, Holm p = 1"
3573 + },
3574 + {
3575 + "value": "0.171",
3576 + "label": "AVM mean absolute log error, full test partition"
3577 + },
3578 + {
3579 + "value": "+0.10",
3580 + "label": "log-point overvaluation of every agentic arm in 2026"
3581 + },
3582 + {
3583 + "value": "$1.52",
3584 + "label": "cost of a two-round debate per property"
3585 + }
3586 + ],
3587 + "contributions": [
3588 + {
3589 + "title": "A design that isolates communication",
3590 + "description": "The independent committee supplies the round-0 estimates of the debate, a cost-matched single agent supplies the same tokens without roles, a placebo committee the same views without mandates; the four primary contrasts were preregistered and frozen before any test property was valued."
3591 + },
3592 + {
3593 + "title": "A precise null on deliberation",
3594 + "description": "Debate improves on the silent committee by 0.001 log points (95% interval [−0.006, 0.005]), on the placebo by nothing ([−0.005, 0.006]), on the cost-matched agent by 0.006 ([−0.017, 0.004]); in the 2026 holdout and prospective set the debated committee is marginally worse."
3595 + },
3596 + {
3597 + "title": "Rounds contract dispersion, not error",
3598 + "description": "As the aggregation theory predicts, the interquartile range of agents' estimates roughly halves after the first round and keeps falling (anchoring index 0.7–0.8) while the error of the median does not move."
3599 + },
3600 + {
3601 + "title": "Committees match the AVM but misstate uncertainty",
3602 + "description": "Agentic arms are statistically indistinguishable from the conformal AVM on the test subsample, yet their intervals are about 30 percent narrower, covering 86–89 percent at nominal 90 (78–81 percent at the small-model tier)."
3603 + },
3604 + {
3605 + "title": "Common-model drift in the 2026 holdout",
3606 + "description": "Every agentic arm overvalues by ten log points where the AVM has bias 0.002, a shared error that no committee size or deliberation removes."
3607 + },
3608 + {
3609 + "title": "Fundamental value kept separate from price",
3610 + "description": "On 92,089 sales with a later resale, the boosted baselines lie closer to the market-adjusted resale value than the realised first price does; the same test for the agentic arms is specified and deferred."
3611 + }
3612 + ],
3613 + "data": [
3614 + {
3615 + "title": "Registered Quebec residential sales",
3616 + "description": "631,866 arm's-length sales (422,542 houses, 122,976 condominiums, 86,348 small plexes) from the public land register joined to the municipal assessment roll, January 2021 to October 2026; chronological splits (train to 2024-06, validation to 2025-03, test 2025-04 to 2025-12, 2026 holdout), a three-region spatial holdout and a prospective set of 1,182 post-freeze registrations."
3617 + },
3618 + {
3619 + "title": "Evidence packets and external layers",
3620 + "description": "Per-property packets with roll attributes, up to twelve comparables sold before the valuation date, local market state and flood, transit and census context; coordinates rounded to three decimals, no addresses, nine leakage checks enforcing a sale-date-minus-one-day cutoff."
3621 + },
3622 + {
3623 + "title": "Resales, rents and user cost",
3624 + "description": "92,089 sales with a later resale, rental-board leases and a user-cost diagnostic used to triangulate fundamental value independently of the first transaction price."
3625 + }
3626 + ],
3627 + "methodology": [
3628 + {
3629 + "title": "Information-aggregation theory",
3630 + "description": "Eleven propositions, each mapped to a preregistered contrast: correlated-signal pools have an MSE floor, revision toward a committee statistic contracts dispersion by (1−λ)^r without changing error, and no linear deliberation beats the optimal round-0 pool."
3631 + },
3632 + {
3633 + "title": "Ten arms on identical evidence",
3634 + "description": "Four tuned baselines (hedonic ridge, hierarchical repeat-sales, LightGBM, conformal AVM) and six agentic arms (single LLM, twelve independent roles, 1–4 round debate, debate with blinded skeptic and verification, placebo roles, token-matched single agent) at matched list prices across two providers."
3635 + },
3636 + {
3637 + "title": "Preregistered paired inference",
3638 + "description": "Primary endpoint mean absolute log error; paired block bootstrap over municipality-by-month cells (2,000 draws) with Holm correction over four primary contrasts; protocol frozen with a digest before any test property was valued."
3639 + },
3640 + {
3641 + "title": "Calibration and faithfulness audits",
3642 + "description": "Coverage at 50/80/90/95 percent, PIT histograms, verification that every cited evidence identifier exists in the packet, that 96–99 percent of adjustments lie within forty percent of the raw comparable price and that about 99 percent of points lie inside the comparables' hull."
3643 + },
3644 + {
3645 + "title": "Cost and run-to-run variation",
3646 + "description": "Full cost ledger per arm (about $0.02 for one model, $0.23 for the silent committee, $1.52 and 5.6 minutes for a two-round debate) and measured non-determinism of hosted models (mean run-to-run change 0.007 log points)."
3647 + }
3648 + ],
3649 + "reproducibility": [
3650 + "Preregistration in docs/preregistration.md and protocol freeze results/freeze.json (2026-10-28, digest f0e9d6a9…); role prompts released with their hashes",
3651 + "Full Python pipeline (scripts 01–62: data, features, baselines, runs, evaluation, theory, fundamental value, prospective set) driven by a Makefile; 143 tests on synthetic data with a mock provider",
3652 + "Labelled synthetic dataset and mock LLM adapter let the whole pipeline run without API keys or private microdata; live-run predictions, manifests and ledgers are committed",
3653 + "All figures and LaTeX tables regenerated by scripts; full LaTeX source of the 129-page paper with proofs, prompts and output contract in the appendices"
3654 + ],
3655 + "keywords": [
3656 + "automated valuation models",
3657 + "large language models",
3658 + "multi-agent deliberation",
3659 + "information aggregation",
3660 + "house prices",
3661 + "calibration",
3662 + "appraisal",
3663 + "preregistration",
3664 + "Quebec"
3665 + ],
3666 + "repo": "https://git.spboucher.ai/wp26_uqo",
3667 + "jel": "R31, C53, C45, D83, G21"
3668 + },
3669 + {
3670 + "slug": "wp27",
3671 + "hero": {
3672 + "headline": "LLM traders make a market deep, calm and slow to learn",
3673 + "subheadline": "Raising the LLM share from 0 to 95 percent of slots tightens spreads from 18.6 to 6.1 basis points, multiplies depth by seven and leaves pricing error unchanged, while the five-tick variance ratio climbs from 1.1 to 2.1."
3674 + },
3675 + "abstract": [
3676 + "What happens to price discovery when a growing share of the traders in a market are autonomous language-model agents? The paper builds an event-sourced continuous double-auction exchange with exact accounting, a latent dividend process that separates fundamental value from public value and transaction price, seven calibrated classical trader types, and an adapter that lets a language model trade from a canonical snapshot through a strictly validated JSON action. Randomized, matched-seed experiments assign 0 to 100 percent of trader slots to a small reasoning model (gpt-5.4-nano), holding fundamentals, news, endowments, latencies and information sets fixed, under a pre-registered analysis plan with seven matched seeds per live cell and thirty per classical cell.",
3677 + "A stylized model with correlated signal errors predicts that a shared model degrades informativeness, raises order-flow correlation and amplifies leverage-driven spirals, and that model diversity restores informativeness. The simulator confirms the predictions on coordination and diversity and rejects them on prices: raising the LLM share tightens quoted spreads from 19 to 6 basis points, multiplies depth by seven, cuts price impact by 60 percent and leaves the pricing error unchanged, while the market absorbs latent information more slowly and returns become predictable (variance ratio 2.1, lag-one autocorrelation 0.32 at full penetration).",
3678 + "No bubble or crash appears in 112 live runs, and with leveraged traders present the LLM agents halve the pricing error and remove the bubble episodes that leverage creates. Prompt diversity and a two-model mix lower coordination and shorten absorption at a liquidity cost; a communication channel and memory raise coordination and pricing error slightly. The unifying mechanism is patient, public-value-anchored liquidity provision by agents who act infrequently. All results are causal statements about the simulator, reproducible from a one-command replication package; the Anthropic cells of the design are implemented but could not be run."
3679 + ],
3680 + "findings": [
3681 + {
3682 + "value": "18.6 → 6.1 bp",
3683 + "label": "quoted spread as LLM share rises to 95%"
3684 + },
3685 + {
3686 + "value": "×7",
3687 + "label": "order-book depth at full LLM penetration"
3688 + },
3689 + {
3690 + "value": "−60%",
3691 + "label": "price impact per 100 lots"
3692 + },
3693 + {
3694 + "value": "0",
3695 + "label": "bubbles, crashes or halts in 112 live runs"
3696 + },
3697 + {
3698 + "value": "0.02 → 0.32",
3699 + "label": "lag-one return autocorrelation, 0% to 95% LLM"
3700 + },
3701 + {
3702 + "value": "2.16% → 0.96%",
3703 + "label": "tracking error with leverage, 0% vs 25% LLM"
3704 + }
3705 + ],
3706 + "contributions": [
3707 + {
3708 + "title": "A counterfactual market with randomized LLM share",
3709 + "description": "One seed fixes dividends, news, endowments, latencies and slot assignments; the treatment replaces a stratified subset of slots by LLM agents inheriting their information set, so matched pairs differ only in who decides."
3710 + },
3711 + {
3712 + "title": "Four prices kept apart",
3713 + "description": "Latent fundamental value, public value from a Kalman filter on announced dividends and news, transaction price and quote midpoint are distinct, so price discovery is measured against the truth rather than a known target."
3714 + },
3715 + {
3716 + "title": "Theory confirmed on coordination, rejected on prices",
3717 + "description": "LLM agents deciding at the same tick are on the same side nine times out of ten and a communication channel raises coordination, yet pricing error is unchanged (−0.18 percentage points, interval [−0.67, +0.30]) and spreads fall by 12.5 basis points (d = −2.2)."
3718 + },
3719 + {
3720 + "title": "LLM agents dampen leverage spirals",
3721 + "description": "With 19 percent leveraged traders, replacing 25 percent of slots by LLM agents lowers tracking error from 2.16 to 0.96 percent (p = 0.03) and removes the bubble episodes (0.7 per run to 0) that leverage creates in the classical economy."
3722 + },
3723 + {
3724 + "title": "The cost is slower, predictable price paths",
3725 + "description": "Latent information is absorbed more slowly as the share rises; the five-tick variance ratio climbs from 1.14 to 2.12 and return autocorrelation from 0.02 to 0.32 (d = 3.5), a drift that trend followers can exploit."
3726 + },
3727 + {
3728 + "title": "Diversity trades coordination for liquidity",
3729 + "description": "Diverse prompts or a two-model mix at 25 percent lower pairwise flow correlation from 0.40 to 0.32–0.30 and absorb shocks four ticks faster, at a spread cost of 0.5–4.6 basis points."
3730 + }
3731 + ],
3732 + "data": [
3733 + {
3734 + "title": "Simulated exchange economy",
3735 + "description": "Entirely synthetic: 100-trader markets of 600 ticks with dividends every 20 ticks, a latent AR(1) innovation, private signals and news; seven classical trader types calibrated to stylized facts (near-zero return autocorrelation, fat tails, volatility clustering, 10–30 bp spreads); every run stores config, manifest, ticks, trades, agents and LLM decisions as parquet."
3736 + },
3737 + {
3738 + "title": "Live LLM decision records",
3739 + "description": "80,031 decisions by gpt-5.4-nano (and gpt-5.4-mini in two-model and equal-budget arms) in the confirmatory design plus 35,652 in the dose arm, seven matched seeds per live cell across six shares, three cognition arms, a leverage interaction, two safeguards and two fairness arms; total API spend 56 USD."
3740 + },
3741 + {
3742 + "title": "Classical validation design",
3743 + "description": "Sixteen cells × thirty seeds without any language model, plus a 360-run calibration sweep, reproducing bubbles under leverage and trend-heavy populations, misleading-news effects and two-asset contagion through common funding."
3744 + }
3745 + ],
3746 + "methodology": [
3747 + {
3748 + "title": "Event-sourced limit-order-book simulator",
3749 + "description": "Continuous double auction with price–time priority, partial fills, fees, per-agent latency, order TTL, price bands, circuit breaker, position limits, call auctions and margin-driven liquidation; integer-cent accounting checked after every fill with deterministic replay (24 unit, property and integration tests)."
3750 + },
3751 + {
3752 + "title": "Stylized correlated-error model",
3753 + "description": "Six propositions on informativeness versus shared-model share, order-flow correlation and synchrony, effective number of models, feedback amplification, leverage spirals and safeguards; each primitive maps to a simulator parameter and the sign of every prediction was pre-registered."
3754 + },
3755 + {
3756 + "title": "Common random numbers and paired inference",
3757 + "description": "Named random streams per seed; outcomes at the run level (pricing error, absorption time, variance ratios, spreads, depth, Kyle lambda, bubble and crash thresholds, synchrony); paired t with 95 percent intervals, Cohen's d, Wilcoxon, seed-fixed-effects OLS, Holm within hypothesis families and local projections."
3758 + },
3759 + {
3760 + "title": "Strictly validated LLM adapter",
3761 + "description": "OpenAI Responses API with JSON schema, invalid actions mapped to hold and logged, bounded memory, optional cohort communication channel and prompt personas, USD and token caps, SQLite decision cache and a frozen k-NN decision library for zero-cost replay at 1,000 traders."
3762 + }
3763 + ],
3764 + "reproducibility": [
3765 + "One-command replication: make replicate regenerates classical designs, theory checks, figures and tables without an API key in about five minutes; make confirmatory reruns the live design for about 60 USD",
3766 + "Pre-analysis plan in docs/identification.md and docs/protocol.md; every post-plan change logged in docs/research_log.md; literature ledger of 143 verified references",
3767 + "Every run directory carries a manifest with config hash, git revision, seeds, model ids, prompt version and realized LLM shares; 24 tests including hypothesis-based property tests",
3768 + "Full LaTeX source of the 101-page paper with proofs, prompts, algorithms and hyperparameters; SSRN 7679801"
3769 + ],
3770 + "keywords": [
3771 + "price discovery",
3772 + "large language models",
3773 + "agent-based markets",
3774 + "limit order book",
3775 + "herding",
3776 + "liquidity spirals",
3777 + "randomized simulation experiments",
3778 + "market microstructure",
3779 + "algorithmic monoculture"
3780 + ],
3781 + "repo": "https://git.spboucher.ai/wp27_uqo",
3782 + "jel": "G14, G12, D53, C63, C15, D83"
3783 + },
3784 + {
3785 + "slug": "wp28",
3786 + "hero": {
3787 + "headline": "One synchronization round is worth a doubling of local jackknife order",
3788 + "subheadline": "Local order-r debiasing plus q aggregated Newton steps matches the centralized MLE whenever K = o(N^(1−1/(2^(q+1) r))); the half-block jackknife with one step reaches K = o(N^(7/8)) against the K = O(N^(1/2)) regime of plain averaging."
3789 + },
3790 + "abstract": [
3791 + "Distributed maximum likelihood estimation trades statistical efficiency for communication. The paper studies a two-phase protocol in which each of K workers corrects the bias of its local maximum likelihood estimator by an r-th order split-sample extrapolation over nested sub-blocks, the coordinator averages the corrected estimates, and q globally synchronized Newton corrections follow. For fixed-dimensional regular likelihoods with N observations it proves a joint order–round frontier: first-order equivalence to the centralized estimator holds whenever K = o(N^(1−1/(2^(q+1) r))), so that one extra local extrapolation level and one extra synchronization round are exchangeable in the exponent, and the half-block jackknife followed by one Newton step tolerates K = o(N^(7/8)) partitions, against the K = O(N^(1/2)) regime stated for the plain-average one-step estimator.",
3792 + "In the exponential model exact bias, variance and Newton identities are derived; the exponent is shown to be sharp for the construction, with limiting shift −θ0(βr c^r)^(2^q) when K ~ cN^α(r,q), and the Newton basin of attraction is characterized. A perturbation theorem for inexact curvature recovers the known single-machine-Hessian frontiers as special cases and gives the number of bits to which transmitted derivatives may be quantized at each local order.",
3793 + "Monte Carlo experiments with paired replications (exponential, logistic and Poisson likelihoods, N up to 2^22), an exact-model boundary study, ablations and timed multi-node benchmarks on an Apple-silicon cluster quantify the finite-sample variance cost of higher-order corrections and the wall-clock exchange rate between local fits and synchronization rounds. The components combined are known; the joint frontier, its sharpness and the bit-precision result are new. No universal communication lower bound is claimed."
3794 + ],
3795 + "findings": [
3796 + {
3797 + "value": "K = o(N^(7/8))",
3798 + "label": "half-block jackknife plus one Newton step"
3799 + },
3800 + {
3801 + "value": "1 − 1/(2^(q+1) r)",
3802 + "label": "joint order–round exponent, fixed dimension"
3803 + },
3804 + {
3805 + "value": "−4θ0c^4",
3806 + "label": "sharp boundary shift, exponential model, (r,q) = (2,1)"
3807 + },
3808 + {
3809 + "value": "27",
3810 + "label": "Lean 4 / Mathlib theorems, no sorry"
3811 + },
3812 + {
3813 + "value": "N ≤ 2^22",
3814 + "label": "paired Monte Carlo, 2,000 replications per cell"
3815 + },
3816 + {
3817 + "value": "54 ms → 1.83 s",
3818 + "label": "derivative round on 16 hosts, p = 8 to 64"
3819 + }
3820 + ],
3821 + "contributions": [
3822 + {
3823 + "title": "A joint order–round frontier",
3824 + "description": "Theorem 1: first-order equivalence to the centralized MLE whenever K = o(N^(1−1/(2^(q+1) r))); the exponent depends on (r, q) only through the depth D = 2^q r, keeping the centered random remainder of order (m√K)^(−1)."
3825 + },
3826 + {
3827 + "title": "Sharpness in an exactly solvable model",
3828 + "description": "Theorem 2: in the exponential model the scaled gap converges to −θ0(βr c^r)^(2^q) at the boundary K ~ cN^α(r,q) and diverges beyond it, so the sufficient exponent is necessary for this construction; the Newton basin is (0, 2θ̂N)."
3829 + },
3830 + {
3831 + "title": "The price of inexact curvature",
3832 + "description": "Theorem 3: stale averaged local Hessians give exponent 1 − 1/(2(r+q)), a single-machine Hessian 1 − 1/(2r+q), recovering the (t+1)/(t+2) and 2/3 frontiers of Fan et al. (2023) and Jordan et al. (2019); quantization is free once b ≥ (1/2 − r(1−α)) log2 N bits."
3833 + },
3834 + {
3835 + "title": "Exact variance cost of the half-block correction",
3836 + "description": "Var J2 = θ0² m²(m² − 6m + 17)/((m−4)(m−2)²(m−1)²), whose first two terms coincide with the raw MLE; order 3 beats order 2 only at the smallest block sizes in the simulations."
3837 + },
3838 + {
3839 + "title": "Measured exchange rate between rounds and local levels",
3840 + "description": "On a 16-host Apple-silicon LAN the synchronized derivative phase costs 54 ms at p = 8 (network round trip about 0.4 ms) against a local phase that grows from 0.26 to 0.96 s from order 1 to 4, so one more round is cheaper than one more local level."
3841 + }
3842 + ],
3843 + "data": [
3844 + {
3845 + "title": "Paired Monte Carlo designs",
3846 + "description": "Seeded synthetic samples (seed base 20261028): exponential model with N in {2^14, …, 2^22} and m in {8, …, 1024}, 2,000 replications per cell (500 at N = 2^22); logistic and Poisson regressions with p in {3, 8} and a p = 20 stress design (1,000 and 500 replications); robustness designs with misspecification, ill-conditioning, one percent contamination and very small blocks; 40 exponential cells on a laptop and 136 cells spread over 18 MacLustr nodes."
3847 + },
3848 + {
3849 + "title": "Exact-model boundary study and ablations",
3850 + "description": "Exponential-model runs along K ~ cN^α(r,q) that compare the finite-m gap with the predicted limit, plus ablations over split patterns, Newton rounds and derivative quantization."
3851 + },
3852 + {
3853 + "title": "Cluster benchmarks",
3854 + "description": "A real coordinator–worker protocol (Python/NumPy over TCP) on up to 16 Apple-silicon hosts with K ≤ 256 workers and p in {8, 32, 64}: 23 designs, 75 timed runs, each logging per-stage wall clock, per-worker latency and bytes per message."
3855 + }
3856 + ],
3857 + "methodology": [
3858 + {
3859 + "title": "Generalized split-sample jackknife",
3860 + "description": "Each worker fits the MLE on its block and on nested sub-blocks of sizes m/s_j, combining them with Vandermonde weights that cancel the first r − 1 bias terms while preserving the first-order influence function exactly."
3861 + },
3862 + {
3863 + "title": "Aggregated Newton synchronization",
3864 + "description": "The coordinator averages the corrected estimates, broadcasts, and applies q Newton steps from aggregated scores and Hessians; variants use stale, single-machine or b-bit quantized curvature."
3865 + },
3866 + {
3867 + "title": "Uniform stochastic expansions",
3868 + "description": "Proofs rest on a bias expansion of order r and local Newton regularity under fixed p and i.i.d. data, with exact identities in the exponential model and a perturbation argument for inexact Hessians."
3869 + },
3870 + {
3871 + "title": "Machine-checked and symbolic verification",
3872 + "description": "The algebraic content of the exact identities is formalized in 27 Lean 4 / Mathlib theorems without sorry and re-derived in SymPy; probabilistic steps are proved in the appendix."
3873 + },
3874 + {
3875 + "title": "Cost model calibrated on measurements",
3876 + "description": "Proposition 5 gives the wall-clock exchange rate between one local extrapolation level and one synchronization round, calibrated on the benchmark timings for this hardware."
3877 + }
3878 + ],
3879 + "reproducibility": [
3880 + "End-to-end pipeline in experiments/ (Monte Carlo, boundary, ablations, skewed, benchmarks, processing, tables) with an EXPERIMENT_MANIFEST.csv and JSON manifests recording commit, versions, host and SHA-256 per raw file",
3881 + "30 pytest tests, SymPy symbolic checks of every identity and scripts/check_lean.sh for the 27 Lean 4 theorems",
3882 + "All figures and LaTeX tables regenerated from results/ by process_results.py and make_tables.py; raw Monte Carlo outputs committed",
3883 + "Full LaTeX source of the 56-page paper with proofs in the appendices; provenance disclosure and frozen prior note in references/initial_pamle/ with SHA256SUMS"
3884 + ],
3885 + "keywords": [
3886 + "distributed estimation",
3887 + "maximum likelihood",
3888 + "split-sample jackknife",
3889 + "one-step Newton estimator",
3890 + "communication complexity",
3891 + "asymptotic equivalence",
3892 + "Monte Carlo",
3893 + "Lean 4",
3894 + "bias correction"
3895 + ],
3896 + "repo": "https://git.spboucher.ai/wp28_uqo",
3897 + "jel": "C13, C15, C55, C63"
3898 + },
3899 + {
3900 + "slug": "wp29",
3901 + "hero": {
3902 + "headline": "Protocol structure buys auditability, not correctness, for bounded research tasks",
3903 + "subheadline": "Across 288 live Claude Code runs on 24 tasks, verified completion moves from 0.903 to 0.917 between the open instruction and the full method (+0.014, interval [−0.028, +0.056]) while inference cost per run rises two- to three-fold."
3904 + },
3905 + "abstract": [
3906 + "Instruction files that steer coding agents through research tasks range from one sentence to several thousand words of phases, gates and integrity rules, but whether that structure changes what the agent delivers has not been measured with the agent held fixed. The paper reports 288 live Claude Code runs (362 sessions; client 2.1.314, model claude-opus-5-5) on 24 bounded scientific tasks in six families, three fresh sessions per task under each of four regimes: an open instruction; a structured protocol with phases, ledgers, stopping and recovery rules; the protocol plus a controller-enforced executable gate; and the same plus a separate review-and-repair session. Executors ran in bare mode on isolated hosts; deterministic hidden oracles, validated on seeded artefacts, scored every run on a separate host; design, tasks and analysis were frozen with hashes before the confirmatory runs.",
3907 + "Intervention-free verified completion was 0.903 under the open instruction and 0.917 under the full method; the paired task-level difference is +0.014 (task-cluster bootstrap 95% interval [−0.028, +0.056]), the mechanism contrasts are equally null, and critical integrity violations were rare in every arm. The protocol, gate and reviewer bought full compliance with the deliverable contract at two to three times the inference cost per run. Every non-success of the frozen scoring was, on inspection, a defect of the oracles rather than of the executors (an undefined feasibility label, a generator artefact that twelve sessions correctly excluded, a rounding tolerance, a regular-expression false positive); under a corrected rubric every arm completes every task and the null is unchanged. Arm D's review-repair loop, though available, was never triggered.",
3908 + "Interruption-and-resumption, stale-input, duplicate-job and missing-fixture experiments, a length-matched control and three held-out end-to-end mini-projects complete the evidence, together with an instrumented account of how the paper was itself produced under a mandate of the same kind. For bounded tasks with known answers, the marginal value of protocol structure lies in auditability, not correctness, and the scarce resource is a trustworthy oracle."
3909 + ],
3910 + "findings": [
3911 + {
3912 + "value": "288",
3913 + "label": "live Claude Code runs, 362 sessions, 24 tasks"
3914 + },
3915 + {
3916 + "value": "0.903 vs 0.917",
3917 + "label": "verified completion, open instruction vs full method"
3918 + },
3919 + {
3920 + "value": "+0.014",
3921 + "label": "D−A contrast, 95% interval [−0.028, +0.056]"
3922 + },
3923 + {
3924 + "value": "0.24 → 0.64 USD",
3925 + "label": "mean inference cost per run, arm A to D"
3926 + },
3927 + {
3928 + "value": "0",
3929 + "label": "arm C or D deliverables failing the contract check"
3930 + },
3931 + {
3932 + "value": "4",
3933 + "label": "oracle defects behind every frozen non-success"
3934 + }
3935 + ],
3936 + "contributions": [
3937 + {
3938 + "title": "First controlled comparison of instruction regimes",
3939 + "description": "Four arms (open, protocol, protocol plus executable gate, plus review-and-repair) with the agent, tools, inputs and budget held fixed, oracles protected from the executor, and the design preregistered and frozen with hashes before the confirmatory runs."
3940 + },
3941 + {
3942 + "title": "A precise null on correctness",
3943 + "description": "Completion 0.903 (A) against 0.917 (D), paired difference +0.014 with a task-cluster bootstrap interval of [−0.028, +0.056]; the B−A, C−B and D−C contrasts are equally flat and integrity violations are rare in every arm."
3944 + },
3945 + {
3946 + "title": "Structure buys contract compliance at a price",
3947 + "description": "No arm C or D deliverable failed the post-hoc contract check, against 7 in arm A and 9 in arm B; mean inference cost per run rose from 0.24 USD to 0.42, 0.45 and 0.64 USD and median wall time from 68 to 176 s, a flat quality–cost frontier."
3948 + },
3949 + {
3950 + "title": "The failures were the oracles'",
3951 + "description": "Every non-success of the frozen scoring traced to an evaluator defect: an undefined feasibility label, a generator artefact that twelve independent sessions correctly excluded, a rounding tolerance and a regular-expression false positive; under a corrected rubric every arm completes every task."
3952 + },
3953 + {
3954 + "title": "A delegation-contract reference implementation",
3955 + "description": "Controller with durable SQLite state, isolated bare-mode executors, oracle-free runtime gate, separate reviewer, deterministic hidden evaluators and authentic capture, with every claimed invariant enforced by code; 24 confirmatory and 7 development tasks plus three transfer mini-projects."
3956 + },
3957 + {
3958 + "title": "A reflexive, instrumented account",
3959 + "description": "The paper documents its own production from a 6,900-word mandate by a Claude Code builder session, kept separate from the confirmatory evidence, with an intervention ledger recording only the mandate and the launch instruction."
3960 + }
3961 + ],
3962 + "data": [
3963 + {
3964 + "title": "Benchmark of bounded scientific tasks",
3965 + "description": "24 confirmatory tasks in six families (data audit, estimation and inference, forecasting under information constraints, source-grounded synthesis, theory and numerical validation, integrated mini-projects) plus 7 development tasks, with provenance for 14 real-data and 5 source-document tasks and hidden deterministic oracles validated on seeded correct and incorrect artefacts."
3966 + },
3967 + {
3968 + "title": "Run records",
3969 + "description": "288 confirmatory runs (362 sessions) of Claude Code 2.1.314 with claude-opus-5-5 in bare mode on cluster nodes, under a 4.00 USD / 1,500 s envelope; all splits together (smoke, pilot, main, rerun, transfer, recovery, ablation) total 171.76 USD of inference, with every number regenerated from runs/state.sqlite."
3970 + },
3971 + {
3972 + "title": "Corpus of researcher-written mandates",
3973 + "description": "Eleven hashed source mandates coded to motivate the protocol arms, together with a verified literature matrix and BibTeX."
3974 + }
3975 + ],
3976 + "methodology": [
3977 + {
3978 + "title": "Four-arm randomized design",
3979 + "description": "Three fresh sessions per task per arm; primary endpoint intervention-free verified completion within a shared resource envelope; primary contrast D−A with task-cluster bootstrap intervals and permutation tests over the 24 task clusters."
3980 + },
3981 + {
3982 + "title": "Isolation and hidden oracles",
3983 + "description": "Executors run on hosts holding no copy of the reference answers; evaluators are deterministic Python programs on a separate host, never an LLM judging itself, and the executor cannot read them."
3984 + },
3985 + {
3986 + "title": "Controller-enforced executable gate",
3987 + "description": "Arm C adds a runtime gate that checks the deliverable contract and reproducibility and returns failures to the session; arm D adds a separate review-and-repair session of the same model."
3988 + },
3989 + {
3990 + "title": "Fault and recovery experiments",
3991 + "description": "Interruption-and-resumption, stale-input, duplicate-job and missing-fixture faults, a length-matched unstructured instruction as control, and three held-out end-to-end mini-projects."
3992 + },
3993 + {
3994 + "title": "Full resource accounting",
3995 + "description": "Cost and time read from logged inference usage and wall clock, never self-report; cost per verified completion rises from 0.27 to 0.70 USD across arms."
3996 + }
3997 + ],
3998 + "reproducibility": [
3999 + "Every number in the paper regenerated from runs/state.sqlite by analysis/ scripts (build_analysis_dataset, primary_analysis, robustness, figures, tables) via make analyze && make paper",
4000 + "Preregistration, deliverable contract, treatments, evaluation plan and amendments ledger in protocol/; design, tasks and analysis frozen with hashes before the confirmatory runs",
4001 + "Infrastructure test suite (state machine, gate, evaluator, bundle hygiene) runs without provider access; evaluator validation on seeded artefacts in evaluators/run_validations.py",
4002 + "Full LaTeX source of the 110-page paper in the UQO template; the 6,900-word research mandate preserved verbatim with its hash"
4003 + ],
4004 + "keywords": [
4005 + "autonomous research agents",
4006 + "Claude Code",
4007 + "research protocols",
4008 + "executable verification",
4009 + "reproducibility",
4010 + "agent evaluation",
4011 + "delegation",
4012 + "preregistration",
4013 + "hidden oracles"
4014 + ],
4015 + "repo": "https://git.spboucher.ai/wp29_uqo",
4016 + "jel": "C18, C63, C88, O33"
4017 + },
4018 + {
4019 + "slug": "wp3",
4020 + "hero": {
4021 + "headline": "The machine-learning advantage does not survive unseen housing markets",
4022 + "subheadline": "On 788,842 U.S. listings XGBoost scores R² = 0.833 under a random split but 0.425 under a ten-state holdout; cluster-robust standard errors are 8–12 times the HC3 errors and leave only 33 of 63 OLS coefficients significant."
4023 + },
4024 + "abstract": [
4025 + "This paper compares econometric and machine-learning approaches to hedonic housing valuation on 788,842 active Zillow listings covering all 50 U.S. states and the District of Columbia, and asks how much of the machine-learning advantage survives when models must price homes in markets they have not seen. A semi-log OLS model with 62 regressors (R² = 0.634) yields listing-price gradients consistent with the hedonic literature; ZIP3 fixed effects raise out-of-sample R² to 0.725 and residual Moran's I = 0.27 shows strong spatial dependence. Allowing for that dependence matters: cluster-robust standard errors are, on median, 8 (ZIP3) to 12 (state) times the HC3 errors, and only 33 of 63 coefficients remain significant under state clustering.",
4026 + "State-by-state estimation shows that the pooled coefficients average markedly different local price functions (living-area elasticities from 0.06 to 1.15) and that a fully interacted model gains 6.1 points of out-of-sample R² over state fixed effects. Quantile regression reveals distributional heterogeneity, with paired cluster-bootstrap Wald tests rejecting equality of the 10th- and 90th-percentile coefficients for 10 of 13 key attributes. XGBoost reaches R² = 0.833 under random validation but 0.425 under a ten-state holdout and a median of 0.653 across 51 leave-one-state-out models; once 0.6% of corrupt records are removed, OLS is as accurate out of state (0.539) as the best tree ensemble.",
4027 + "The gap is traced to the prediction of market price levels rather than within-market structure (within-state R² = 0.62), to region dummies and to neighbourhood variables that are state-median imputations for 93% of listings, and to boosting capacity: held-out-state accuracy peaks after 75 rounds and declines thereafter while within-training accuracy keeps rising. SHAP rankings are stable across ensembles (ρ > 0.89), and the third-ranked feature is a state identifier. A coding error that set ln(lot) to zero for 16.5% of listings is corrected (lot elasticity 0.095, not 0.018). Estimates are interpreted throughout as conditional associations in asking prices, not willingness-to-pay parameters."
4028 + ],
4029 + "findings": [
4030 + {
4031 + "value": "788,842",
4032 + "label": "active Zillow listings, 50 states and DC"
4033 + },
4034 + {
4035 + "value": "0.833 vs 0.425",
4036 + "label": "XGBoost R², random split versus ten-state holdout"
4037 + },
4038 + {
4039 + "value": "0.539",
4040 + "label": "OLS R² out of state after removing corrupt records"
4041 + },
4042 + {
4043 + "value": "33 of 63",
4044 + "label": "OLS coefficients significant under state clustering"
4045 + },
4046 + {
4047 + "value": "0.06–1.15",
4048 + "label": "living-area elasticity across state-by-state regressions"
4049 + },
4050 + {
4051 + "value": "93%",
4052 + "label": "listings whose school variables are state-median imputations"
4053 + }
4054 + ],
4055 + "contributions": [
4056 + {
4057 + "title": "Spatial leakage quantified on national data",
4058 + "description": "A 10-state holdout, grouped 5-fold cross-validation by state (pooled R² 0.620), leave-one-state-out (median 0.653) and a size-matched random split (0.829) separate the effect of unseen markets from sample size."
4059 + },
4060 + {
4061 + "title": "The collapse is a level error, not a structure error",
4062 + "description": "Within-state R² is a stable 0.62 across specifications; the out-of-market loss comes from memorised state price levels through region dummies, tax rates and imputed neighbourhood variables."
4063 + },
4064 + {
4065 + "title": "Boosting capacity overfits geography",
4066 + "description": "Held-out-state R² peaks at 0.550 after 75 rounds and falls to 0.425 at 1,000 while within-training accuracy keeps rising, so standard early stopping on a random subset selects an over-fitted model."
4067 + },
4068 + {
4069 + "title": "Cluster-robust inference changes the gradients' precision",
4070 + "description": "ZIP3- and state-clustered standard errors are 7.6 and 11.6 times the HC3 errors on median; significant coefficients fall from 57 to 35 and 33 of 63."
4071 + },
4072 + {
4073 + "title": "One national function hides 51 local ones",
4074 + "description": "State-by-state OLS gives living-area elasticities of 0.06–1.15 and sign reversals for bedrooms, pools, luxury and HOA in 19–23 states; the fully interacted model gains 6.1 points of out-of-sample R²."
4075 + },
4076 + {
4077 + "title": "Distributional heterogeneity confirmed under clustering",
4078 + "description": "The garage gradient is ten times larger at the 10th than at the 90th percentile and the pool premium emerges only above the median; 10 of 13 inter-quantile differences survive a paired ZIP3-cluster bootstrap."
4079 + }
4080 + ],
4081 + "data": [
4082 + {
4083 + "title": "Zillow active listings",
4084 + "description": "788,842 listings covering all 50 states and DC, drawn from a 839,313-row DuckDB extract with 116 fields; 62 regressors spanning structure, lot, amenities, neighbourhood and market blocks; 885 ZIP3 areas; list-to-Zestimate ratio median 1.020 for the 55% of listings with a Zestimate."
4085 + }
4086 + ],
4087 + "methodology": [
4088 + {
4089 + "title": "Semi-log OLS with a spatial fixed-effects ladder",
4090 + "description": "62 regressors with region dummies, then state and 885 ZIP3 fixed effects; HC3, ZIP3-clustered and state-clustered standard errors; Halvorsen–Palmquist and Kennedy corrections for dummies; 51 state-by-state regressions and a fully interacted model."
4091 + },
4092 + {
4093 + "title": "Quantile regression with paired cluster bootstrap",
4094 + "description": "Conditional quantiles at τ = 0.10 to 0.90 on a 150,000-listing subsample; inter-quantile Wald tests with 200 paired observation and ZIP3-cluster bootstrap draws."
4095 + },
4096 + {
4097 + "title": "Gradient boosting under two validation designs",
4098 + "description": "XGBoost, LightGBM and random forest evaluated under a random 80/20 split, a ten-state holdout, grouped 5-fold cross-validation by state and leave-one-state-out; R² decomposed into level and within-state components; boosting-round curves."
4099 + },
4100 + {
4101 + "title": "Spatial diagnostics and feature audit",
4102 + "description": "Moran's I on OLS and tree-ensemble residuals (0.263 versus 0.079 for XGBoost), feature-block ablation, SHAP rankings across ensembles, and an audit of missingness and implausible values (ln lot = 0 for 130,271 listings)."
4103 + }
4104 + ],
4105 + "reproducibility": [
4106 + "Eight numbered Python scripts from stored artifacts: spatial robustness, quantile regression and imputation, figures, tables, and four revision scripts (geo validation, inference, QR bootstrap, tables and figures)",
4107 + "Published R² values 0.4250 (ten-state holdout) and 0.8327 (random) reproduced exactly from src/wp3/models.py",
4108 + "14 figures and 19 generated LaTeX table fragments; 18 revision CSVs in results/revision/",
4109 + "Full LaTeX source of the 82-page paper; pre-revision PDF, review report and changelog archived in docs/"
4110 + ],
4111 + "keywords": [
4112 + "hedonic price function",
4113 + "listing prices",
4114 + "quantile regression",
4115 + "gradient boosting",
4116 + "SHAP",
4117 + "spatial cross-validation",
4118 + "cluster-robust inference",
4119 + "automated valuation model",
4120 + "United States"
4121 + ],
4122 + "repo": "https://git.spboucher.ai/wp3_uqo",
4123 + "jel": "R31, C21, C45, C52"
4124 + },
4125 + {
4126 + "slug": "wp30",
4127 + "hero": {
4128 + "headline": "Lossless handoffs are interchangeable; every lossy choice depends on model and regime",
4129 + "subheadline": "Across 13,280 trials on two 3B models, text, zstd text and native or 8-bit caches give identical answers; 4-bit quantization is free for Llama and fatal for Qwen, and attention-based eviction keeps 78–98% of private-fact accuracy with the question but only 12–19% without it."
4130 + },
4131 + "abstract": [
4132 + "Agents built on language models hand information to one another as text, summaries, structured evidence, shared-memory references or, when they share a checkpoint, attention state (the KV cache). This paper asks under which task, model, cache, network and reuse conditions each representation preserves task-relevant information at the lowest end-to-end cost. The study is executed by an autonomous research operator (Claude Code) on Apple-Silicon hardware with the MLX runtime, whose caches are inspectable tensors, so that all byte counts are measured on serialized payloads and all latencies are synchronized wall-clock spans.",
4133 + "Twenty-two conditions (truncation, extractive selection, summaries, structured evidence, metered retrieval, a lossless codec, LLMLingua-2, native and serialized caches, packed 8/4-bit quantization and four eviction rules) are measured on Qwen2.5-3B and Llama-3.2-3B across private-fact ledgers, LongBench QA and executable tasks, in question-blind and question-aware regimes, over 13,280 trials under a frozen protocol with cluster bootstrap and Holm-corrected tests. Further experiments measure two-node cache transfer over a 1 GbE LAN, relays of 1–8 handoffs with an evaluator-only fact ledger, fan-in with conflicting evidence, a same-family learned KV mapper, hosted receivers and a development-fitted router.",
4134 + "Lossless representations are interchangeable in quality and differ only in cost, whose ranking inverts between cold and reused regimes; on the measured link, moving an 8-bit cache beats re-reading the text at every length. Lossy choices are model- and regime-dependent: 4-bit quantization is free for Llama and fatal for Qwen; attention-based eviction keeps 78–98% of private-fact accuracy with the question and 12–19% without it; no eviction survives a code specification. At a 512-token budget prose summaries beat structured evidence (38–44 vs 12–15%); relays are nearly lossless after the first hop and invent almost nothing. A ridge KV mapper transfers no content while producing coherent abstention, and the router halves amortized latency at equal quality when questions are unknown. Cross-family cache transfer was not executed."
4135 + ],
4136 + "findings": [
4137 + {
4138 + "value": "13,280",
4139 + "label": "trials across 22 conditions on two 3B models"
4140 + },
4141 + {
4142 + "value": "78–98%",
4143 + "label": "private-fact accuracy kept by question-aware eviction"
4144 + },
4145 + {
4146 + "value": "12–19%",
4147 + "label": "the same eviction without the question"
4148 + },
4149 + {
4150 + "value": "38–44 vs 12–15%",
4151 + "label": "prose summaries vs structured evidence at 512 tokens"
4152 + },
4153 + {
4154 + "value": "1,676 vs 3,667 ms",
4155 + "label": "8-bit cache transfer vs remote prefill at 8k tokens"
4156 + },
4157 + {
4158 + "value": "−934 ms",
4159 + "label": "router latency saving at equal quality, questions unknown"
4160 + }
4161 + ],
4162 + "contributions": [
4163 + {
4164 + "title": "A common harness for inter-agent handoffs",
4165 + "description": "Twenty-two text, cache and evidence representations are measured under one protocol with validated cache adapters, deterministic scorers, evaluator-isolated answer keys and document-level cluster bootstrap, over 13,280 trials."
4166 + },
4167 + {
4168 + "title": "Lossless representations are interchangeable",
4169 + "description": "Text, its zstd encoding, the native cache, the reloaded cache and the 8-bit packed cache give identical answers; the choice is a cost question whose ranking flips between cold (text wins) and reused (cache wins) regimes."
4170 + },
4171 + {
4172 + "title": "Lossy compression depends on model and regime",
4173 + "description": "4-bit per-token quantization is free for Llama-3.2-3B and fatal for Qwen2.5-3B; attention-based eviction retains 78–98% of private-fact accuracy with the question and 12–19% without it; no eviction survives a code specification."
4174 + },
4175 + {
4176 + "title": "Measured two-node break-even on 1 GbE",
4177 + "description": "On an M3 Ultra receiver, an 8-bit cache transferred over the LAN answers an 8k-token context in 1,676 ms against 3,667 ms for remote text prefill, so moving the cache already wins at about 2,000 tokens on this link."
4178 + },
4179 + {
4180 + "title": "Relays lose at the first hop only",
4181 + "description": "Over 1–8 handoffs with an evaluator-only fact ledger, agents compress by omission rather than invention (about 0.05 unsupported additions per message); at a 512-token budget prose summaries carry more than structured evidence (38–44 vs 12–15%)."
4182 + },
4183 + {
4184 + "title": "A learned KV mapper that transfers nothing",
4185 + "description": "A same-family ridge mapper explaining 68% of the receiver's cache variance carries no content and yields coherent abstention (−75 points vs the receiver's own reading), an argument for compatibility manifests and negative controls in any cache-transfer protocol."
4186 + }
4187 + ],
4188 + "data": [
4189 + {
4190 + "title": "Procedural private-fact ledgers",
4191 + "description": "Synthetic documents with invented names, identifiers, amounts, dates, negations, timestamped revisions and unanswerable questions, generated to test communication dependence without pre-training contamination; answer keys are held by the evaluator only."
4192 + },
4193 + {
4194 + "title": "LongBench QA subsets and executable tasks",
4195 + "description": "Public long-document QA (QASPER, 2WikiMultihopQA) scored with LongBench F1 for external validity, plus executable specification tasks simplified on development data before any test item was run."
4196 + },
4197 + {
4198 + "title": "Per-attempt trial records",
4199 + "description": "13,280 core trials plus relay, fan-in, two-node, mapper, hosted-API and router runs on Qwen2.5-3B and Llama-3.2-3B (bf16 mlx-community checkpoints), each with measured serialized bytes, synchronized latency spans and peak MLX memory, released as JSONL."
4200 + }
4201 + ],
4202 + "methodology": [
4203 + {
4204 + "title": "Taxonomy of communication representations",
4205 + "description": "Text compression, KV compression, cache reuse, cache transfer and parametric transfer are separated, and each method is tagged with its question-access regime, compatibility requirements and composability across sources."
4206 + },
4207 + {
4208 + "title": "Validated cache adapter layer",
4209 + "description": "Identity round trip, prefix equivalence, position preservation, packed-byte accounting and manifest-checked loading are tested on a real 1.5B model before any performance number is recorded."
4210 + },
4211 + {
4212 + "title": "Frozen protocol and confirmatory statistics",
4213 + "description": "Ten hypotheses were registered before any held-out outcome; six confirmatory comparisons are Holm-corrected, with document-level cluster bootstrap intervals; everything else is exploratory."
4214 + },
4215 + {
4216 + "title": "Two-node transfer and break-even model",
4217 + "description": "Mac Studio M3 Ultra sender and receiver on a 940 Mbit/s LAN (1.2 ms RTT) compare cache transfer against remote text prefill and local recompute, with a measured per-token model of receiver prefill versus link time."
4218 + },
4219 + {
4220 + "title": "Relays, fan-in, mapper and router",
4221 + "description": "Chains of 1–8 handoffs with metered repair, fan-in with conflicting evidence, a ridge KV mapper (Qwen2.5-1.5B to 3B) with an identity-transfer negative control, and a router fitted on development data and evaluated on held-out data against the best static policy."
4222 + }
4223 + ],
4224 + "reproducibility": [
4225 + "Harness src/wp30 with CLI (doctor, data prepare/validate, models probe, run, reproduce), pinned lock file and 20 correctness, leakage and scoring tests",
4226 + "Protocol frozen with a hash before the test split existed; registered hypotheses and analysis plan in protocol/",
4227 + "Every per-attempt record in runs/raw; make reproduce-analysis rebuilds all tables and figures with a provenance index and claim ledger",
4228 + "Full LaTeX source of the 49-page paper; minimal reproduction profile re-runs a fixed subset and compares greedy outputs"
4229 + ],
4230 + "keywords": [
4231 + "KV cache compression",
4232 + "cache reuse",
4233 + "multi-agent systems",
4234 + "knowledge transfer",
4235 + "prompt compression",
4236 + "structured evidence",
4237 + "reproducibility",
4238 + "Apple Silicon",
4239 + "MLX"
4240 + ],
4241 + "repo": "https://git.spboucher.ai/wp30_uqo",
4242 + "jel": "C63, C88, D83, L86, O33"
4243 + },
4244 + {
4245 + "slug": "wp31",
4246 + "hero": {
4247 + "headline": "Learned spatial routing in a sparse latent workspace does not pay",
4248 + "subheadline": "Sparse one-or-two-slot edits cut a frozen backbone's held-out loss by 2.8%, yet a global-recurrence adapter trained identically matches it at every budget, the primary effect is +0.9 points against a 3-point threshold, and the learned controller ranks actions near chance."
4249 + },
4250 + "abstract": [
4251 + "PRISME augments a frozen causal decoder with a read-only source bank, an ephemeral sixteen-slot latent workspace re-created at every position, a shared sparse editor that rewrites one or two slots per action, a value controller trained on counterfactual intervention gains measured by executing paired edit/no-edit branches, and a differential readout with an exact zero-budget bypass. The paper reproduces the proposal's exact backbone parameter counts and analytic operation table, proves its structural properties (causality, bitwise identity, exact locality, bounded displacement, one-step selection regret), and tests seven pre-registered hypotheses with a complete implementation, a fifteen-invariant test suite, an incremental inference engine and seeded comparisons against a parameter-matched plain decoder, a dense-slot editor, a global-recurrence adapter trained with the identical recipe, five reference routers and exhaustive one- and two-step oracles.",
4252 + "The zero-initialised readout prescribed by the proposal does not learn because of a scale mismatch; a scale-aware non-zero initialisation does. Sparse edits then lower a frozen backbone's held-out loss by 2.8% and raise compositional exact match by about two points, but the global-recurrence adapter matches or exceeds this at every budget, and where to write does not matter for fixed schedules. An oracle that reads the target finds large opportunity (−15.4% of loss), but 25% of the variance of intervention gains is state-level rather than action-level, the opportunity is concentrated in high-uncertainty states where a wrong action costs more than a right one gains, and the learned controller ranks actions within a state near chance; it is worse than no routing on held-out data.",
4253 + "On the locked test regimes the pre-registered primary effect is +0.9 points against a three-point threshold. At 103M parameters on 98M tokens of web text, neither the sparse editor nor the global adapter moves the frozen backbone by more than 0.1% of loss and a parameter-matched plain decoder does better than both. Measured on Apple Silicon, per-action dispatch overhead erases most of the arithmetic savings of sparsity. The paper reports the complete negative result on learned spatial routing, its mechanistic explanation, and the conditions under which the hypothesis could still be rescued."
4254 + ],
4255 + "findings": [
4256 + {
4257 + "value": "+0.9 pts",
4258 + "label": "pre-registered primary effect vs a 3-point threshold"
4259 + },
4260 + {
4261 + "value": "−2.8%",
4262 + "label": "held-out loss from sparse edits on a frozen backbone"
4263 + },
4264 + {
4265 + "value": "−15.4%",
4266 + "label": "loss reduction found by a target-reading oracle"
4267 + },
4268 + {
4269 + "value": "25%",
4270 + "label": "of intervention-gain variance is state-level, not action-level"
4271 + },
4272 + {
4273 + "value": "≤ 0.1%",
4274 + "label": "loss change of either adapter at 103M on 98M tokens"
4275 + },
4276 + {
4277 + "value": "3×",
4278 + "label": "measured latency of eight sparse edits vs analytic 1.6×"
4279 + }
4280 + ],
4281 + "contributions": [
4282 + {
4283 + "title": "A complete, tested reference implementation",
4284 + "description": "PRISME is implemented with exact parameter and operation accounting for 103M, 459M and 1.18B configurations, a fifteen-invariant test suite, an incremental inference engine and a bitwise-exact zero-budget bypass."
4285 + },
4286 + {
4287 + "title": "Seven pre-registered hypotheses, several refuted",
4288 + "description": "On the locked test regimes the primary effect is +0.9 points against a three-point threshold; the editor-usefulness hypothesis holds only weakly and the spatial-selection, intervention-supervision and adaptivity hypotheses are rejected."
4289 + },
4290 + {
4291 + "title": "Global recurrence matches sparse editing",
4292 + "description": "Over five adapter seeds, sparse edits lower held-out loss by 2.8% and raise exact match by about two points, but a parameter-matched global-recurrence adapter trained with the identical recipe does at least as well at every budget and dominates the quality-versus-cost frontier."
4293 + },
4294 + {
4295 + "title": "Why the controller cannot capture the oracle's gain",
4296 + "description": "A one-step oracle lowers loss by 15.4% on the synthetic suite and 3.6% on web text, yet 25% of gain variance is state-level, opportunity sits in high-uncertainty states where wrong actions cost more than right ones gain, and the controller's within-state ranking is at chance."
4297 + },
4298 + {
4299 + "title": "The prescribed zero-initialised readout does not learn",
4300 + "description": "A scale mismatch prevents the zero-initialised differential readout from training; a scale-aware non-zero bootstrap is required, a correction to the original proposal."
4301 + },
4302 + {
4303 + "title": "Dispatch overhead erases sparse savings at small scale",
4304 + "description": "On an Apple M5 Max, eight sparse edits cost three times the backbone's decode latency at 103M against an analytic 1.6 times, and dense sixteen-slot writes are only 17% slower than sparse ones."
4305 + }
4306 + ],
4307 + "data": [
4308 + {
4309 + "title": "Synthetic compositional suite",
4310 + "description": "A seeded, deterministically regenerated suite of templated compositional tasks with locked dev and test splits, used for the frozen-backbone experiments with five adapter seeds and for the exhaustive one- and two-step oracles."
4311 + },
4312 + {
4313 + "title": "FineWeb-Edu subset (103M pilot)",
4314 + "description": "98.3M training tokens streamed in canonical shard order from FineWeb-Edu sample-10BT (ODC-By 1.0), exactly deduplicated and hashed, with a locked 327,680-token test split; the backbone was pretrained in 2.3 hours on one Apple M3 Ultra (perplexity 45.72)."
4315 + },
4316 + {
4317 + "title": "Counterfactual intervention labels",
4318 + "description": "About 100k executed paired edit/no-edit branches with logged propensities, collected from an editor-unseen stream, used to train the value controller under four feature sets and three loss variants."
4319 + }
4320 + ],
4321 + "methodology": [
4322 + {
4323 + "title": "Architecture with provable invariants",
4324 + "description": "Source bank, ephemeral 16-slot workspace, shared two-block sparse editor (24 actions plus STOP) and differential readout; causality, bitwise identity at budget zero, exact locality, bounded displacement and one-step selection regret are proved and tested."
4325 + },
4326 + {
4327 + "title": "Counterfactual supervision of the controller",
4328 + "description": "Intervention gains g(s,a;y) are measured by executing paired edit/no-edit branches at training time; the controller is trained with calibration-based early stopping and evaluated against five reference routers and exhaustive oracles."
4329 + },
4330 + {
4331 + "title": "Matched-cost controlled baselines",
4332 + "description": "A parameter-matched plain decoder, a dense sixteen-slot editor and a global-recurrence adapter trained with the identical recipe; comparisons use five seeds with t-intervals and per-seed paired item bootstraps."
4333 + },
4334 + {
4335 + "title": "Pre-registration and locked test manifest",
4336 + "description": "Seven hypotheses, a three-point primary threshold and a decision rule were registered before evaluation; deviations are logged, and every manuscript number is checked against run files by a claim-verification script."
4337 + },
4338 + {
4339 + "title": "Latency and operation accounting",
4340 + "description": "Analytic operation tables for 103M, 459M and 1.18B configurations are compared with measured per-token latency on Apple Silicon (PyTorch MPS), isolating per-action dispatch overhead."
4341 + }
4342 + ],
4343 + "reproducibility": [
4344 + "Pipeline in prisme/, training/ (stages A–D), baselines/ and evaluation/, with 23 passing pytest invariants and cache-equivalence tests",
4345 + "Pre-registered protocol, locked test manifest and deviation log in protocol/; every run has an immutable ID, config hash, commit, device and wall-clock record",
4346 + "All tables and figures regenerated by scripts/make_tables.py and scripts/reproduce_figures.py; scripts/verify_manuscript_claims.py checks every number in the manuscript",
4347 + "scripts/reproduce_pilot.sh runs a miniature end-to-end pipeline on CPU in about one minute; full LaTeX source in paper/"
4348 + ],
4349 + "keywords": [
4350 + "small language models",
4351 + "adaptive computation",
4352 + "latent workspace",
4353 + "counterfactual supervision",
4354 + "sparse routing",
4355 + "quality–compute frontier",
4356 + "negative result",
4357 + "reproducibility",
4358 + "pre-registration"
4359 + ],
4360 + "repo": "https://git.spboucher.ai/wp31_uqo",
4361 + "jel": "C45, C63, C88"
4362 + },
4363 + {
4364 + "slug": "wp32",
4365 + "hero": {
4366 + "headline": "Z(P(n,5)) = 12 for all n ≥ 23, and the threshold is sharp",
4367 + "subheadline": "The k = 5 case is closed: values 6, 8, 9, 8, 8, 8, 10, 10, 10, 10, 10, 11 for n = 11–22, then 12 forever, certified by 129 externally checked DRAT refutations and two independent exhaustive enumerators."
4368 + },
4369 + "abstract": [
4370 + "The zero forcing number Z(G) of a graph is the minimum size of a set of initially black vertices from which the colour-change rule (a black vertex with exactly one white neighbour forces that neighbour black) eventually colours the whole graph; it bounds the maximum nullity of the matrices described by G. For the generalized Petersen graphs P(n,k) the sharp values were reported for k = 2, proved for k = 3 and k = 4 (2026), and known for k = 5 only at n = 11 and at n ≡ 0 (mod 24). This paper determines Z(P(n,5)) for every n ≥ 11: the values are 6, 8, 9, 8, 8, 8, 10, 10, 10, 10, 10, 11 for n = 11, …, 22, and Z(P(n,5)) = 12 for all n ≥ 23, so the stabilization threshold is exactly n₀ = 23.",
4371 + "The upper bound 12 is the published 2k + 2 construction. The lower bound for all n ≥ 26 follows the first-force boundary method of Krishnan, Ma and Zhang, adapted to k = 5: the minimum boundary of an 18-vertex set in P(n,5) is at least 12 for every n ≥ 26, by a column-deletion lemma that reduces the statement to the finite window 26 ≤ n ≤ 154, which is settled by 129 unsatisfiability certificates in DRAT format, each verified by the external checker drat-trim and rebuilt from the graph definition by an independent checker. The method provably fails at n = 25 for every parameter p ≤ 24 (two inner pentagons give an obstruction), so n = 23, 24, 25 and all n ≤ 22 are settled by exhaustive enumeration with dihedral orbit reduction, cross-validated by a second, independently written enumeration based on first-force normalization (through n = 40) and, for n ≤ 23, by DRAT-certified fort-cover refutations.",
4372 + "The paper also shows that the k = 4 parameter p = 16 cannot work for k = 5 at any n ≥ 22, that the certified bound has zero slack (β₁₈(n) = 12 exactly for 26 ≤ n ≤ 40), and that a pre-registered holdout of six values of n confirmed the registered conjectures. All proofs are written out; all certificates, code and a one-command reproduction script are released. The work was first produced by the author's AutoScholar.ai autonomous research pipeline and is reissued unchanged in content as a UQO working paper."
4373 + ],
4374 + "findings": [
4375 + {
4376 + "value": "n₀ = 23",
4377 + "label": "sharp threshold: Z = 12 for all n ≥ 23"
4378 + },
4379 + {
4380 + "value": "129",
4381 + "label": "DRAT certificates for 26 ≤ n ≤ 154, drat-trim verified"
4382 + },
4383 + {
4384 + "value": "8",
4385 + "label": "new exact values of n between 12 and 23"
4386 + },
4387 + {
4388 + "value": "432 billion",
4389 + "label": "subsets enumerated exhaustively in 4,126 s on 16 threads"
4390 + },
4391 + {
4392 + "value": "39.0 GB",
4393 + "label": "of DRAT proofs in the original run, digests recorded"
4394 + },
4395 + {
4396 + "value": "6 / 6",
4397 + "label": "pre-registered holdout values confirmed as 12"
4398 + }
4399 + ],
4400 + "contributions": [
4401 + {
4402 + "title": "The k = 5 case closed",
4403 + "description": "Z(P(n,5)) is determined for every n ≥ 11; the values at n ∈ {12, 13, 15, 17, 18, 20, 22, 23}, the statement Z = 12 for all n ≥ 23 and the sharpness of n₀ = 23 are new."
4404 + },
4405 + {
4406 + "title": "Boundary method adapted to k = 5",
4407 + "description": "The parameter p = 18 is shown to be the smallest that works; p = 16 (used for k = 4) fails for every n ≥ 22, and no p ≤ 24 works at n = 25, where two inner pentagons give an 18-set of boundary 10."
4408 + },
4409 + {
4410 + "title": "A finite window closed by certificates",
4411 + "description": "A column-deletion lemma reduces the lower bound for all n ≥ 26 to 26 ≤ n ≤ 154; each of the 129 instances is refuted by CaDiCaL with a DRAT proof checked by drat-trim and re-verified by an independent checker that rebuilds the CNF from the graph definition."
4412 + },
4413 + {
4414 + "title": "Certified table with explicit status",
4415 + "description": "Every value for 11 ≤ n ≤ 48 has an upper-bound witness replayed by an independent program and a lower bound supported by an externally checkable or exhaustive argument; the three-valued status (fully certified / solver-supported / candidate) is never collapsed."
4416 + },
4417 + {
4418 + "title": "A falsification campaign found no error",
4419 + "description": "Pre-registered holdout H = {32, 35, 37, 41, 44, 48}, a second exhaustive enumerator (344 billion sets) by a different reduction principle, exhaustive validation of the SAT encodings on small instances, negative controls for the proof checker and heuristic attacks on the theorem."
4420 + }
4421 + ],
4422 + "data": [
4423 + {
4424 + "title": "Graph instances generated from the definition",
4425 + "description": "No external data: every P(n,5) is built by two independent constructors with canonical SHA-256-digested edge lists; the exhaustive oracle enumerated 432,226,268,255 subsets (4,125.7 s wall-clock, 16 threads) for n ≤ 27 and the second enumerator 344 billion sets through n = 40."
4426 + },
4427 + {
4428 + "title": "Certificate corpus",
4429 + "description": "129 DRAT unsatisfiability certificates for the boundary window 26 ≤ n ≤ 154 plus one at n = 24 (solve times 17–267 s, 346,502,796 proof lines, 39.0 GB; largest CNF 14,323 variables and 28,933 clauses), 14 fort-cover certificates for n ≤ 23, and DIMACS files, fort lists and digests shipped with the archive."
4430 + },
4431 + {
4432 + "title": "Solver and verification records",
4433 + "description": "Every solver run is a JSON record with UTC time, graph digest, method, bound, solver version, wall time, timeout flag and certificate SHA-256 (results/*.jsonl); independent-checker verdicts are stored for all 135 boundary and 14 fort-cover certificates."
4434 + }
4435 + ],
4436 + "methodology": [
4437 + {
4438 + "title": "Foundational propositions proved in full",
4439 + "description": "Order independence of the closure, the fort characterization, dihedral orbit reduction, graph facts and the isomorphism P(n,k) ≅ P(n,l) for kl ≡ 1 (mod n), each with a step-numbered written proof."
4440 + },
4441 + {
4442 + "title": "Upper bound by explicit witnesses",
4443 + "description": "Twelve consecutive outer vertices force P(n,5) for n ≥ 12 (the 2k + 2 construction); every witness is replayed by an independent program."
4444 + },
4445 + {
4446 + "title": "First-force boundary lower bound and SAT encodings",
4447 + "description": "The boundary lemma with p = 18 and a column-deletion lemma reduce the lower bound to a finite window; two SAT encodings with correctness lemmas are refuted by CaDiCaL with DRAT proofs."
4448 + },
4449 + {
4450 + "title": "Exhaustive enumeration with symmetry reduction",
4451 + "description": "A C enumerator with D_n orbit reduction settles n ≤ 27 (unavoidable at n = 25), cross-validated by a second enumerator using first-force normalization and by fort-cover DRAT refutations."
4452 + },
4453 + {
4454 + "title": "Pre-registration and falsification",
4455 + "description": "Conjectures and a six-value holdout were frozen before fitting; negative controls, stress tests of the deletion lemma and simulated-annealing attacks were run against the theorem."
4456 + }
4457 + ],
4458 + "reproducibility": [
4459 + "One-command script code/reproduce.sh with QUICK (10 min), default (25 min on 16 cores) and FULL (several hours, complete window 26–154) modes; shipped artifacts are never modified",
4460 + "Certificates re-verified with drat-trim and an independent checker that rebuilds each CNF from the graph definition; proofs absent from disk are reported as skipped, never as verified",
4461 + "Pre-registered plan, frozen holdout, conjecture registry and robustness report in plan/ and results/; provenance checksums for every artifact",
4462 + "Full LaTeX source of the 36-page paper plus the original AutoScholar manuscript (AS-WP-2026-0001); no Lean formalization is claimed"
4463 + ],
4464 + "keywords": [
4465 + "zero forcing",
4466 + "generalized Petersen graphs",
4467 + "minimum rank",
4468 + "forts",
4469 + "SAT certificates",
4470 + "DRAT",
4471 + "computer-assisted proof",
4472 + "exhaustive enumeration",
4473 + "pre-registration"
4474 + ],
4475 + "repo": "https://git.spboucher.ai/wp32_uqo",
4476 + "jel": "C61, C63 (MSC 2020: 05C50, 05C69, 05C85, 68V15)"
4477 + },
4478 + {
4479 + "slug": "wp33",
4480 + "hero": {
4481 + "headline": "Bounded expert residency is free; prefetch, fused kernels and gigabit remote tiers are not",
4482 + "subheadline": "TITAN-MoE matches mlx-lm bit for bit (489 vs 494 tok/s), yet the best causal predictor only ties demand loading, kernel fusion moves nothing at batch one, and remote experts over 1 GbE cost 2.5 ms per layer; only the expert-contiguous storage layout wins (8 to 36 tok/s)."
4483 + },
4484 + "abstract": [
4485 + "Sparse mixture-of-experts (MoE) language models touch only a few experts per token, so a model whose weights exceed a machine's memory can still be served if the right experts are in the right place at the right time. This preliminary working paper reports the design, implementation and measurement of TITAN-MoE, an MLX-native inference runtime for Apple Silicon that manages expert residency with bounded per-layer slot tables, loads experts on demand from local NVMe, prefetches them with causal predictors and, in its distributed mode, executes them on other Macs of a heterogeneous cluster connected by gigabit Ethernet. Every design choice is validated against mlx-lm on identical weights: with all experts resident the runtime is bit-identical in logits and within one percent of mlx-lm's decode throughput, and with bounded residency the router's choices and the logits are unchanged by construction.",
4486 + "Five hypotheses (causal prefetch, topology-aware placement, hierarchical caching, fused Metal kernels, and adaptive orchestration on the latency–throughput–energy frontier) are tested with a pre-registered protocol on a live inventory of a 21-node Mac cluster. The results of version 0.1 are mostly negative and fully attributed: on the small model, the best causal predictor (a per-layer logistic probe, recall@8 of 0.76, within 0.16 of the oracle hit rate) only ties with demand loading because installing a prefetched expert costs as much graph time as the read it saves; kernel fusion is immaterial because decode at batch size one is launch-bound at 28% of memory bandwidth; and executing half the experts on a second Mac Studio over gigabit Ethernet costs 2.5 ms per layer, more than a cold-NVMe miss, so the remote tier is a capacity mechanism rather than a latency one.",
4487 + "The storage layout, by contrast, matters: an expert-contiguous record file raises cold-NVMe decoding from 8 to 36 tokens per second. Energy measurements and the adaptive scheduler are pending and labelled as such; every reported number has a raw log and the next gates (deeper prefetch horizons, the medium model on nodes that cannot hold it, cost-aware placement) are fixed by the protocol."
4488 + ],
4489 + "findings": [
4490 + {
4491 + "value": "489 vs 494 tok/s",
4492 + "label": "TITAN all-resident vs mlx-lm, bit-identical logits"
4493 + },
4494 + {
4495 + "value": "0.76",
4496 + "label": "recall@8 of the best causal predictor, 0.16 below oracle"
4497 + },
4498 + {
4499 + "value": "8 → 36 tok/s",
4500 + "label": "cold-NVMe decode with expert-contiguous storage layout"
4501 + },
4502 + {
4503 + "value": "2.5 ms",
4504 + "label": "per-layer cost of remote experts over 1 GbE"
4505 + },
4506 + {
4507 + "value": "28%",
4508 + "label": "of memory bandwidth used by launch-bound batch-one decode"
4509 + },
4510 + {
4511 + "value": "21",
4512 + "label": "Macs in the live cluster inventory, all on gigabit Ethernet"
4513 + }
4514 + ],
4515 + "contributions": [
4516 + {
4517 + "title": "An exact-semantics MoE runtime for MLX",
4518 + "description": "Expert registry over safetensors byte ranges, pread NVMe tier, per-layer slot tables with LRU/LFU eviction, an exact execution block over gather_qmm and an asynchronous prefetcher with four causal predictors; placement decides speed, never which experts run."
4519 + },
4520 + {
4521 + "title": "Correctness and zero-overhead baseline",
4522 + "description": "Bit-identical logits (max |Δ| = 0.0) and token-identical greedy decoding versus mlx-lm on OLMoE-1B-7B-0125 4-bit at 64/32/16/8 slots; all-resident decode at 489 tok/s against 494 for mlx-lm, and 121.6 vs 121.7 tok/s on Qwen3-30B-A3B."
4523 + },
4524 + {
4525 + "title": "Causal prefetch does not pay on the small model",
4526 + "description": "One-layer-ahead prefetch raises the hit rate from 0.48 to 0.63 but lowers throughput; the paper derives the lead-time condition d·τ_layer ≥ λ_tier and shows the best predictor (recall@8 0.76) only ties demand loading (35.0 vs 36.4 tok/s)."
4527 + },
4528 + {
4529 + "title": "Kernel fusion is immaterial at batch one",
4530 + "description": "Decode is launch-bound at about 28% of nominal memory bandwidth, so fusing the up and gate projections changes throughput by ±2%."
4531 + },
4532 + {
4533 + "title": "The gigabit remote tier adds capacity, not speed",
4534 + "description": "Executing odd-numbered experts on a second Mac Studio over 1 GbE keeps logits bit-identical but drops OLMoE decoding from 70–87 to 15 tok/s (2.5 ms per layer, 20× a cold-NVMe miss); a LAN fetch of one expert costs about 28 ms, ruling out moving weights during decode."
4535 + },
4536 + {
4537 + "title": "Storage layout is the one clear win",
4538 + "description": "An expert-contiguous record file raises cold-NVMe decoding at 16 slots from 8 to 36 tok/s, more than any prediction or caching policy tested."
4539 + }
4540 + ],
4541 + "data": [
4542 + {
4543 + "title": "Models and workloads",
4544 + "description": "OLMoE-1B-7B-0125-Instruct and Qwen3-30B-A3B-Instruct-2507 as pinned mlx-community 4-bit conversions; synthetic prompt workloads written for the study and the routing traces derived from them (no external datasets)."
4545 + },
4546 + {
4547 + "title": "Cluster inventory and benchmark logs",
4548 + "description": "A sanitized live inventory of 21 reachable Macs (M1–M6, 8–96 GB; iperf3 0.945 Gbit/s, 0.5–1.0 ms RTT, no Thunderbolt link) and immutable raw decode-throughput logs under benchmarks/raw with SHA-256 manifests, 12–18 runs per configuration."
4549 + }
4550 + ],
4551 + "methodology": [
4552 + {
4553 + "title": "Pre-registered protocol with gates A–G",
4554 + "description": "Five hypotheses stated before measurement, a frozen protocol at Gate C, and a research log; nothing is reported without a raw log, and pending gates (energy, adaptive scheduler) are labelled as such."
4555 + },
4556 + {
4557 + "title": "Exact comparison against mlx-lm",
4558 + "description": "Same weights, quantization and kernels; numerical tests check logits and greedy tokens, and llama.cpp and colibri are reported as references rather than like-for-like competitors."
4559 + },
4560 + {
4561 + "title": "Trace-driven simulator and analytic cost model",
4562 + "description": "A simulator calibrated on measured constants (τ_layer = 0.128 ms, NVMe 8.5 GB/s random reads, 0.3 ms expert install) and a formal placement–prefetch problem with a deterministic cost-aware heuristic."
4563 + },
4564 + {
4565 + "title": "Statistical audit with bootstrap intervals",
4566 + "description": "Medians and 95% bootstrap confidence intervals (10,000 resamples) for every throughput comparison and its paired difference."
4567 + },
4568 + {
4569 + "title": "Two-node TCP transport",
4570 + "description": "An authenticated transport with a bit-exact remote expert worker measured between two Mac Studios (M3U96a and M3U96b) on the cluster's gigabit Ethernet."
4571 + }
4572 + ],
4573 + "reproducibility": [
4574 + "Runtime in titan/ with pytest correctness tests against mlx-lm that write evidence to benchmarks/raw/tests/",
4575 + "Benchmark scripts (bench_decode, collect_traces, simulate_cache, ssd_bandwidth) with immutable raw logs and artifacts/manifest.json listing every artifact's SHA-256",
4576 + "Figures and tables regenerated from benchmarks/raw by scripts/make_figures.py and scripts/make_tables.py; pinned requirements-lock.txt and pinned model revisions",
4577 + "Full LaTeX source of the 30-page paper; prior-art matrix of 59 verified works in research/prior_art.csv; AI-assistance disclosure in the reproducibility appendix"
4578 + ],
4579 + "keywords": [
4580 + "mixture of experts",
4581 + "inference systems",
4582 + "Apple Silicon",
4583 + "expert offloading",
4584 + "predictive prefetching",
4585 + "distributed inference",
4586 + "energy efficiency",
4587 + "MLX",
4588 + "negative results"
4589 + ],
4590 + "repo": "",
4591 + "jel": "C63, C88, L86, O33"
4592 + },
4593 + {
4594 + "slug": "wp4",
4595 + "hero": {
4596 + "headline": "A quarter of scraped Airbnb prices were not nightly rates at all",
4597 + "subheadline": "A two-regime mixture of hedonic regressions isolates 976 stay-price listings; on the 2,480 nightly prices that remain, capacity, bathrooms, photos and rating are priced, Superhost and Guest Favourite badges are not, and gradient boosting lifts out-of-sample R² from 0.47 to 0.56."
4598 + },
4599 + "abstract": [
4600 + "This paper estimates hedonic price functions for a province-wide cross-section of Airbnb listings in Quebec, Canada, from a scrape of 5,000 listing cards collected in early 2026 (3,456 with a displayed price). The displayed price field is bimodal: about 28% of priced listings, mostly small Montreal rental units, show the total price of a stay of roughly three weeks rather than a nightly rate. A two-regime mixture of hedonic regressions estimated by EM separates the two populations sharply and leaves 2,480 nightly prices, on which all estimation is based.",
4601 + "A semi-logarithmic hedonic model with city fixed effects, estimated with heteroskedasticity-robust and clustered standard errors, explains 51% of the variance of log prices. Guest capacity (elasticity 0.33), bathrooms (0.44) and bedrooms (0.16), a shared bathroom (−32%), the number of photos, the displayed rating (3.4% per tenth of a point) and local listing density are the priced attributes; the Superhost and Guest Favourite badges carry no premium conditional on capacity, type and location. Quantile regressions show that cheap listings are priced per guest and certified by reviews, whereas expensive listings are priced per room and are cheaper when heavily reviewed.",
4602 + "Spatial lag, error and Durbin models find moderate spatial dependence (ρ̂ ≈ 0.20, λ̂ ≈ 0.28) with direct impacts within a few percent of the OLS coefficients. Gradient-boosting models raise the out-of-sample R² from 0.47 to 0.56 under random cross-validation and from 0.41 to 0.48 when whole areas are held out, and their SHAP decomposition ranks the same attributes as the hedonic model. The paper provides the first provincial hedonic evidence for Quebec's short-term rental market and a template for detecting and handling stay-price contamination in scraped platform prices."
4603 + ],
4604 + "findings": [
4605 + {
4606 + "value": "5,000",
4607 + "label": "Airbnb listing cards scraped, 3,456 with a price"
4608 + },
4609 + {
4610 + "value": "28%",
4611 + "label": "priced listings showing a stay total, not nightly"
4612 + },
4613 + {
4614 + "value": "2,480",
4615 + "label": "nightly-price listings used in every estimation"
4616 + },
4617 + {
4618 + "value": "51%",
4619 + "label": "log-price variance explained with city fixed effects"
4620 + },
4621 + {
4622 + "value": "−32%",
4623 + "label": "price discount for a shared bathroom"
4624 + },
4625 + {
4626 + "value": "0.56 vs 0.47",
4627 + "label": "out-of-sample R², XGBoost vs OLS, random folds"
4628 + }
4629 + ],
4630 + "contributions": [
4631 + {
4632 + "title": "Diagnosing stay-price contamination",
4633 + "description": "Log prices are bimodal and a two-regime mixture of hedonic regressions (EM, intercept shift δ̂ = 3.15 log points, about 23 nights) classifies 976 of 3,456 priced listings as stay totals; a ratio of medians by bedroom count is enough to detect the defect."
4634 + },
4635 + {
4636 + "title": "Implicit prices of a nightly rate",
4637 + "description": "On the 2,480 nightly prices, elasticities are 0.33 for guests, 0.44 for ln(1 + bathrooms) and 0.16 for bedrooms; a shared bathroom costs 32%, the rating adds 3.4% per tenth of a point, and the Superhost and Guest Favourite badges are insignificant."
4638 + },
4639 + {
4640 + "title": "Pricing rules differ along the distribution",
4641 + "description": "The bedroom elasticity rises from 0.01 at τ = 0.10 to 0.19 at τ = 0.90 and the review-count coefficient falls from +0.042 to −0.042: cheap listings are priced per guest and certified by reviews, expensive ones per room."
4642 + },
4643 + {
4644 + "title": "Spatial dependence does not bias the coefficients",
4645 + "description": "SAR, SEM and SDM estimates give ρ̂ = 0.20–0.26 and λ̂ = 0.28 with a spatial multiplier of about 1.25, but LeSage–Pace direct impacts stay within a few percent of OLS."
4646 + },
4647 + {
4648 + "title": "Machine learning refines the hedonic surface",
4649 + "description": "XGBoost reaches R² 0.559 against 0.473 for OLS under random 5-fold cross-validation and 0.483 against 0.407 under spatially blocked folds; SHAP ranks capacity first and badges last, as the hedonic model does."
4650 + },
4651 + {
4652 + "title": "Version 1.0 results retracted in full",
4653 + "description": "Every estimate of the first version was computed on the contaminated mixture and on constructed regressors; none survives, and the corrections are documented so that the lesson transfers to other scraped platform prices."
4654 + }
4655 + ],
4656 + "data": [
4657 + {
4658 + "title": "Quebec Airbnb search-card scrape",
4659 + "description": "5,000 Airbnb search-result cards for Quebec collected in early 2026 with 19 fields (title, city, property-type label, displayed price, rating, bedrooms, bathrooms, guests, amenity count, photos, reviews, Superhost, Guest Favourite, pets, coordinates); 1,544 cards carry no price. The committed parquet holds the 3,456 priced cards; the script parses titles to recover ratings and repair 465 bathroom and 63 bedroom counts, then classifies cards into the nightly or stay-price regime (2,480 nightly)."
4660 + }
4661 + ],
4662 + "methodology": [
4663 + {
4664 + "title": "Price-regime mixture",
4665 + "description": "A marginal normal mixture of log prices followed by a two-regime mixture of hedonic regressions estimated by EM with common slopes, a regime-specific intercept and variance and a logistic regime equation; posterior below 0.5 defines the analysis sample."
4666 + },
4667 + {
4668 + "title": "Nested semi-log hedonic OLS",
4669 + "description": "Five nested specifications (M1–M5) adding structural, presentation, host-reputation and location blocks plus city or spatial-cell fixed effects, with HC3 and group-clustered standard errors, Kennedy premia, VIF and block F-tests."
4670 + },
4671 + {
4672 + "title": "Quantile regression",
4673 + "description": "The M5 specification re-estimated at τ ∈ {0.10, 0.25, 0.50, 0.75, 0.90} with fixed effects; paired bootstrap (B = 100) tests of the τ = 0.90 − τ = 0.10 differences."
4674 + },
4675 + {
4676 + "title": "Spatial econometrics",
4677 + "description": "k-nearest-neighbour weights (k = 5, 10) on projected coordinates, Moran's I, SAR, SEM and SDM by maximum likelihood, likelihood-ratio tests and LeSage–Pace impacts with simulated standard errors."
4678 + },
4679 + {
4680 + "title": "Machine-learning benchmarks",
4681 + "description": "OLS, elastic net, random forest, gradient boosting, XGBoost and LightGBM compared under 5-fold random and spatially blocked (20 blocks) cross-validation on R², RMSE, MAE and MdAPE, with exact TreeSHAP for XGBoost and a 12-variant robustness table."
4682 + }
4683 + ],
4684 + "reproducibility": [
4685 + "One script, scripts/10_revision_reanalysis.py, produces every table, figure and quoted number of v1.1 in about 10 minutes on an Apple-silicon laptop; no table is typed by hand",
4686 + "Committed input data/processed/airbnb_clean_wp5.parquet (3,456 priced cards) and the generated analysis sample of 2,480 listings",
4687 + "Key numbers in results/tables/rev_key_numbers.json; six regenerated figures; the v1.0 pipeline is kept unchanged for the record",
4688 + "Full LaTeX source with 50 Crossref-verified references; pre-revision PDF, review report and changelog in docs/"
4689 + ],
4690 + "keywords": [
4691 + "hedonic price function",
4692 + "short-term rentals",
4693 + "Airbnb",
4694 + "implicit prices",
4695 + "quantile regression",
4696 + "spatial econometrics",
4697 + "machine learning",
4698 + "mixture of regressions",
4699 + "Quebec"
4700 + ],
4701 + "repo": "https://git.spboucher.ai/wp4_uqo",
4702 + "jel": "C21, C31, C45, L83, R31, Z30"
4703 + },
4704 + {
4705 + "slug": "wp5",
4706 + "hero": {
4707 + "headline": "The Airbnb–rent correlation in Quebec is mostly between neighbourhoods",
4708 + "subheadline": "With city fixed effects one more Airbnb within 500 m means rents 0.4% higher; with forward-sortation-area fixed effects the coefficient falls to 0.08% and is indistinguishable from zero under Conley or neighbourhood-clustered standard errors."
4709 + },
4710 + "abstract": [
4711 + "This paper examines the relationship between short-term rental (STR) activity and residential rents in Quebec, Canada, using a 2026 cross-section of 3,456 Airbnb listings and 8,303 residential rental listings with precise coordinates. For each rental listing the number of Airbnb listings within 250 m, 500 m, 1 km and 2 km (Haversine distances) is counted and hedonic rent equations are estimated with building-type and location fixed effects. With city fixed effects, one additional Airbnb listing within 500 m is associated with monthly rents about 0.4% higher (0.3–0.5% across specifications); the per-listing coefficient decays with distance, the association is larger at the upper quantiles of the conditional rent distribution, and it survives spatial lag and spatial error models (total impact 0.40%).",
4712 + "Two further results qualify this picture. First, heteroskedasticity-robust standard errors overstate precision: Conley spatial HAC and neighbourhood-clustered standard errors are about three times larger, although the city-fixed-effects estimate remains significant. Second, with fixed effects for forward sortation areas (postal-code neighbourhoods), the coefficient falls to 0.08% per listing and is not distinguishable from zero under spatial inference, with the same pattern for logarithmic and extensive-margin exposure measures. The Airbnb–rent association in these data is therefore predominantly a between-neighbourhood phenomenon, consistent with STR supply concentrating in high-amenity, high-rent neighbourhoods rather than with a within-neighbourhood causal effect.",
4713 + "Spatially blocked cross-validation shows that the apparent advantage of tree-based machine-learning models over the linear hedonic model disappears when test neighbourhoods are unseen, and a hedonic model of Airbnb nightly prices finds no premium attached to city-level rents. The paper discusses the implications for the interpretation of cross-sectional STR–rent correlations and for Quebec's registration-based regulation."
4714 + ],
4715 + "findings": [
4716 + {
4717 + "value": "8,303",
4718 + "label": "Quebec rental listings matched to 3,456 Airbnb listings"
4719 + },
4720 + {
4721 + "value": "+0.4%",
4722 + "label": "rent per additional Airbnb within 500 m, city fixed effects"
4723 + },
4724 + {
4725 + "value": "0.08%",
4726 + "label": "same coefficient with neighbourhood (FSA) fixed effects, insignificant"
4727 + },
4728 + {
4729 + "value": "≈ 3×",
4730 + "label": "Conley and FSA-clustered errors relative to HC1 errors"
4731 + },
4732 + {
4733 + "value": "0.40%",
4734 + "label": "total impact per listing in the spatial lag model"
4735 + },
4736 + {
4737 + "value": "0.65 → 0.43",
4738 + "label": "random-forest R² from random to FSA-blocked folds, below OLS at 0.47"
4739 + }
4740 + ],
4741 + "contributions": [
4742 + {
4743 + "title": "First listing-level STR–rent analysis for a Canadian market",
4744 + "description": "Exact distance-based matching of 8,303 rental listings to 3,456 Airbnb listings yields exposure counts at four radii (mean 7.92 within 500 m, 44% zeros) and hedonic rent equations with building-type, city and neighbourhood fixed effects."
4745 + },
4746 + {
4747 + "title": "The identification problem is made measurable",
4748 + "description": "Moving from 187 city to 289 forward-sortation-area fixed effects cuts the per-listing coefficient from 0.0039 to 0.0008; log and extensive-margin exposures behave the same way, and within neighbourhoods any Airbnb presence is associated with 2.8% lower rents."
4749 + },
4750 + {
4751 + "title": "Spatial inference triples the standard errors",
4752 + "description": "Conley spatial HAC (1–10 km) and FSA-clustered standard errors are 0.0007 against an HC1 error of 0.0002, taking the city-FE t-statistic from 17 to about 5.5 and the FSA-FE one to 1.1–2.0."
4753 + },
4754 + {
4755 + "title": "Spatial lag and error models confirm the city-FE estimate",
4756 + "description": "Kelejian–Prucha GMM gives ρ = 0.134 and λ = 0.544; LeSage–Pace impacts are 0.0034 direct, 0.0005 indirect and 0.0040 total, equal to OLS."
4757 + },
4758 + {
4759 + "title": "Machine learning's edge is neighbourhood interpolation",
4760 + "description": "Random-forest and gradient-boosting R² fall from 0.65 under random five-fold cross-validation to 0.43 and 0.41 under FSA-blocked folds, below the linear hedonic model at 0.47."
4761 + },
4762 + {
4763 + "title": "Airbnb prices do not respond to city rents",
4764 + "description": "The scraped price field is bimodal (26.5% of listings above $1,000); on the plausibly priced subsample the entire-home and size premia reappear, the superhost discount reported in the earlier version disappears, and the city-rent coefficient is 0.003 (s.e. 0.052)."
4765 + }
4766 + ],
4767 + "data": [
4768 + {
4769 + "title": "Airbnb listings, Quebec",
4770 + "description": "5,000 listings scraped in early 2026, of which 3,456 with a displayed price; coordinates, property type, rating, reviews, superhost status, capacity and amenities; price field bimodal and partly total-stay prices."
4771 + },
4772 + {
4773 + "title": "Residential rental listings",
4774 + "description": "8,303 Realtor.ca rental listings scraped on 8 January 2026 (from 8,356 raw) with monthly asking rent, coordinates, bedrooms, bathrooms, building type, interior size and postal code mapped to 333 forward sortation areas across 253 municipalities."
4775 + },
4776 + {
4777 + "title": "Spatial exposure measures",
4778 + "description": "For every rental listing and radius of 250 m, 500 m, 1 km and 2 km, six Airbnb metrics (count, density, mean nightly price, entire-home share, and more) computed by chunked vectorised Haversine distances, plus city-level aggregates."
4779 + }
4780 + ],
4781 + "methodology": [
4782 + {
4783 + "title": "Hedonic rent equations with nested fixed effects",
4784 + "description": "Log monthly rent on Airbnb exposure, structural controls and building type with city fixed effects (Model 1c) and forward-sortation-area fixed effects (Model F2), including log and any-Airbnb exposure variants."
4785 + },
4786 + {
4787 + "title": "Spatially robust inference",
4788 + "description": "HC1, city-clustered, FSA-clustered and Conley spatial HAC standard errors with 1–10 km cutoffs, compared side by side for both fixed-effects designs."
4789 + },
4790 + {
4791 + "title": "Spatial lag and spatial error models",
4792 + "description": "Kelejian–Prucha GMM on a k-nearest-neighbour weight matrix built on projected kilometre coordinates, with White-robust errors and LeSage–Pace direct, indirect and total impacts."
4793 + },
4794 + {
4795 + "title": "Quantile regressions and buffer decomposition",
4796 + "description": "Conditional quantiles of rent, per-listing coefficients by radius and by ring, subsample, trimming and leave-one-city-out checks (0.0038–0.0040)."
4797 + },
4798 + {
4799 + "title": "Spatially blocked machine-learning benchmark",
4800 + "description": "Random forest, gradient boosting and OLS compared under random and FSA-blocked five-fold cross-validation on 4,370 listings with 181 neighbourhood groups."
4801 + }
4802 + ],
4803 + "reproducibility": [
4804 + "Twelve numbered Python scripts; steps 04–12 and the paper rebuild in under one minute from the five committed parquet datasets (seed 42)",
4805 + "New script 12 (FSA fixed effects and Conley/cluster inference) and a run log of the 2026-10-17 pipeline in results/logs/",
4806 + "18 LaTeX table fragments and 17 figures regenerated from results; readable variable labels generated from src/latex_tables.py",
4807 + "Full LaTeX source of the 59-page paper; pre-revision PDF, review report and changelog archived in docs/"
4808 + ],
4809 + "keywords": [
4810 + "Airbnb",
4811 + "short-term rentals",
4812 + "housing rents",
4813 + "hedonic price function",
4814 + "neighbourhood fixed effects",
4815 + "spatial econometrics",
4816 + "Conley standard errors",
4817 + "Quebec"
4818 + ],
4819 + "repo": "https://git.spboucher.ai/wp5_uqo",
4820 + "jel": "R21, R31, R52, L83, C21"
4821 + },
4822 + {
4823 + "slug": "wp6",
4824 + "hero": {
4825 + "headline": "Volatility flows from U.S. equity futures to everything else, including crypto",
4826 + "subheadline": "Cross-asset shocks account for 54.8% of forecast-error variance; E-mini S&P 500 and Nasdaq 100 futures are the only robust net transmitters (+36.0 and +25.6 points), and bitcoin's exposure to traditional assets rose from under 10% before 2020 to 20–46% since."
4827 + },
4828 + "abstract": [
4829 + "This paper measures how realized volatility is transmitted across equity index futures, major exchange rates, gold and crude oil futures, the VIX, and the two largest cryptocurrencies. Daily realized volatilities are built from 5-minute bars for ten instruments over 2008–2026, and the connectedness of their logarithms is estimated on the 2,514 trading days common to all series (March 2016–March 2026) with the generalized variance decomposition of a VAR(4) at a ten-day horizon, with residual-bootstrap confidence intervals.",
4830 + "Cross-asset shocks account for 54.8% (90% interval 53.1–56.9) of forecast-error variance. E-mini S&P 500 and Nasdaq 100 futures are the only robust net transmitters (+36.0 and +25.6 percentage points), while the realized volatility of the VIX (−20.6), crude oil (−14.2) and sterling (−13.2) are net receivers; the net positions of bitcoin and ether are statistically indistinguishable from zero. Bitcoin and ether form a tightly linked pair (27–28% of each other's variance) whose exposure to traditional assets rises from under 10% of forecast-error variance before 2020 to 20–46% in 2020–2025, with information flowing predominantly from U.S. equities to crypto: in the 2024–2026 spot-ETF period bitcoin is a significant net receiver (−12.4, interval −20.4 to −2.1). Rolling estimates show that total connectedness rose during the COVID-19 crash and reached its highest level in 2025.",
4831 + "Intraday, after converting exchange time stamps to UTC, equity and VIX volatility concentrate in the U.S. cash session (53–63% of daily variance), whereas bitcoin's hourly profile is nearly flat (peak-to-trough ratio 1.2); U.S.-session equity volatility predicts bitcoin's next Asian-session volatility, but neither bitcoin's overnight nor its weekend volatility predicts U.S. equity volatility. The paper documents how calendar alignment of 24/7 and exchange-traded series materially affects connectedness estimates."
4832 + ],
4833 + "findings": [
4834 + {
4835 + "value": "2,514",
4836 + "label": "common trading days, ten instruments, 2016–2026"
4837 + },
4838 + {
4839 + "value": "54.8%",
4840 + "label": "total spillover index, 90% interval 53.1–56.9"
4841 + },
4842 + {
4843 + "value": "+36.0 / +25.6",
4844 + "label": "net transmission of S&P 500 and Nasdaq 100 futures"
4845 + },
4846 + {
4847 + "value": "−20.6",
4848 + "label": "net position of VIX realized volatility, a receiver"
4849 + },
4850 + {
4851 + "value": "<10% → 20–46%",
4852 + "label": "bitcoin variance from traditional assets, before/after 2020"
4853 + },
4854 + {
4855 + "value": "53–63%",
4856 + "label": "equity and VIX daily variance in the U.S. session"
4857 + }
4858 + ],
4859 + "contributions": [
4860 + {
4861 + "title": "A volatility connectedness hierarchy with intervals",
4862 + "description": "On log realized volatility from 5-minute bars, cross-asset shocks explain 54.8% of forecast-error variance; only ES (+36.0) and NQ (+25.6) are robust net transmitters, and VIX, crude oil and sterling are robust net receivers, across lag orders, horizons and sub-samples."
4863 + },
4864 + {
4865 + "title": "Crypto is integrated, and the flow runs from equities",
4866 + "description": "Bitcoin's own-shock share is 67%, the share of its variance explained by traditional assets rises from 6.8% in 2019 to 34% in 2020 and 46% in 2025, and in the spot-ETF period bitcoin (−12.4) and ether (−19.5) are significant net receivers."
4867 + },
4868 + {
4869 + "title": "Sessions measured on a correct clock",
4870 + "description": "After converting Eastern time stamps to UTC, equity and VIX volatility peak at the U.S. cash open and the U.S. session carries 53% (ES), 54% (NQ) and 63% (VIX) of daily variance; bitcoin's hourly profile is nearly flat (peak-to-trough 1.2)."
4871 + },
4872 + {
4873 + "title": "Equity volatility predicts crypto, not the reverse",
4874 + "description": "The previous day's U.S.-session volatility of equity futures predicts bitcoin's Asian-session volatility, whereas bitcoin's overnight and weekend volatility do not predict U.S.-session equity volatility (weekend coefficient −0.04, s.e. 0.05)."
4875 + },
4876 + {
4877 + "title": "Data construction moves the estimates",
4878 + "description": "Replacing a forward-filled calendar-day panel with common trading days changed the total index by five points and the net spillover of equity futures by fifteen, and removed the apparent 2018 pre-Volmageddon spike (81.6% → 56.4%) and the near-complete isolation of crypto reported in the earlier version."
4879 + }
4880 + ],
4881 + "data": [
4882 + {
4883 + "title": "FirstRate Data 5-minute bars, ten instruments",
4884 + "description": "5-minute OHLCV bars 2008–2026 in U.S. Eastern time for continuous futures (ES, NQ, GC, CL), spot FX (EURUSD, USDJPY, GBPUSD), spot bitcoin and ether (composite of exchanges) and the VIX index, stored in four DuckDB databases (about 235 million bars, not redistributed); daily realized volatility is the sum of squared 5-minute log returns, with the VAR panel restricted to the 2,514 CBOE trading days on which all ten series have an own value."
4885 + },
4886 + {
4887 + "title": "1-minute bars for the intraday stage",
4888 + "description": "Same-vendor 1-minute bars (shared with Working Paper No. 12) converted from Eastern time to UTC and aggregated to 5 minutes, March 2016–March 2026, for hourly profiles, session shares and session-ordered predictive regressions."
4889 + }
4890 + ],
4891 + "methodology": [
4892 + {
4893 + "title": "Realized volatility and log transformation",
4894 + "description": "Daily realized variance from 5-minute returns per instrument (at least 20 bars; returns beyond 20 sample standard deviations dropped), annualised and log-transformed for the VAR, with ADF and KPSS tests reported."
4895 + },
4896 + {
4897 + "title": "Diebold–Yilmaz generalized variance decomposition",
4898 + "description": "VAR(4) chosen by Hannan–Quinn, generalized forecast-error variance decomposition at a ten-day horizon, total, directional, net and net pairwise spillovers, with a residual bootstrap (B = 500) for 90% intervals."
4899 + },
4900 + {
4901 + "title": "Robustness grid, rolling windows and regimes",
4902 + "description": "Fifteen specifications (levels vs logs, lag orders 1–6, horizons 5 and 20, weekend-inclusive crypto, exclusion of VIX or crypto, 2018–2026 sub-sample), rolling windows of 200, 250 and 500 trading days, four regimes and calendar years with bootstrap intervals."
4903 + },
4904 + {
4905 + "title": "Granger causality and session regressions",
4906 + "description": "Daily Granger tests on log realized volatility with classical F and HAC Wald statistics (16 pairs), and session-ordered predictive regressions between equity futures, the VIX and bitcoin including a weekend-bitcoin to Monday-equity test."
4907 + }
4908 + ],
4909 + "reproducibility": [
4910 + "Three-stage Python pipeline: 01 builds daily realized volatility from the raw DuckDB bars (about 10 minutes); 02 reproduces every connectedness table and figure from the committed results/daily_rv_all.csv in about 1 minute; 03 runs the UTC-correct intraday analysis from 1-minute bars",
4911 + "Construction audit table reconciles the v1.0 numbers (forward-filled panel) with the revised ones, and the full 15-specification robustness grid is computed by script",
4912 + "Eight figures and all LaTeX tables generated from results/rev_*.csv; full LaTeX source of the 48-page paper",
4913 + "Pre-revision PDF, review report and changelog preserved in docs/; the raw 5-minute bars are not redistributed"
4914 + ],
4915 + "keywords": [
4916 + "realized volatility",
4917 + "volatility spillovers",
4918 + "connectedness",
4919 + "generalized variance decomposition",
4920 + "high-frequency data",
4921 + "cryptocurrency",
4922 + "trading sessions",
4923 + "bitcoin",
4924 + "VIX"
4925 + ],
4926 + "repo": "https://git.spboucher.ai/wp6_uqo",
4927 + "jel": "C32, C58, G12, G15"
4928 + },
4929 + {
4930 + "slug": "wp7",
4931 + "hero": {
4932 + "headline": "The option surface forecasts variance, not returns",
4933 + "subheadline": "Implied features explain 0.3% of 5-day stock returns and only one of fourteen strategies survives a 10 bp cost, while the surface lifts the HAR-RV R² from 0.421 to 0.515 at the weekly horizon in every subperiod."
4934 + },
4935 + "abstract": [
4936 + "This paper measures the information content of the implied volatility surface for returns and realized variance in a panel of 69 liquid US underlyings (49 large stocks, 17 ETFs and three equity indices) over 2010–2025, built from 3.83 billion end-of-day option records and 5-minute prices. Option-implied features predict the cross-section of weekly returns in a statistically detectable but economically marginal way: they explain 0.3% of the variation in 5-day stock returns, 0.6% for ETFs and 1.0% for indices; under two-way clustered standard errors only the 25-delta skew remains significant, with a positive sign consistent with compensation for skewness risk; and of fourteen long-short strategies implemented in calendar time with measured turnover, only a sort on the implied volatility level is profitable after a 10 basis-point cost, with a t-statistic of 2.2 that does not survive a multiple-testing correction.",
4937 + "The surface is more informative about second moments. On a common sample it raises the in-sample R² of the HAR-RV model from 0.415 to 0.457 for next-day realized variance and from 0.421 to 0.515 for weekly realized variance, in every one of nine subperiods; out of sample the gain is reliable under mean squared error but not under QLIKE, and at the weekly horizon the level of implied volatility alone is the best forecaster under both losses. The ratio of implied to realized correlation among S&P 500 constituents predicts stress episodes at 5- to 20-day horizons beyond the VIX (t of 2.3–4.1 with overlap-robust standard errors). At-the-money implied volatility Granger-causes realized variance in every ticker, but implied-volatility shocks account for 20–29% of its 20-day forecast-error variance.",
4938 + "Open-interest concentration does not attract the underlying price, and tree-based learners trained on the SPX surface do not beat HAR-RV out of sample. The revision corrects a target-alignment error in the previous version, whose weekly return results (R² of 5–19% and a long-short Sharpe ratio of 2.33) are withdrawn."
4939 + ],
4940 + "findings": [
4941 + {
4942 + "value": "3.83 billion",
4943 + "label": "end-of-day option records behind the panel"
4944 + },
4945 + {
4946 + "value": "69",
4947 + "label": "liquid US underlyings, 264,383 ticker-days, 2010–2025"
4948 + },
4949 + {
4950 + "value": "0.3%",
4951 + "label": "R² of implied features for 5-day stock returns"
4952 + },
4953 + {
4954 + "value": "0.421 → 0.515",
4955 + "label": "HAR-RV R² for weekly realized variance with the surface"
4956 + },
4957 + {
4958 + "value": "1 of 14",
4959 + "label": "long-short strategies profitable after 10 bp costs"
4960 + },
4961 + {
4962 + "value": "20–29%",
4963 + "label": "20-day RV forecast-error variance due to IV shocks"
4964 + }
4965 + ],
4966 + "contributions": [
4967 + {
4968 + "title": "Return predictability is marginal and mostly non-tradable",
4969 + "description": "Weekly R² is 0.29% for stocks, 0.61% for ETFs and 1.01% for indices; only the 25-delta skew survives two-way clustering (+19 bp per standard deviation, t = 2.91), and daily predictability is absent."
4970 + },
4971 + {
4972 + "title": "Calendar-time strategies with measured turnover",
4973 + "description": "Fourteen long-short sorts are implemented à la Jegadeesh–Titman with Newey–West t-statistics; only the ATM implied-volatility sort keeps a positive net Sharpe ratio (0.34 at 10 bp, break-even cost 27 bp, gross t = 2.19)."
4974 + },
4975 + {
4976 + "title": "The surface improves HAR-RV in every subperiod",
4977 + "description": "On a common 119,093-observation sample the in-sample R² rises by 9.9% at one day and 22.3% at five days; out of sample the median MSE ratio is 0.967 (better for 14 of 30 tickers) but the QLIKE ratio is 1.17 (worse for 15)."
4978 + },
4979 + {
4980 + "title": "Implied correlation signals market stress",
4981 + "description": "The implied-to-realized correlation ratio among S&P 500 constituents predicts stress at 5, 10 and 20 days beyond the VIX, with Newey–West t-statistics of 3.27, 4.13 and 2.33 over February 2016–December 2025."
4982 + },
4983 + {
4984 + "title": "Granger causality without dominance",
4985 + "description": "ATM implied volatility Granger-causes realized variance in 100% of tickers (mean F = 62.4), yet Cholesky-orthogonalised FEVD attributes only 20.2–28.5% of the 20-day RV variance to IV shocks."
4986 + },
4987 + {
4988 + "title": "A corrected and fully regenerated paper",
4989 + "description": "The 5-day targets of v1.1 were trailing windows; with strictly forward targets the weekly R² of 5–19%, the kurtosis-sort Sharpe of 2.33 and the R²_OOS of 0.99 disappear, and all 27 tables are now generated from the result files."
4990 + }
4991 + ],
4992 + "data": [
4993 + {
4994 + "title": "Option chains and intraday prices",
4995 + "description": "3.83 billion end-of-day option records with Greeks for 11,077 underlyings and 11.5 billion 1- and 5-minute OHLCV rows (2010–2025) from the HF Market Data lake, stored in DuckDB."
4996 + },
4997 + {
4998 + "title": "Analysis panel",
4999 + "description": "264,383 ticker-days for a fixed ex-post universe of 69 liquid names (119,081 with all ten predictors): ATM IV at 30 and 90 days, term slope, 25-delta skew, smile-based skewness and kurtosis proxies, put–call ratios, net gamma, and 5-minute realized variances; five derived parquets shipped."
5000 + }
5001 + ],
5002 + "methodology": [
5003 + {
5004 + "title": "Cross-sectional return regressions",
5005 + "description": "Pooled OLS with HC1, Newey–West and two-way clustered errors plus Fama–MacBeth with Newey–West lags, on strictly forward 1- and 5-day returns; quintile and decile sorts with overlap-robust t-statistics."
5006 + },
5007 + {
5008 + "title": "Calendar-time portfolio strategies",
5009 + "description": "Overlapping-cohort long-short portfolios for every sort at 1- and 5-day horizons with measured turnover, net Sharpe ratios at 5–50 bp and break-even costs; Bonferroni adjustment across strategies."
5010 + },
5011 + {
5012 + "title": "HAR-RV versus implied-volatility forecasting",
5013 + "description": "HAR-RV with daily, weekly and monthly components against IV-augmented models on a common sample; rolling out-of-sample windows (500/250), MSE, QLIKE and Diebold–Mariano tests with Newey–West variance for 30 tickers."
5014 + },
5015 + {
5016 + "title": "Correlation divergence and VAR analysis",
5017 + "description": "Equal-weight implied correlation versus 22-day realized correlation in three separate stress specifications with Newey–West errors; per-ticker Granger tests and bivariate VAR(5) with Cholesky-orthogonalised IRF and FEVD under both orderings."
5018 + },
5019 + {
5020 + "title": "Robustness battery",
5021 + "description": "Nine subperiods, VIX regimes, leave-one-year-out, winsorisation, alternative lag lengths, Spearman information coefficients, controls and a within-ticker permutation placebo (R² 0.0004 versus 0.0031)."
5022 + }
5023 + ],
5024 + "reproducibility": [
5025 + "Parquet-only pipeline (scripts 02–14) reproducible in under two minutes via make pipeline; WP7_LEGACY_TARGETS=1 reproduces the mis-aligned v1.1 targets for audit",
5026 + "All 27 LaTeX table fragments generated by scripts/14_make_paper_tables.py; no hand-typed numbers except the frozen raw-data inventory",
5027 + "v1.1 result CSVs preserved in results/_pre-revision-20261017/; Crossref verification logs for all 97 references in docs/verify/",
5028 + "Full LaTeX source with an anonymised version (main_anon.pdf) for journal submission; pre-revision PDF and review report in docs/"
5029 + ],
5030 + "keywords": [
5031 + "option-implied information",
5032 + "implied volatility surface",
5033 + "realized variance",
5034 + "HAR-RV",
5035 + "implied correlation",
5036 + "portfolio sorts",
5037 + "transaction costs",
5038 + "Granger causality"
5039 + ],
5040 + "repo": "https://git.spboucher.ai/wp7_uqo",
5041 + "jel": "G12, G13, G14, G17, C53, C58"
5042 + },
5043 + {
5044 + "slug": "wp8",
5045 + "hero": {
5046 + "headline": "Blockchain state prices sum to one, within the spread, and lead nothing",
5047 + "subheadline": "The complete set of claims is priced at 1.017 (Fed) and 1.015 (election), an overround within the half-spreads that declines towards resolution; the implied policy expectation moves 2–5 times more on FOMC and employment days, yet neither leads nor lags Treasuries, equities, the VIX or Bitcoin."
5048 + },
5049 + "abstract": [
5050 + "Negative-risk events on Polymarket list one binary contract for every outcome of a decision and settle exactly one of them at one dollar, so that their prices are observable analogues of Arrow–Debreu state prices. The paper uses the five-outcome market on the June 2026 FOMC decision (146 daily closes, December 2025 to May 2026) and the seventeen-contract market on the 2024 U.S. presidential election (304 daily closes) to ask how the Arrow–Debreu pricing restrictions fare in a blockchain market. Because spanning is built into the contract design, the testable content is the law of one price for the complete set of claims, whose executable no-arbitrage bounds are derived from the platform's minting and conversion mechanics.",
5051 + "The sum of state prices averages 1.017 (Fed) and 1.015 (election); the overround is statistically positive but of the order of the quoted half-spreads of the low-priced legs, declines steadily towards resolution, reverts within two days, and is bracketed by the bid- and ask-side sums of the order book. It has the wrong sign to be explained by the time value of money, which places the complete set below one. The normalised prices define a risk-neutral distribution over the FOMC decision whose mean, a Polymarket analogue of the futures-implied expected policy change, moves two to five times more, in variance terms, on FOMC and employment-report days than on other days, while weekend trading carries about one third of weekday variance.",
5052 + "Once Polymarket closes are aligned to the U.S. trading calendar and lag orders are chosen by information criteria, daily changes in this measure neither lead nor lag the ten-year Treasury yield, the S&P 500, the VIX or Bitcoin. The paper documents how a one-day misalignment of blockchain time stamps and a minimum-p lag search generated spurious lead–lag findings in an earlier version."
5053 + ],
5054 + "findings": [
5055 + {
5056 + "value": "146 / 304",
5057 + "label": "daily closes, Fed (5 claims) and election (17)"
5058 + },
5059 + {
5060 + "value": "1.017 / 1.015",
5061 + "label": "mean sum of state prices, Fed and election"
5062 + },
5063 + {
5064 + "value": "5.1 → 0.4 pp",
5065 + "label": "Fed overround, first 30 days to last 30 days"
5066 + },
5067 + {
5068 + "value": "0.999 / 1.004",
5069 + "label": "bid-side and ask-side sums of the order book"
5070 + },
5071 + {
5072 + "value": "4.9×",
5073 + "label": "variance ratio of implied policy change, release days"
5074 + },
5075 + {
5076 + "value": "p > 0.18",
5077 + "label": "all Granger tests, both directions, four markets"
5078 + }
5079 + ],
5080 + "contributions": [
5081 + {
5082 + "title": "Executable no-arbitrage bounds from platform mechanics",
5083 + "description": "Minting and conversion mechanics imply e^{−rτ} − c ≤ Σ ≤ 1 + c for the complete set; the mean discounted lower bound is 0.989, so the time value of money cannot explain a sum above one."
5084 + },
5085 + {
5086 + "title": "The overround is not an arbitrage",
5087 + "description": "Σ averages 1.0167 (Fed, HAC t = 3.8) and 1.0146 (election), falls from 5.1 to 0.4 percentage points over the Fed market's life, has a half-life of 2.1 days, and the 22 May 2026 order book shows bid and ask sums of 0.999 and 1.004."
5088 + },
5089 + {
5090 + "title": "A risk-neutral policy expectation that responds to news",
5091 + "description": "The implied expected June 2026 policy change moves 1.15 basis points on the 13 scheduled release days against 0.51 on the other 132 (variance ratio 4.9, Brown–Forsythe p = 0.008), and the modal state price moves 5.5 versus 1.4 points; weekends carry about one third of weekday variance."
5092 + },
5093 + {
5094 + "title": "No daily lead–lag with traditional markets",
5095 + "description": "On the U.S. trading-day grid with lag order chosen by AIC/BIC/HQIC and HAC tests, all Granger tests between the Polymarket measure and the 10-year yield, SPY, VIX and Bitcoin have p-values between 0.19 and 0.90; the contemporaneous correlation with the 10-year yield change is 0.26."
5096 + },
5097 + {
5098 + "title": "An earlier lead–lag result traced to a calendar error",
5099 + "description": "The version 1.0 finding that Polymarket led the VIX (p = 0.001) came from labelling 00:00 UTC closes as the following day and selecting lags by minimum p-value; the paper reconciles the two versions explicitly."
5100 + }
5101 + ],
5102 + "data": [
5103 + {
5104 + "title": "Polymarket negative-risk groups",
5105 + "description": "Daily price histories, order-book snapshot and metadata from the Polymarket Gamma and CLOB APIs for the Fed June 2026 group (event 101772, 5 outcomes, $37.8 million volume, 146 closes) and the 2024 Presidential Election Winner event (903193, 17 contracts, $3.7 billion volume, 304 closes), snapshot of 22 May 2026; closes relabelled to the UTC day they end."
5106 + },
5107 + {
5108 + "title": "Macro and market series",
5109 + "description": "Effective federal funds rate, CPI and unemployment releases from FRED; SPY, VIX, 10-year yield index and BTC-USD from Yahoo Finance, aligned to the U.S. trading calendar (110 common trading days for the Fed sample)."
5110 + }
5111 + ],
5112 + "methodology": [
5113 + {
5114 + "title": "Pricing of the complete set",
5115 + "description": "Daily sums Σ_t, HAC test of the overround, shares within ±0.5/1/2/5 points, discounted bounds from the federal funds rate, AR(1) half-life, ADF and a term-structure regression on time to resolution, with order-book bid, ask and depth sums."
5116 + },
5117 + {
5118 + "title": "Risk-neutral distribution",
5119 + "description": "Normalised prices q_j = p_j/Σ define the distribution over the five policy states valued at −50, −25, 0, +25 and +50 basis points; its mean and standard deviation give an implied expected policy change and its dispersion."
5120 + },
5121 + {
5122 + "title": "Announcement-day analysis",
5123 + "description": "Day-0 and cumulative changes for 4 FOMC, 4 CPI and 5 employment releases; release versus other days compared by variance ratios, Brown–Forsythe and one-sided Mann–Whitney tests, and weekend versus weekday variance."
5124 + },
5125 + {
5126 + "title": "Lead–lag tests",
5127 + "description": "Bivariate Granger tests and a five-variable VAR(1) on the trading-day grid with lag order chosen by AIC, BIC and HQIC, HAC Wald tests and fixed-lag sensitivity; GARCH, GJR and HAR volatility models relegated to an appendix after ARCH-LM fails to reject."
5128 + }
5129 + ],
5130 + "reproducibility": [
5131 + "scripts/02_revision_analysis.py regenerates every table, figure and quoted number (results/revision_stats.json) offline from the raw snapshot in about 1 minute",
5132 + "Download and cleaning pipeline in src/ (Polymarket, FRED, Yahoo clients; close-of-day relabelling; AIC lag selection) with a FRED key for the network stage",
5133 + "Nine LaTeX table fragments and twelve figures generated by script; full LaTeX source of the 44-page paper with 62 verified references",
5134 + "Appendix C reconciles version 1.0 with version 1.1; pre-revision PDF, review report and changelog kept in docs/"
5135 + ],
5136 + "keywords": [
5137 + "prediction markets",
5138 + "Arrow–Debreu securities",
5139 + "state prices",
5140 + "law of one price",
5141 + "monetary policy expectations",
5142 + "macroeconomic announcements",
5143 + "Polymarket",
5144 + "FOMC",
5145 + "lead–lag"
5146 + ],
5147 + "repo": "https://git.spboucher.ai/wp8_uqo",
5148 + "jel": "G13, G14, E52, D47, C58"
5149 + },
5150 + {
5151 + "slug": "wp9",
5152 + "hero": {
5153 + "headline": "In Canada, the neighbourhood is worth at least as much as the dwelling",
5154 + "subheadline": "Across 142,121 MLS listings, 1,155 neighbourhood fixed effects lift R² from 44% to 78%; a Shapley–Owen decomposition credits structure with 34 points and location with 36, and neighbourhood premia span a factor of about ten."
5155 + },
5156 + "abstract": [
5157 + "This paper estimates a national hedonic price function for the Canadian residential market on a cross-section of 142,121 active MLS listings from nine provinces. A semi-logarithmic specification relates the log list price to structural attributes, dwelling type and ownership form, and 1,155 absorbed neighbourhood (Forward Sortation Area) fixed effects estimated by absorbing least squares with neighbourhood-clustered inference. Moving from a structural model to the full specification raises the explained share of log-price variation from 44% to 78%. Because structure and location are correlated, the attribution of this gain depends on the order in which blocks enter: a Shapley–Owen decomposition assigns about 34 percentage points to structure and 36 to location, so that the two are of comparable importance and the neighbourhood is the single largest block once province and neighbourhood are grouped.",
5158 + "Conditional on location, the living-area elasticity is 0.53 (0.49–0.62 across samples), each full bathroom adds about 12%, and bedroom counts are economically negligible. The implicit prices of area and bathrooms survive province-level wild-cluster bootstrap inference, whereas the small half-bathroom and bedroom effects do not. Neighbourhood effects remove 87% of the spatial autocorrelation in residuals (Moran's I from 0.50 to 0.06). The model values held-out listings in neighbourhoods seen in training with an out-of-sample R² of 0.77 and a median absolute percentage error of 15.6%; for neighbourhoods never seen in training, spatial interpolation of the neighbourhood effect still yields R² = 0.74 and a median error of 17%.",
5159 + "Floor space displays mild diminishing returns, the location premium falls by about 5.5% per doubling of distance to the nearest metropolitan centre, and implicit prices vary across the price distribution and across provinces. Neighbourhood premia span a factor of about ten between the most and least expensive areas. Because the dependent variable is the asking price rather than the transaction price, all estimates describe the pricing of listings, not of sales."
5160 + ],
5161 + "findings": [
5162 + {
5163 + "value": "142,121",
5164 + "label": "active MLS listings across nine provinces"
5165 + },
5166 + {
5167 + "value": "1,155",
5168 + "label": "absorbed neighbourhood (FSA) fixed effects"
5169 + },
5170 + {
5171 + "value": "44% → 78%",
5172 + "label": "explained log-price variance, structural to full model"
5173 + },
5174 + {
5175 + "value": "34 / 36 pp",
5176 + "label": "Shapley–Owen shares of R² for structure and location"
5177 + },
5178 + {
5179 + "value": "0.53",
5180 + "label": "living-area elasticity conditional on neighbourhood"
5181 + },
5182 + {
5183 + "value": "15.6%",
5184 + "label": "out-of-sample MdAPE in neighbourhoods seen in training"
5185 + }
5186 + ],
5187 + "contributions": [
5188 + {
5189 + "title": "An order-free answer to how much location is worth",
5190 + "description": "The sequential ladder credits structure with 44 points of R² and location with 32; the Shapley–Owen decomposition gives 34 and 36, making location the largest block once province and neighbourhood are grouped."
5191 + },
5192 + {
5193 + "title": "Stable structural implicit prices",
5194 + "description": "Living-area elasticity 0.529 (0.49–0.62 across eight sample cuts), +11.9% per full bathroom, bedrooms −0.006 and half-baths −0.034; area and bathroom effects survive province-level wild-cluster bootstrap (p = 0.004), the small effects do not."
5195 + },
5196 + {
5197 + "title": "Neighbourhood effects absorb spatial dependence",
5198 + "description": "Residual Moran's I falls from 0.50 to 0.06 (k = 10), with the same pattern at k = 5 and k = 20; a transparent fixed-effects design removes 87% of the autocorrelation that motivates heavier spatial machinery."
5199 + },
5200 + {
5201 + "title": "Valuation accuracy depends on spatial support",
5202 + "description": "Random 80/20 holdouts give R² 0.772 and MdAPE 15.6% (five seeds: 0.769 ± 0.010); holding out 20% of FSAs gives R² 0.74 and MdAPE 17.3% with inverse-distance interpolation of the neighbourhood effect, versus 0.52 and 25% with province means."
5203 + },
5204 + {
5205 + "title": "Dwelling type and urban gradient",
5206 + "description": "Relative to an apartment of the same size and neighbourhood, a house lists for +46%, a triplex +43% and a duplex +16% (Kennedy-corrected); the location premium declines 5.5% per doubling of distance to the nearest of nine metros, from +21% within 5 km to −38% beyond 320 km."
5207 + },
5208 + {
5209 + "title": "Neighbourhood premia span a factor of ten",
5210 + "description": "Vancouver's west side (V6S +226%, V6T +201%), Whistler and West Vancouver top the ranking; rural Newfoundland (A0B −69%) and Saskatchewan bottom it; the first Ontario FSA (M5R) ranks 14th."
5211 + }
5212 + ],
5213 + "data": [
5214 + {
5215 + "title": "Canadian MLS listings",
5216 + "description": "142,121 active listings (83,524 houses, 57,853 condos, 744 other) with asking prices, drawn from a 172,019-listing Realtor.ca DuckDB snapshot; province derived from the postal-code letter, 1,146 FSAs with at least 25 listings plus nine pooled residual categories; median price $639,900 and living area 135.3 m²."
5217 + }
5218 + ],
5219 + "methodology": [
5220 + {
5221 + "title": "Absorbing least squares with a model ladder",
5222 + "description": "Semi-log price on log area, structural attributes, dwelling type and ownership with M1–M5 nesting from structure to province to 1,155 FSA effects; FSA-clustered baseline inference, with HC1, province-clustered, two-way and restricted wild-cluster bootstrap (Rademacher, 512 draws) comparisons."
5223 + },
5224 + {
5225 + "title": "Shapley–Owen variance decomposition",
5226 + "description": "R² attributed to structure, type, province and neighbourhood blocks by averaging over all orderings, contrasted with the sequential ladder."
5227 + },
5228 + {
5229 + "title": "Out-of-sample validation with and without spatial support",
5230 + "description": "Repeated random 80/20 splits (five seeds) with Duan-smeared retransformation, and a spatial holdout of 20% of FSAs with province-mean or inverse-distance interpolation of the neighbourhood effect."
5231 + },
5232 + {
5233 + "title": "Nonlinearity, gradients and heterogeneity",
5234 + "description": "Quadratic and spline floor-area terms, quantile hedonic regressions, a distance-to-metro gradient with metro intercepts and a quadratic, eight robustness cuts, nine provincial subsamples and leave-one-province-out transfer."
5235 + },
5236 + {
5237 + "title": "Spatial diagnostics",
5238 + "description": "Moran's I on structural and full-model residuals with k = 5, 10 and 20 nearest neighbours and permutation p-values."
5239 + }
5240 + ],
5241 + "reproducibility": [
5242 + "Six numbered Python scripts from the committed analysis parquet (sample build, core and extended estimation, figures, tables, revision analyses); total wall-clock about 2.5 minutes with fixed seeds",
5243 + "All v1.3 numbers come from results/reproduced/; the v1.2 reference outputs are kept untouched and reconciled in Appendix C of the paper",
5244 + "Seven new tables (Shapley, standard-error comparison, splines, spatial holdout, Moran sensitivity, type effects, gradient) and figures generated programmatically",
5245 + "Full LaTeX source of the 52-page paper; pre-revision PDF, review report and changelog archived in docs/"
5246 + ],
5247 + "keywords": [
5248 + "hedonic pricing",
5249 + "housing markets",
5250 + "Canada",
5251 + "high-dimensional fixed effects",
5252 + "neighbourhood effects",
5253 + "spatial autocorrelation",
5254 + "automated valuation model",
5255 + "list prices",
5256 + "Shapley decomposition"
5257 + ],
5258 + "repo": "https://git.spboucher.ai/wp9_uqo",
5259 + "jel": "R31, R21, C21, C55"
5260 + },
5261 + {
5262 + "slug": "chapter-1",
5263 + "repo": "https://git.spboucher.ai/phd_thesis",
5264 + "hero": {
5265 + "headline": "Speculators Dampen Macro News Shocks in Energy Futures",
5266 + "subheadline": "Does speculative trading amplify macroeconomic news in commodity futures? No — greater speculation dampens price drift, volatility, and spreads while improving liquidity."
5267 + },
5268 + "abstract": [
5269 + "Using five-minute futures data from 2007 to 2024 and 26 macroeconomic announcement releases, this essay asks whether speculative trading amplifies or dampens the impact of macro news on commodity futures. Speculation intensity is measured with an NLS proxy built from the CFTC disaggregated Commitments of Traders report, distinguishing money managers from swap dealers across energy (crude oil, natural gas) and metals (gold, silver, copper, palladium) contracts.",
5270 + "The evidence points to a stabilizing role for speculators: increased speculative trading dampens the impact of standardized macro surprises on price drift, volatility, and bid-ask spreads, improving liquidity and price discovery. The damping effect is stronger for procyclical commodities such as oil and natural gas than for safe havens like gold, and the beneficial effects are driven by money managers rather than swap dealers."
5271 + ],
5272 + "findings": [
5273 + {
5274 + "value": "26",
5275 + "label": "Macroeconomic announcement releases studied"
5276 + },
5277 + {
5278 + "value": "6",
5279 + "label": "Futures contracts (CL, NG, GC, SI, HG, PA)"
5280 + },
5281 + {
5282 + "value": "5-min",
5283 + "label": "Sampling frequency of futures data, 2007–2024"
5284 + },
5285 + {
5286 + "value": "14",
5287 + "label": "Tables of results in the article"
5288 + },
5289 + {
5290 + "value": "6",
5291 + "label": "Figures documenting speculation and news effects"
5292 + }
5293 + ],
5294 + "contributions": [
5295 + {
5296 + "title": "Speculation conditions macro news impact",
5297 + "description": "Shows that the intensity of speculative trading systematically conditions how commodity futures react to standardized macroeconomic surprises, rather than treating announcement effects as uniform."
5298 + },
5299 + {
5300 + "title": "Evidence that speculators stabilize markets",
5301 + "description": "Increased speculative activity dampens the impact of macro surprises on price drift, volatility, and bid-ask spreads, countering the view that speculation amplifies commodity price shocks."
5302 + },
5303 + {
5304 + "title": "Trader-type decomposition of effects",
5305 + "description": "Using the CFTC disaggregated Commitments of Traders data, the beneficial liquidity and price-discovery effects are attributed to money managers, not swap dealers."
5306 + },
5307 + {
5308 + "title": "Procyclical versus safe-haven contrast",
5309 + "description": "Documents that the damping effect is stronger for procyclical commodities such as crude oil and natural gas than for safe havens like gold."
5310 + },
5311 + {
5312 + "title": "NLS speculation proxy at announcement times",
5313 + "description": "Builds a nonlinear-least-squares speculation intensity proxy from disaggregated positioning data and interacts it with high-frequency announcement-window reactions in energy and metals futures."
5314 + }
5315 + ],
5316 + "data": [
5317 + {
5318 + "title": "Five-minute commodity futures, 2007–2024",
5319 + "description": "High-frequency price data for crude oil (CL), natural gas (NG), gold (GC), silver (SI), copper (HG), and palladium (PA) futures."
5320 + },
5321 + {
5322 + "title": "26 macroeconomic announcement series",
5323 + "description": "Scheduled U.S. macroeconomic releases converted into standardized surprises to measure announcement-window reactions in returns, volatility, and spreads."
5324 + },
5325 + {
5326 + "title": "CFTC disaggregated Commitments of Traders",
5327 + "description": "Weekly positioning data separating money managers from swap dealers, used to construct the NLS speculation intensity proxy."
5328 + }
5329 + ],
5330 + "methodology": [
5331 + {
5332 + "title": "WLS-EWMA event regressions",
5333 + "description": "Weighted least squares with exponentially weighted moving average variance estimates to measure announcement effects on high-frequency returns and volatility."
5334 + },
5335 + {
5336 + "title": "GARCH volatility modelling",
5337 + "description": "GARCH-type specifications capture conditional volatility dynamics around macroeconomic announcement releases in energy and metals futures."
5338 + },
5339 + {
5340 + "title": "NLS speculation proxy from COT data",
5341 + "description": "A nonlinear-least-squares proxy for speculation intensity built from CFTC disaggregated positions, interacted with standardized macro surprises."
5342 + },
5343 + {
5344 + "title": "COVID and ZLB robustness appendices",
5345 + "description": "Dedicated appendices test the robustness of the results across the COVID-19 period and the zero-lower-bound monetary policy regime."
5346 + }
5347 + ],
5348 + "reproducibility": [
5349 + "Full LaTeX source lives in phd_chap1_20260731/ as a frozen July 31, 2026 snapshot: a monolithic main.tex plus tables.tex (14 tables), figures.tex (6 figures), and master.bib.",
5350 + "The folder includes COVID and ZLB robustness appendices and the compiled main.pdf alongside standalone figure files (FIG_NLS, FIG_MSCT, FIG_WT, per-commodity plots).",
5351 + "The chapter is also integrated into the full thesis in these-ulaval/, where labels are prefixed ch1: and the bibliography is merged into a consolidated 271-key BibTeX file."
5352 + ],
5353 + "keywords": [
5354 + "speculative trading",
5355 + "energy futures",
5356 + "macroeconomic announcements",
5357 + "high-frequency data",
5358 + "CFTC Commitments of Traders",
5359 + "liquidity",
5360 + "price discovery",
5361 + "volatility"
5362 + ]
5363 + },
5364 + {
5365 + "slug": "chapter-2",
5366 + "repo": "https://git.spboucher.ai/phd_thesis",
5367 + "hero": {
5368 + "headline": "Intraday iNAV Reveals Jump-Driven ETF Volatility Transmission",
5369 + "subheadline": "How does volatility flow between commodity ETFs and their underlyings? The intraday indicative NAV shows transmission runs through jumps, not diffusion."
5370 + },
5371 + "abstract": [
5372 + "This essay asks how volatility flows between commodity ETFs and their underlying assets, and what the intraday indicative NAV (iNAV) reveals that daily data cannot. It builds a novel minute-level iNAV dataset for four single-commodity ETFs — gold (GLD), silver (SLV), oil (USO), and natural gas (UNG) — from roughly 45 million tick observations spanning 2010 to 2023.",
5373 + "Realized variance is decomposed into continuous and jump components and modelled with HAR-X and HAR-CJ-X regressions at 1, 5, and 30 minutes, plus a Bayesian VAR. The iNAV yields a sharper image of the ETF–underlying volatility relationship: transmission runs primarily through jumps, sampling frequency matters (1-minute estimates are up to twice as large as 30-minute ones), and precious metals show unidirectional transmission while energy is bidirectional and asymmetric."
5374 + ],
5375 + "findings": [
5376 + {
5377 + "value": "≈45M",
5378 + "label": "Tick observations underlying the minute-level iNAV dataset, 2010–2023"
5379 + },
5380 + {
5381 + "value": "4",
5382 + "label": "Single-commodity ETFs studied (GLD, SLV, USO, UNG)"
5383 + },
5384 + {
5385 + "value": "2×",
5386 + "label": "1-minute transmission estimates up to twice as large as 30-minute ones"
5387 + },
5388 + {
5389 + "value": "3",
5390 + "label": "Sampling frequencies compared: 1, 5, and 30 minutes"
5391 + },
5392 + {
5393 + "value": "13",
5394 + "label": "Tables and 8 figures (realized volatility and IRF plots)"
5395 + }
5396 + ],
5397 + "contributions": [
5398 + {
5399 + "title": "Novel minute-level iNAV dataset",
5400 + "description": "Constructs an original minute-frequency indicative NAV dataset for four single-commodity ETFs from roughly 45 million tick observations covering 2010 to 2023."
5401 + },
5402 + {
5403 + "title": "Sharper view of ETF–underlying dynamics",
5404 + "description": "Shows the intraday iNAV gives a sharper image of the volatility relationship between ETFs and their underlying assets than daily data can provide."
5405 + },
5406 + {
5407 + "title": "Jumps as the transmission channel",
5408 + "description": "Decomposing realized variance shows volatility transmission runs primarily through jump components rather than the continuous diffusion component."
5409 + },
5410 + {
5411 + "title": "Sampling frequency matters for inference",
5412 + "description": "Comparing 1-, 5-, and 30-minute estimates shows 1-minute transmission coefficients can be up to twice as large as 30-minute ones."
5413 + },
5414 + {
5415 + "title": "Asset-class contrast in transmission direction",
5416 + "description": "Precious metals show unidirectional iNAV-to-ETF transmission consistent with passive arbitrage, while energy ETFs display bidirectional and asymmetric volatility flows."
5417 + }
5418 + ],
5419 + "data": [
5420 + {
5421 + "title": "Minute-level iNAV series, 2010–2023",
5422 + "description": "A novel intraday indicative NAV dataset for four single-commodity ETFs, built from approximately 45 million tick observations."
5423 + },
5424 + {
5425 + "title": "Commodity ETF prices: GLD, SLV, USO, UNG",
5426 + "description": "High-frequency prices for the gold, silver, crude oil, and natural gas ETFs, matched to their underlying assets at 1-, 5-, and 30-minute frequencies."
5427 + }
5428 + ],
5429 + "methodology": [
5430 + {
5431 + "title": "HAR-X and HAR-CJ-X models",
5432 + "description": "Heterogeneous autoregressive realized-volatility regressions with cross-market terms, estimated at 1-, 5-, and 30-minute sampling frequencies."
5433 + },
5434 + {
5435 + "title": "Jump decomposition (Barndorff-Nielsen–Shephard)",
5436 + "description": "Realized variance is decomposed into continuous and jump components to identify which channel carries volatility between ETFs and underlyings."
5437 + },
5438 + {
5439 + "title": "Minnesota-prior Bayesian VAR",
5440 + "description": "A Bayesian VAR with Minnesota prior traces impulse responses and the direction and asymmetry of volatility transmission between markets."
5441 + }
5442 + ],
5443 + "reproducibility": [
5444 + "Full LaTeX source in phd_chap2_20260731/ is organized as main.tex plus a sections/ folder (introduction, data, methods, results, conclusion), with 13 tables, 8 figures, and master.bib.",
5445 + "The folder is a frozen July 31, 2026 submission snapshot for the Journal of Futures Markets and includes the compiled main.pdf.",
5446 + "The thesis-adapted version lives in these-ulaval/chapitre2/, with labels prefixed ch2: and references merged into the consolidated thesis bibliography."
5447 + ],
5448 + "keywords": [
5449 + "commodity ETFs",
5450 + "indicative NAV",
5451 + "volatility transmission",
5452 + "realized volatility",
5453 + "jumps",
5454 + "HAR models",
5455 + "Bayesian VAR",
5456 + "high-frequency data"
5457 + ]
5458 + },
5459 + {
5460 + "slug": "chapter-3",
5461 + "repo": "https://git.spboucher.ai/phd_thesis",
5462 + "hero": {
5463 + "headline": "Fed Tone Moves Prices, Novelty Moves Volatility",
5464 + "subheadline": "When the Fed speaks, what moves markets — what it says or how new it is? Tone drives directional returns; novelty drives volatility."
5465 + },
5466 + "abstract": [
5467 + "This essay asks what moves markets when the Federal Reserve speaks: the policy tone of the statement or its informational novelty. FOMC statements (217 releases, 2000–2025) are decomposed into hawkish/dovish tone and novelty using a dual-model NLP ensemble (MiniLM and BERT with TSDAE+MNRL fine-tuning and PCA-based reference selection), then linked to 1-minute futures data across seven contracts around 148 FOMC events from 2008 to 2025.",
5468 + "The two dimensions play distinct roles. Tone predicts directional returns: a one-standard-deviation dovish shift generates equity gains building to about +12 basis points within two hours. Novelty predicts volatility: the stance-by-novelty interaction on the VIX persists from 5 to 120 minutes (t = -5.06), and stance moves realized volatility in six of seven contracts at the 1% level, while pre-announcement placebo tests are null."
5469 + ],
5470 + "findings": [
5471 + {
5472 + "value": "+12 bps",
5473 + "label": "Equity gain within two hours after a 1σ dovish tone shift"
5474 + },
5475 + {
5476 + "value": "t = −5.06",
5477 + "label": "Stance × novelty interaction on VIX, persisting 5–120 minutes"
5478 + },
5479 + {
5480 + "value": "6 of 7",
5481 + "label": "Contracts where stance moves realized volatility (p < 0.01)"
5482 + },
5483 + {
5484 + "value": "148",
5485 + "label": "FOMC events with 1-minute futures data, 2008–2025"
5486 + },
5487 + {
5488 + "value": "217",
5489 + "label": "FOMC statement releases scored for tone and novelty, 2000–2025"
5490 + },
5491 + {
5492 + "value": "7",
5493 + "label": "Futures contracts studied: ES, VX, ZN, ZF, DX, CL, GC"
5494 + }
5495 + ],
5496 + "contributions": [
5497 + {
5498 + "title": "Separating policy tone from novelty",
5499 + "description": "Decomposes FOMC statements into hawkish/dovish tone and informational novelty, showing the two dimensions have distinct, complementary effects on returns and volatility."
5500 + },
5501 + {
5502 + "title": "Dual-model NLP ensemble for Fed text",
5503 + "description": "Combines MiniLM and BERT with TSDAE+MNRL fine-tuning and PCA-based reference selection to score 217 FOMC statements from 2000 to 2025."
5504 + },
5505 + {
5506 + "title": "Tone predicts directional returns",
5507 + "description": "A one-standard-deviation dovish shift generates equity gains that build to roughly +12 basis points within two hours of the announcement."
5508 + },
5509 + {
5510 + "title": "Novelty predicts volatility dynamics",
5511 + "description": "The stance-by-novelty interaction on the VIX persists from 5 to 120 minutes after the release, with a t-statistic of -5.06."
5512 + },
5513 + {
5514 + "title": "Broad cross-market volatility evidence",
5515 + "description": "Policy stance moves realized volatility in six of seven futures contracts at the 1% significance level, spanning equities, rates, currencies, and commodities."
5516 + },
5517 + {
5518 + "title": "Placebo-validated announcement effects",
5519 + "description": "Pre-announcement placebo tests are null, confirming the estimated tone and novelty effects are driven by the FOMC announcements themselves."
5520 + }
5521 + ],
5522 + "data": [
5523 + {
5524 + "title": "FOMC statements, 2000–2025",
5525 + "description": "217 statement releases scored for hawkish/dovish policy tone and informational novelty via the dual-model NLP ensemble."
5526 + },
5527 + {
5528 + "title": "1-minute futures data around 148 FOMC events",
5529 + "description": "Minute-level prices from 2008 to 2025 for seven contracts: ES, VX, ZN, ZF, DX, CL, and GC."
5530 + }
5531 + ],
5532 + "methodology": [
5533 + {
5534 + "title": "NLP ensemble (MiniLM + BERT)",
5535 + "description": "A dual-model text ensemble with TSDAE+MNRL fine-tuning and PCA-based reference selection extracts tone and novelty from FOMC statements."
5536 + },
5537 + {
5538 + "title": "Event regressions and minute-level panels",
5539 + "description": "High-frequency event-window regressions and minute-level panel specifications link tone and novelty to returns and realized volatility across contracts."
5540 + },
5541 + {
5542 + "title": "Jordà local projections",
5543 + "description": "Local projections trace the dynamic response of returns and volatility to tone and novelty shocks over the post-announcement window."
5544 + },
5545 + {
5546 + "title": "Placebo tests and five inference methods",
5547 + "description": "Pre-announcement placebo windows and five alternative inference methods validate that the estimated effects are announcement-driven and statistically robust."
5548 + }
5549 + ],
5550 + "reproducibility": [
5551 + "Full LaTeX source in PHD_chapitre3_theses_20260731/ is organized as chapitre3.tex plus sections/, tables/ (24 tables), 19 figures, a mathematical proofs appendix, and master.bib, with the compiled chapitre3.pdf included.",
5552 + "The folder is a frozen July 31, 2026 snapshot of the March 6, 2026 manuscript; thesis adaptations live separately in these-ulaval/chapitre3/.",
5553 + "In the assembled thesis, the chapter's proofs and extras appear as appendices A–B and its labels are prefixed ch3:, with references merged into the 271-key consolidated bibliography."
5554 + ],
5555 + "keywords": [
5556 + "FOMC announcements",
5557 + "monetary policy tone",
5558 + "informational novelty",
5559 + "NLP",
5560 + "high-frequency finance",
5561 + "local projections",
5562 + "realized volatility",
5563 + "event study"
5564 + ]
5565 + }
5566 + ]
5567 +}
\ No newline at end of file
added content/seed/taxonomy.json +200 −0
@@ -0,0 +1,200 @@
1 +{
2 + "series": [
3 + { "slug": "uqo-working-papers", "code": "WP", "name_en": "UQO Working Papers", "name_fr": "Cahiers de recherche UQO", "description_en": "Working papers by Simon-Pierre Boucher, Université du Québec en Outaouais (Département des sciences administratives). Numbers are permanent; number 17 was reassigned to the FIRC series before public release and is therefore a gap.", "description_fr": "Cahiers de recherche de Simon-Pierre Boucher, Université du Québec en Outaouais (Département des sciences administratives). Les numéros sont permanents ; le numéro 17 a été transféré à la série FIRC avant diffusion publique et constitue donc un trou.", "sort": 1 },
4 + { "slug": "firc-research-papers", "code": "FIRC", "name_en": "FIRC Research Papers", "name_fr": "Cahiers de recherche FIRC", "description_en": "Papers from the FIRC-funded team project (Kammoun, Tandjambianda, Boucher) on corporate disclosures and AI.", "description_fr": "Cahiers issus du projet d'équipe financé par le FIRC (Kammoun, Tandjambianda, Boucher) sur les divulgations d'entreprises et l'IA.", "sort": 2 }
5 + ],
6 + "authors": [
7 + { "slug": "simon-pierre-boucher", "given_name": "Simon-Pierre", "family_name": "Boucher", "display_name": "Simon-Pierre Boucher", "is_primary": true, "email": "simon-pierre.boucher@uqo.ca", "website": "https://www.spboucher.ai", "affiliation_en": "Université du Québec en Outaouais", "affiliation_fr": "Université du Québec en Outaouais", "department_en": "Département des sciences administratives", "department_fr": "Département des sciences administratives", "bio_en": "Professor at the Université du Québec en Outaouais. Research in financial econometrics, real estate economics, high-frequency markets and AI systems, with an emphasis on preregistered, reproducible and computationally verified designs.", "bio_fr": "Professeur à l'Université du Québec en Outaouais. Recherche en économétrie financière, en économie immobilière, en marchés à haute fréquence et en systèmes d'IA, avec un accent sur les devis préenregistrés, reproductibles et vérifiés par le calcul.", "sort": 1 },
8 + { "slug": "manel-kammoun", "given_name": "Manel", "family_name": "Kammoun", "display_name": "Manel Kammoun", "is_primary": false, "sort": 2 },
9 + { "slug": "djerrycharli-tandjambianda", "given_name": "Djerrycharli", "family_name": "Tandjambianda", "display_name": "Djerrycharli Tandjambianda", "is_primary": false, "sort": 3 }
10 + ],
11 + "domains": [
12 + { "slug": "financial-economics", "name_en": "Financial economics", "name_fr": "Économie financière", "accent": "#1f3a5f", "description_en": "Asset pricing, prediction markets, derivatives and the economics of financial information.", "description_fr": "Évaluation des actifs, marchés de prédiction, produits dérivés et économie de l'information financière." },
13 + { "slug": "market-microstructure", "name_en": "Market microstructure", "name_fr": "Microstructure des marchés", "accent": "#5b2333", "description_en": "Price discovery, liquidity, fragility and the behaviour of trading agents.", "description_fr": "Découverte des prix, liquidité, fragilité et comportement des agents de marché." },
14 + { "slug": "high-frequency-econometrics", "name_en": "High-frequency econometrics", "name_fr": "Économétrie à haute fréquence", "accent": "#3b4d61", "description_en": "Realized measures, volatility forecasting and intraday transmission.", "description_fr": "Mesures réalisées, prévision de la volatilité et transmission intrajournalière." },
15 + { "slug": "artificial-intelligence", "name_en": "Artificial intelligence", "name_fr": "Intelligence artificielle", "accent": "#2d4a3e", "description_en": "Language-model systems, agents, inference runtimes and their measured behaviour.", "description_fr": "Systèmes de modèles de langage, agents, moteurs d'inférence et leur comportement mesuré." },
16 + { "slug": "computational-economics", "name_en": "Computational economics", "name_fr": "Économie computationnelle", "accent": "#6b4e16", "description_en": "General-equilibrium theory solved numerically, agent-based markets and simulation experiments.", "description_fr": "Théorie de l'équilibre général résolue numériquement, marchés à base d'agents et expériences de simulation." },
17 + { "slug": "real-estate-economics", "name_en": "Real estate economics", "name_fr": "Économie immobilière", "accent": "#7a3b2e", "description_en": "Hedonic pricing, automated valuation, housing indices, taxation and housing theory.", "description_fr": "Prix hédoniques, évaluation automatisée, indices immobiliers, fiscalité et théorie du logement." },
18 + { "slug": "applied-econometrics", "name_en": "Applied econometrics", "name_fr": "Économétrie appliquée", "accent": "#2f4858", "description_en": "Empirical designs on web-scraped and administrative data: retail prices, used cars, regional economies.", "description_fr": "Devis empiriques sur données administratives et moissonnées : prix de détail, voitures d'occasion, économies régionales." },
19 + { "slug": "scientific-computing", "name_en": "Scientific computing", "name_fr": "Calcul scientifique", "accent": "#444c5c", "description_en": "Distributed estimation, certified proofs, SAT-based discovery and computational mathematics.", "description_fr": "Estimation distribuée, preuves certifiées, découverte par SAT et mathématiques computationnelles." },
20 + { "slug": "interdisciplinary-research", "name_en": "Interdisciplinary research", "name_fr": "Recherche interdisciplinaire", "accent": "#5c4b6b", "description_en": "Research methodology, autonomous research agents and social-science hypotheses tested with formal models.", "description_fr": "Méthodologie de la recherche, agents de recherche autonomes et hypothèses de sciences sociales testées par des modèles formels." }
21 + ],
22 + "topics": [
23 + { "slug": "hedonic-pricing", "name_en": "Hedonic pricing", "name_fr": "Prix hédoniques" },
24 + { "slug": "automated-valuation", "name_en": "Automated valuation models", "name_fr": "Modèles d'évaluation automatisée" },
25 + { "slug": "housing-price-indices", "name_en": "Housing price indices", "name_fr": "Indices de prix immobiliers" },
26 + { "slug": "property-taxation", "name_en": "Property taxation and assessment", "name_fr": "Fiscalité foncière et évaluation" },
27 + { "slug": "short-term-rentals", "name_en": "Short-term rentals", "name_fr": "Locations de courte durée" },
28 + { "slug": "housing-theory", "name_en": "Housing theory", "name_fr": "Théorie du logement" },
29 + { "slug": "volatility", "name_en": "Volatility and realized measures", "name_fr": "Volatilité et mesures réalisées" },
30 + { "slug": "prediction-markets", "name_en": "Prediction markets", "name_fr": "Marchés de prédiction" },
31 + { "slug": "options-and-derivatives", "name_en": "Options and derivatives", "name_fr": "Options et produits dérivés" },
32 + { "slug": "llm-agents", "name_en": "LLM agents", "name_fr": "Agents LLM" },
33 + { "slug": "inference-systems", "name_en": "Inference systems", "name_fr": "Systèmes d'inférence" },
34 + { "slug": "multi-agent-systems", "name_en": "Multi-agent systems", "name_fr": "Systèmes multi-agents" },
35 + { "slug": "corporate-disclosure", "name_en": "Corporate disclosure", "name_fr": "Divulgation d'entreprise" },
36 + { "slug": "retail-prices", "name_en": "Retail prices and cost of living", "name_fr": "Prix de détail et coût de la vie" },
37 + { "slug": "regional-economics", "name_en": "Regional economics", "name_fr": "Économie régionale" },
38 + { "slug": "used-car-markets", "name_en": "Used-car markets", "name_fr": "Marchés de voitures d'occasion" },
39 + { "slug": "social-complexity", "name_en": "Social complexity and institutions", "name_fr": "Complexité sociale et institutions" },
40 + { "slug": "cryptography", "name_en": "Cryptography", "name_fr": "Cryptographie" },
41 + { "slug": "combinatorics", "name_en": "Combinatorics and graph theory", "name_fr": "Combinatoire et théorie des graphes" },
42 + { "slug": "distributed-estimation", "name_en": "Distributed estimation", "name_fr": "Estimation distribuée" },
43 + { "slug": "research-methodology", "name_en": "Research methodology", "name_fr": "Méthodologie de la recherche" },
44 + { "slug": "general-equilibrium", "name_en": "General equilibrium", "name_fr": "Équilibre général" },
45 + { "slug": "text-as-data", "name_en": "Text as data", "name_fr": "Le texte comme donnée" },
46 + { "slug": "computer-vision", "name_en": "Computer vision", "name_fr": "Vision par ordinateur" },
47 + { "slug": "preregistered-research", "name_en": "Preregistered research", "name_fr": "Recherche préenregistrée" },
48 + { "slug": "quebec", "name_en": "Quebec evidence", "name_fr": "Données du Québec" },
49 + { "slug": "negative-results", "name_en": "Negative and null results", "name_fr": "Résultats négatifs et nuls" }
50 + ],
51 + "methods": [
52 + { "slug": "hedonic-regression", "name_en": "Hedonic regression", "name_fr": "Régression hédonique", "category": "econometrics" },
53 + { "slug": "quantile-regression", "name_en": "Quantile regression", "name_fr": "Régression quantile", "category": "econometrics" },
54 + { "slug": "spatial-econometrics", "name_en": "Spatial econometrics", "name_fr": "Économétrie spatiale", "category": "econometrics" },
55 + { "slug": "fixed-effects", "name_en": "High-dimensional fixed effects", "name_fr": "Effets fixes de grande dimension", "category": "econometrics" },
56 + { "slug": "gradient-boosting", "name_en": "Gradient boosting and machine-learning benchmarks", "name_fr": "Gradient boosting et bancs d'essai d'apprentissage automatique", "category": "machine-learning" },
57 + { "slug": "cluster-robust-inference", "name_en": "Cluster-robust and bootstrap inference", "name_fr": "Inférence robuste par grappes et bootstrap", "category": "inference" },
58 + { "slug": "monte-carlo", "name_en": "Monte Carlo simulation", "name_fr": "Simulation Monte Carlo", "category": "computation" },
59 + { "slug": "state-space-models", "name_en": "State-space models", "name_fr": "Modèles espace-état", "category": "econometrics" },
60 + { "slug": "event-study", "name_en": "Event study", "name_fr": "Étude d'événement", "category": "econometrics" },
61 + { "slug": "permutation-tests", "name_en": "Permutation and placebo tests", "name_fr": "Tests de permutation et placebos", "category": "inference" },
62 + { "slug": "preregistration", "name_en": "Preregistration with frozen protocol", "name_fr": "Préenregistrement avec protocole gelé", "category": "design" },
63 + { "slug": "agent-based-simulation", "name_en": "Agent-based simulation", "name_fr": "Simulation à base d'agents", "category": "computation" },
64 + { "slug": "llm-extraction", "name_en": "Schema-constrained LLM extraction", "name_fr": "Extraction par LLM à schéma contraint", "category": "machine-learning" },
65 + { "slug": "sentence-embeddings", "name_en": "Sentence embeddings", "name_fr": "Plongements de phrases", "category": "machine-learning" },
66 + { "slug": "meta-analysis", "name_en": "Meta-analysis and meta-regression", "name_fr": "Méta-analyse et méta-régression", "category": "econometrics" },
67 + { "slug": "general-equilibrium-modelling", "name_en": "General-equilibrium modelling", "name_fr": "Modélisation d'équilibre général", "category": "theory" },
68 + { "slug": "connectedness", "name_en": "Diebold–Yilmaz connectedness", "name_fr": "Connectivité de Diebold–Yilmaz", "category": "econometrics" },
69 + { "slug": "har-rv", "name_en": "HAR-RV volatility forecasting", "name_fr": "Prévision de volatilité HAR-RV", "category": "econometrics" },
70 + { "slug": "hazard-models", "name_en": "Hazard and duration models", "name_fr": "Modèles de risque et de durée", "category": "econometrics" },
71 + { "slug": "formal-verification", "name_en": "Formal verification (Lean 4, SAT certificates)", "name_fr": "Vérification formelle (Lean 4, certificats SAT)", "category": "computation" },
72 + { "slug": "randomized-experiments", "name_en": "Randomized controlled experiments", "name_fr": "Expériences contrôlées randomisées", "category": "design" },
73 + { "slug": "variance-decomposition", "name_en": "Shapley variance decomposition", "name_fr": "Décomposition de variance de Shapley", "category": "econometrics" },
74 + { "slug": "exhaustive-enumeration", "name_en": "Exhaustive enumeration with symmetry reduction", "name_fr": "Énumération exhaustive avec réduction par symétrie", "category": "computation" },
75 + { "slug": "portfolio-sorts", "name_en": "Portfolio sorts and Fama–MacBeth regressions", "name_fr": "Tris de portefeuilles et régressions Fama–MacBeth", "category": "econometrics" },
76 + { "slug": "forecast-evaluation", "name_en": "Out-of-sample forecast evaluation (DM, MCS)", "name_fr": "Évaluation de prévisions hors échantillon (DM, MCS)", "category": "econometrics" },
77 + { "slug": "record-linkage", "name_en": "Record linkage and entity resolution", "name_fr": "Appariement d'enregistrements et résolution d'entités", "category": "data" },
78 + { "slug": "distributed-computation", "name_en": "Distributed computation on the MacLustr cluster", "name_fr": "Calcul distribué sur la grappe MacLustr", "category": "computation" }
79 + ],
80 + "datasets": [
81 + { "slug": "vrai-prix-transactions", "name_en": "Vrai Prix registered Quebec residential sales", "name_fr": "Ventes résidentielles enregistrées du Québec (Vrai Prix)", "provider": "Vrai Prix (land registry snapshot)", "coverage": "Quebec, 2021–2026", "frequency": "Transaction-level", "access_conditions": "Research copy; not redistributed", "is_public": false },
82 + { "slug": "quebec-assessment-rolls", "name_en": "Quebec municipal assessment rolls", "name_fr": "Rôles d'évaluation municipaux du Québec", "provider": "Quebec municipalities (public rolls)", "coverage": "Quebec", "frequency": "Triennial rolls", "access_conditions": "Public records, matched by the authors", "is_public": true },
83 + { "slug": "immo-ka-listings", "name_en": "Immo-Ka residential listings", "name_fr": "Annonces résidentielles Immo-Ka", "provider": "Immo-Ka aggregator", "coverage": "Quebec", "frequency": "Listing snapshots", "access_conditions": "Owner-authorised research copy", "is_public": false },
84 + { "slug": "centris-listings", "name_en": "Centris active-listing snapshot", "name_fr": "Instantané d'annonces actives Centris", "provider": "Centris", "coverage": "Quebec", "frequency": "Snapshot", "access_conditions": "Snapshot not redistributed", "is_public": false },
85 + { "slug": "zillow-listings", "name_en": "Zillow active listings (United States)", "name_fr": "Annonces actives Zillow (États-Unis)", "provider": "Zillow", "coverage": "United States", "frequency": "Snapshot", "access_conditions": "Not redistributed", "is_public": false },
86 + { "slug": "airbnb-quebec", "name_en": "Airbnb listings, Quebec", "name_fr": "Annonces Airbnb, Québec", "provider": "Airbnb (search-card scrape)", "coverage": "Quebec", "frequency": "Snapshot", "access_conditions": "Scraped; not redistributed", "is_public": false },
87 + { "slug": "canadian-mls-listings", "name_en": "Canadian MLS listings", "name_fr": "Annonces MLS canadiennes", "provider": "MLS (public listing pages)", "coverage": "Canada", "frequency": "Snapshot", "access_conditions": "Not redistributed", "is_public": false },
88 + { "slug": "firstrate-data", "name_en": "FirstRate Data intraday bars", "name_fr": "Barres intrajournalières FirstRate Data", "provider": "FirstRate Data", "coverage": "US equities, ETFs, futures, FX, crypto", "frequency": "1-minute and 5-minute", "access_conditions": "Commercial licence", "url": "https://firstratedata.com", "is_public": false },
89 + { "slug": "hf-market-data", "name_en": "HF Market Data 1-minute bars", "name_fr": "Barres 1 minute HF Market Data", "provider": "HF Market Data (research lake)", "coverage": "25 instruments across asset classes", "frequency": "1-minute", "access_conditions": "Research access", "is_public": false },
90 + { "slug": "polymarket", "name_en": "Polymarket negative-risk markets", "name_fr": "Marchés à risque négatif Polymarket", "provider": "Polymarket (public API)", "coverage": "Macro-event markets", "frequency": "Intraday", "access_conditions": "Public API", "url": "https://polymarket.com", "is_public": true },
91 + { "slug": "uqo-edgar-research-db", "name_en": "UQO EDGAR Research Database (Russell 3000)", "name_fr": "Base de recherche EDGAR de l'UQO (Russell 3000)", "provider": "SEC EDGAR, processed at UQO", "coverage": "Russell 3000 filings", "frequency": "Filing-level", "access_conditions": "Derived from public filings", "is_public": false },
92 + { "slug": "spid", "name_en": "SPID — SEC Project Intelligence Database", "name_fr": "SPID — base d'intelligence des projets SEC", "provider": "UQO (FIRC project)", "coverage": "Russell 3000 strategic projects", "frequency": "Project-level", "access_conditions": "Research database", "url": "https://www.pdb-api.co", "is_public": false },
93 + { "slug": "fabri-ka-catalogue", "name_en": "Fabri-Ka online store catalogue snapshot", "name_fr": "Instantané des catalogues de boutiques en ligne Fabri-Ka", "provider": "Fabri-Ka aggregator", "coverage": "1,569 Quebec online stores, 17 regions", "frequency": "Snapshot (2026-11-01)", "access_conditions": "Owner-authorised read-only copy; not redistributed", "is_public": false },
94 + { "slug": "food-ka-prices", "name_en": "Food-Ka grocery price feeds", "name_fr": "Flux de prix d'épicerie Food-Ka", "provider": "Food-Ka aggregator", "coverage": "54 Quebec grocery feeds, Aug–Nov 2026", "frequency": "Posted-price events", "access_conditions": "Research copy", "is_public": false },
95 + { "slug": "auto-ka-panel", "name_en": "Auto-Ka daily used-car listing panel", "name_fr": "Panel quotidien d'annonces de voitures d'occasion Auto-Ka", "provider": "Auto-Ka aggregator", "coverage": "273,265 Quebec listings, Aug–Nov 2026", "frequency": "Daily", "access_conditions": "Research copy", "is_public": false },
96 + { "slug": "seshat", "name_en": "Seshat Global History Databank (Polaris-2026)", "name_fr": "Seshat Global History Databank (Polaris-2026)", "provider": "Seshat", "coverage": "Historical polities", "frequency": "Polity-century", "access_conditions": "Public release", "url": "https://seshatdatabank.info", "is_public": true },
97 + { "slug": "d-place-ethnographic-atlas", "name_en": "D-PLACE Ethnographic Atlas", "name_fr": "Atlas ethnographique D-PLACE", "provider": "D-PLACE", "coverage": "Ethnographic societies", "frequency": "Cross-section", "access_conditions": "Public", "url": "https://d-place.org", "is_public": true },
98 + { "slug": "vdem-maddison", "name_en": "V-Dem and Maddison Project panel", "name_fr": "Panel V-Dem et Maddison Project", "provider": "V-Dem Institute; Maddison Project", "coverage": "Countries, modern era", "frequency": "Country-year", "access_conditions": "Public", "is_public": true },
99 + { "slug": "statistics-canada", "name_en": "Statistics Canada reference series", "name_fr": "Séries de référence de Statistique Canada", "provider": "Statistics Canada", "coverage": "Canada / Quebec", "frequency": "Monthly", "access_conditions": "Public (open licence)", "url": "https://www.statcan.gc.ca", "is_public": true },
100 + { "slug": "fineweb-edu", "name_en": "FineWeb-Edu subset", "name_fr": "Sous-ensemble FineWeb-Edu", "provider": "Hugging Face", "coverage": "Web text, educational filter", "frequency": "Static", "access_conditions": "Public", "is_public": true },
101 + { "slug": "longbench", "name_en": "LongBench QA subsets", "name_fr": "Sous-ensembles QA LongBench", "provider": "THUDM", "coverage": "Long-context QA", "frequency": "Static", "access_conditions": "Public", "is_public": true },
102 + { "slug": "synthetic-generated", "name_en": "Generated instances with recorded provenance (no external data)", "name_fr": "Instances générées avec provenance enregistrée (aucune donnée externe)", "provider": "Authors", "coverage": "As described in each paper", "frequency": "n/a", "access_conditions": "Released with the code", "is_public": true }
103 + ],
104 + "domain_of": {
105 + "wp1": "real-estate-economics", "wp2": "real-estate-economics", "wp3": "real-estate-economics", "wp4": "real-estate-economics", "wp5": "real-estate-economics", "wp9": "real-estate-economics", "wp10": "real-estate-economics", "wp11": "real-estate-economics", "wp14": "real-estate-economics", "wp16": "real-estate-economics", "wp18": "real-estate-economics", "wp19": "real-estate-economics", "wp20": "real-estate-economics", "wp21": "real-estate-economics", "wp22": "real-estate-economics", "wp23": "real-estate-economics", "wp26": "real-estate-economics",
106 + "wp6": "high-frequency-econometrics", "wp12": "high-frequency-econometrics", "wp7": "financial-economics", "wp8": "financial-economics", "wp15": "market-microstructure", "wp27": "market-microstructure", "wp13": "artificial-intelligence",
107 + "wp24": "computational-economics", "wp25": "computational-economics", "wp28": "scientific-computing", "wp32": "scientific-computing", "wp34": "scientific-computing",
108 + "wp29": "interdisciplinary-research", "wp35": "interdisciplinary-research", "wp30": "artificial-intelligence", "wp31": "artificial-intelligence", "wp33": "artificial-intelligence",
109 + "wp36": "applied-econometrics", "wp37": "applied-econometrics", "wp38": "applied-econometrics",
110 + "firc1": "financial-economics", "firc2": "financial-economics"
111 + },
112 + "secondary_domains": { "wp13": ["financial-economics"], "wp26": ["artificial-intelligence"], "wp27": ["artificial-intelligence", "computational-economics"], "wp25": ["artificial-intelligence", "financial-economics"], "wp14": ["artificial-intelligence"], "wp20": ["artificial-intelligence"], "wp2": ["artificial-intelligence"], "wp7": ["high-frequency-econometrics"], "wp15": ["high-frequency-econometrics"], "wp29": ["artificial-intelligence"], "wp28": ["applied-econometrics"], "wp30": ["scientific-computing"], "wp33": ["scientific-computing"], "firc1": ["artificial-intelligence", "corporate-disclosure"], "firc2": ["artificial-intelligence"], "wp12": ["financial-economics"], "wp6": ["financial-economics"], "wp8": ["market-microstructure"], "wp36": ["market-microstructure"], "wp35": ["computational-economics"], "wp24": ["financial-economics"], "wp23": ["applied-econometrics"], "wp22": ["applied-econometrics"], "wp21": ["applied-econometrics"], "wp38": ["regional-economics"] },
113 + "topics_of": {
114 + "hedonic-pricing": ["wp1", "wp2", "wp3", "wp4", "wp5", "wp9", "wp11", "wp14", "wp16", "wp21", "wp22", "wp36"],
115 + "automated-valuation": ["wp3", "wp9", "wp11", "wp14", "wp16", "wp20", "wp23", "wp26"],
116 + "housing-price-indices": ["wp11", "wp22", "wp23"],
117 + "property-taxation": ["wp10", "wp16", "wp20"],
118 + "short-term-rentals": ["wp4", "wp5"],
119 + "housing-theory": ["wp18", "wp19"],
120 + "volatility": ["wp6", "wp7", "wp12", "wp15"],
121 + "prediction-markets": ["wp8"],
122 + "options-and-derivatives": ["wp7"],
123 + "llm-agents": ["wp13", "wp25", "wp26", "wp27", "wp29"],
124 + "inference-systems": ["wp30", "wp31", "wp33"],
125 + "multi-agent-systems": ["wp26", "wp27", "wp30"],
126 + "corporate-disclosure": ["firc1", "firc2"],
127 + "retail-prices": ["wp37", "wp38"],
128 + "regional-economics": ["wp38"],
129 + "used-car-markets": ["wp36"],
130 + "social-complexity": ["wp35"],
131 + "cryptography": ["wp34"],
132 + "combinatorics": ["wp32"],
133 + "distributed-estimation": ["wp28"],
134 + "research-methodology": ["wp13", "wp29", "wp32", "wp34"],
135 + "general-equilibrium": ["wp18", "wp24", "wp25"],
136 + "text-as-data": ["wp2", "wp14", "firc1", "firc2"],
137 + "computer-vision": ["wp14", "wp20"],
138 + "preregistered-research": ["wp16", "wp19", "wp22", "wp25", "wp26", "wp29", "wp30", "wp31", "wp32", "wp33", "wp35", "wp36", "wp37", "wp38"],
139 + "quebec": ["wp1", "wp2", "wp4", "wp5", "wp10", "wp11", "wp14", "wp16", "wp20", "wp22", "wp23", "wp26", "wp36", "wp37", "wp38"],
140 + "negative-results": ["wp26", "wp31", "wp33", "wp34", "wp35", "wp38"]
141 + },
142 + "methods_of": {
143 + "hedonic-regression": ["wp1", "wp2", "wp3", "wp4", "wp5", "wp9", "wp11", "wp14", "wp16", "wp22", "wp36"],
144 + "quantile-regression": ["wp1", "wp3", "wp4", "wp5", "wp10"],
145 + "spatial-econometrics": ["wp1", "wp4", "wp5", "wp9"],
146 + "fixed-effects": ["wp5", "wp9", "wp10", "wp14"],
147 + "gradient-boosting": ["wp3", "wp4", "wp5", "wp11", "wp12", "wp23"],
148 + "cluster-robust-inference": ["wp1", "wp3", "wp11", "wp16", "wp36", "wp38"],
149 + "monte-carlo": ["wp19", "wp22", "wp23", "wp28"],
150 + "state-space-models": ["wp22", "wp23"],
151 + "event-study": ["wp8", "wp36", "firc2"],
152 + "permutation-tests": ["wp15", "wp36", "wp37", "wp38"],
153 + "preregistration": ["wp16", "wp19", "wp22", "wp25", "wp26", "wp29", "wp30", "wp31", "wp32", "wp33", "wp35", "wp36", "wp37", "wp38"],
154 + "agent-based-simulation": ["wp25", "wp27", "wp35"],
155 + "llm-extraction": ["wp14", "wp20", "firc1"],
156 + "sentence-embeddings": ["wp2", "firc2"],
157 + "meta-analysis": ["wp21"],
158 + "general-equilibrium-modelling": ["wp18", "wp24", "wp25"],
159 + "connectedness": ["wp6"],
160 + "har-rv": ["wp7", "wp12"],
161 + "hazard-models": ["wp36"],
162 + "formal-verification": ["wp28", "wp32", "wp34"],
163 + "randomized-experiments": ["wp26", "wp27", "wp29"],
164 + "variance-decomposition": ["wp9"],
165 + "exhaustive-enumeration": ["wp32"],
166 + "portfolio-sorts": ["wp7", "firc2"],
167 + "forecast-evaluation": ["wp12", "wp23"],
168 + "record-linkage": ["wp36", "wp37", "firc1"],
169 + "distributed-computation": ["wp28", "wp30", "wp33", "wp34", "firc1"]
170 + },
171 + "datasets_of": {
172 + "vrai-prix-transactions": ["wp16", "wp22", "wp23", "wp26"],
173 + "quebec-assessment-rolls": ["wp10", "wp16", "wp20"],
174 + "immo-ka-listings": ["wp20"],
175 + "centris-listings": ["wp1"],
176 + "zillow-listings": ["wp3"],
177 + "airbnb-quebec": ["wp4", "wp5"],
178 + "canadian-mls-listings": ["wp9"],
179 + "firstrate-data": ["wp6"],
180 + "hf-market-data": ["wp12"],
181 + "polymarket": ["wp8"],
182 + "uqo-edgar-research-db": ["firc2"],
183 + "spid": ["firc1", "firc2"],
184 + "fabri-ka-catalogue": ["wp38"],
185 + "food-ka-prices": ["wp37"],
186 + "auto-ka-panel": ["wp36"],
187 + "seshat": ["wp35"],
188 + "d-place-ethnographic-atlas": ["wp35"],
189 + "vdem-maddison": ["wp35"],
190 + "statistics-canada": ["wp37"],
191 + "fineweb-edu": ["wp31"],
192 + "longbench": ["wp30"],
193 + "synthetic-generated": ["wp28", "wp32", "wp34"]
194 + },
195 + "featured": ["wp38", "wp26", "wp28", "wp30"],
196 + "reserved": [
197 + { "slug": "wp39", "code": "WP", "number": 39, "title": "The Invisible Balance Sheet", "internal_code": "wp39_uqo", "note": "In preparation (manuscript not yet compiled). Registered from the project repository on 2026-11-02; title from the LaTeX source." },
198 + { "slug": "wp40", "code": "WP", "number": 40, "title": "RAID — Research Architecture for Intraday Data: A Metal-Accelerated C++ Framework for Reproducible High-Frequency Financial Econometrics", "internal_code": "wp40_raid", "note": "In preparation (Phase 1, foundation; no manuscript yet). Registered from the project README on 2026-11-02." }
199 + ]
200 +}
added docs/api.md +15 −0
@@ -0,0 +1,15 @@
1 +# Public API
2 +
3 +Base URL `https://wp.spboucher.ai/api/v1` — read-only, no authentication, CORS `*`, 120 requests/min/IP (60 for `/search`). OpenAPI: `/api/v1/openapi.json`; human page: `/api/docs`.
4 +
5 +| Endpoint | Notes |
6 +|---|---|
7 +| `GET /papers` | filters `q, year, domain, topic, author, keyword, method, dataset, series, pub, pdf=1`, `sort=recent|revised|number|title|year|views|downloads`, `page`, `per_page` (≤100) |
8 +| `GET /papers/{id}` | id = `wp38`, `WP38`, `firc2`… full record incl. versions and related papers with explanations and `inferred` flags |
9 +| `GET /papers/{id}/citation.{bib|ris|json}` | citation exports |
10 +| `GET /search?q=&mode=hybrid|lexical|semantic&limit=` | fused retrieval with matched signals and highlighted passages |
11 +| `GET /stats` · `/topics` · `/methods` · `/datasets` · `/authors` · `/series` · `/timeline` · `/graph?kinds=` | discovery data |
12 +
13 +Other machine endpoints: `/api/oai` (OAI-PMH 2.0, `oai_dc`, sets = series), `/feed.xml` (RSS 2.0), `/sitemap.xml`, `/api/files/{fileId}/{name}.pdf` (Range, ETag = SHA-256, `X-Checksum-SHA256`), `/api/preview/{slug}.png`, `/api/health`.
14 +
15 +Every paper page embeds Highwire Press (`citation_*`), Dublin Core (`DC.*`) meta tags and JSON-LD `ScholarlyArticle`.
added docs/architecture.md +32 −0
@@ -0,0 +1,32 @@
1 +# Architecture
2 +
3 +```
4 +Browser ──HTTPS──▶ BHS64 Caddy ──wg1──▶ M3U96b :8560 wp-web (Next.js 16, PM2)
5 + │ └─ sqlPublic (role wp_reader → public_* views) / sql (role wp_app)
6 + │
7 + wp-worker (tsx, PM2) ── jobs table ── poppler, OpenAI embeddings (optional), pg_dump
8 + │
9 + PostgreSQL 17 wp_spboucher (pgvector, pg_trgm, unaccent) ~/data/wp-spboucher-ai (files, previews)
10 +```
11 +
12 +## Data model (drizzle/0001_init.sql)
13 +- `series` (WP, FIRC) · `papers` (permanent `identifier`/`slug`, `number`, status `draft|reserved|published|withdrawn|superseded`, publication status, dates with precision, structured sections as JSONB, keywords, JEL, generated `fts` tsvector and `search_blob`, `embedding vector(1536)`, `lexvec vector(512)`) · `paper_versions` (immutable history, `is_current`, `withdrawn`) · `files` (content-addressed, SHA-256, integrity flag, pdfinfo metadata) · `paper_chunks` (full text by page range, tsvector, embeddings).
14 +- Taxonomy: `authors`, `topics` (kind `domain|topic`), `methods`, `datasets` and join tables carrying `assigned_by = editor|author|inferred`.
15 +- Intelligence: `paper_relations` (`topic|method|dataset|author|semantic|lexical|curated`, weight, bilingual explanation, `inferred`).
16 +- Operations: `users`, `sessions`, `login_attempts`, `audit_log`, `settings`, `redirects`, `events` (one per visitor/day/paper, bot-flagged), `paper_stats_daily`, `search_log`, `ingest_candidates`, `jobs`, `backups`.
17 +- Public views `public_papers`, `public_paper_versions`, `public_paper_chunks`, `public_paper_relations`, `public_files`, `public_settings` are the only relations the read-only role may read, so unpublished records cannot leak through any public code path.
18 +
19 +## Request paths
20 +- Public pages are server components under `src/app/(public)/[locale]`; `src/proxy.ts` rewrites canonical EN URLs to `/en/...` internally and leaves `/fr/...` as is, so ISR caches never mix languages. `/en/...` requested directly is redirected (308) to the canonical URL.
21 +- Paper pages revalidate every 120 s; admin mutations call `revalidatePath` for the affected routes.
22 +- Search (`src/lib/papers/search.ts`): lexical candidates from `papers.fts` (weighted A–D) + trigram similarity on `search_blob` + full-text chunk hits with `ts_headline`, fused by reciprocal rank with semantic candidates (paper embedding and chunk embeddings) when embeddings are available. Every hit reports the signals that matched.
23 +- Downloads stream from disk with Range/ETag support; `events` counts one download per visitor per paper per day; `daily_stats` aggregates and updates counters.
24 +
25 +## Worker
26 +Polls `jobs` (SKIP LOCKED, retries with back-off, 30-min stale lock recovery). Schedules: daily stats 00:10, relations 03:00, backup 04:40 (node local time). `process_pdf` → integrity, metadata, preview, full text → chunks → lexical vector → `embed_paper` → `relations`.
27 +
28 +## Relations
29 +Deterministic links from shared topics, methods, datasets and non-primary co-authors (the series' primary author is on every paper and would add no information). Inferred links: top-4 lexical neighbours (hashed TF-IDF cosine ≥ 0.22) and top-4 semantic neighbours (embedding cosine ≥ 0.45). Curated links are added in `/admin` and never overwritten.
30 +
31 +## Ingestion
32 +`ingest_scan` walks the inbox (+ configured directories), checksums every PDF, stores new files, creates candidates; `ingest_candidate` runs integrity, pdfinfo, title-page heuristics (title, authors, abstract, keywords, JEL, WP number, version, month/year), preview and duplicate detection (checksum, identifier, title similarity, author overlap). The administrator approves (draft paper or new version) or rejects; nothing is published without an explicit status change.
added docs/infrastructure-report.md +20 −0
@@ -0,0 +1,20 @@
1 +# Infrastructure discovery and compatibility report — wp.spboucher.ai (2026-11-02)
2 +
3 +## What was found
4 +| Item | Finding |
5 +|---|---|
6 +| DNS | `wp.spboucher.ai` → `A 51.161.112.61` (BHS64 gateway) already configured at GoDaddy. No change made, no duplicate created. |
7 +| Public ingress | MacLustr Tunnel: WireGuard hub on BHS64 (`wg0` 10.67.0.1, UDP 443 fallback), Caddy with automatic TLS. Before deployment `https://wp.spboucher.ai` returned a TLS alert (no route). `mld deploy` adds the Caddy route `wp.spboucher.ai → M3U96b:8560` over `wg1`. |
8 +| Deployment convention | `mld` on M1M32 (manifest `~/dispatch/apps/<app>.json`, staging, rsync, PM2, health check, tunnel route, registry). Templates: `tools-spboucher-ai` (Next 16 + Postgres 17 on M2M32c) and `akos` (M4M64c). |
9 +| Nodes considered | M3U96b (28 c / 96 GB, Postgres 17.11, poppler, redis, 395 GB free, 2 apps), M4M64c (3 spboucher.ai apps already), M2M32c (tools), M2M32b (unreachable over Tailscale at discovery time). **Chosen: M3U96b** — isolation from other spboucher.ai apps, largest free disk for versioned manuscripts, Postgres already running. |
10 +| PostgreSQL | 17.11 Homebrew on M3U96b; `pgvector` was missing → installed `pgvector 0.8.7` with Homebrew; extensions `vector`, `pg_trgm`, `unaccent`, `pgcrypto` created in a dedicated DB `wp_spboucher` with dedicated roles `wp_app` (read/write) and `wp_reader` (read-only on `public_*` views). Local-socket trust for the superuser; scram over TCP for app roles. |
11 +| Storage | Local SSD, dedicated directory `~/data/wp-spboucher-ai/{files,previews,inbox,tmp}` (content-addressed by SHA-256). Backups `~/backups/wp-spboucher-ai` (encrypted) + rsync to M3U96a (LAN 192.168.2.80). No S3-compatible object store exists in the cluster; the layout is one `rclone`/`mc` sync away from one. |
12 +| Monitoring / logging | PM2 (`pm2 logs`, `pm2-logrotate`), `mld heal` every 5 min on M1M32 (restarts PM2/launchd/wg1/routes), structured JSON logs from the worker, `GET /api/health` used by mld. Admin dashboard shows jobs, backups, AI status. |
13 +| Authentication convention | Same pattern as tools.spboucher.ai: DB-backed sessions (`__Host-` cookie, HttpOnly, Strict), Argon2id, TOTP, CSRF double-submit, same-origin checks, login rate limiting, audit log. |
14 +| AI | OpenAI `text-embedding-3-small` (1536 dims) — model existence verified at runtime with `GET /v1/models/{id}` before first use; disabled automatically if the key or model is unavailable. No LLM generation is used for metadata. |
15 +
16 +## Isolation guarantees
17 +Dedicated DB, roles, PM2 process names (`wp-web`, `wp-worker`), port 8560 (free on M3U96b), storage and backup directories, `.env`. No change to other apps, other `spboucher.ai` subdomains, research databases or tunnel routes.
18 +
19 +## Source manuscripts
20 +The 39 public manuscripts were taken from the already-public catalog of www.spboucher.ai (M2U64 `~/spboucher-ai/public/papers`, 44 MB) together with their verified metadata (`lib/research.ts`, `lib/research-details.ts`, written from the PDFs on 2026-11-01). WP39 and WP40 exist only as drafts in `~/Desktop/UQO/Recherche/Recherche-UQO` (no compiled manuscript): registered as **reserved** identifiers, not public.
added docs/operations.md +44 −0
@@ -0,0 +1,44 @@
1 +# Operations runbook
2 +
3 +## Where
4 +- Node **M3U96b**, directory `~/apps/wp-spboucher-ai` (git clone, origin spbgit `wp-spboucher-ai.git`). PM2: `wp-web` (Next, :8560) and `wp-worker`.
5 +- Data: `~/data/wp-spboucher-ai` (files/previews/inbox/tmp). Backups: `~/backups/wp-spboucher-ai` (+ rsync `M3U96a:backups/wp-spboucher-ai`).
6 +- Secrets: `~/apps/wp-spboucher-ai/.env` (mode 600); laptop copy `~/Desktop/Cluster/secrets/wp-spboucher-ai/.env`; owner initial password in `owner-initial-password.txt` (must be changed at first login).
7 +- Registry / route: `mld status`, `mld tunnel route wp-spboucher-ai`, manifest `M1M32:~/dispatch/apps/wp-spboucher-ai.json` (copy in `infra/mld/`).
8 +
9 +## Everyday
10 +```bash
11 +ssh M3U96b
12 +cd ~/apps/wp-spboucher-ai && pm2 logs wp-web --lines 100 ; pm2 logs wp-worker --lines 100
13 +curl -s localhost:8560/api/health
14 +```
15 +Edit on the node → `pnpm build && pm2 restart wp-web wp-worker` → `git commit && git push` → laptop `git pull`.
16 +
17 +## Publishing a new working paper
18 +1. Put the PDF in `~/data/wp-spboucher-ai/inbox/` (or any directory listed in Settings → Ingestion) and click **Scan directories now** in `/admin/ingest` (or `pnpm ingest:scan`).
19 +2. Open the candidate: check integrity, duplicates, proposed title/authors/abstract/number/date; **Create draft paper** (or **Attach version** to an existing paper).
20 +3. Complete the record in `/admin/papers/<id>` (abstract paragraphs, findings, contributions, methodology, data, reproducibility, artifacts, keywords, JEL, topics, methods, datasets, primary domain).
21 +4. **Publish** from the status bar. The worker extracts the full text, builds previews, embeddings and relations. Verify the public page and `/api/v1/papers/<slug>`.
22 +
23 +Revisions: upload a new version on the paper page (kept immutable; previous versions remain downloadable unless withdrawn with a reason).
24 +
25 +## Schedules (worker, node local time)
26 +00:10 daily statistics · 03:00 relations rebuild · 04:40 encrypted backup (pg_dump + manuscripts; 14-day retention; rsync to M3U96a). Manual: `/admin/system` or `pnpm backup`. Restore drill: `pnpm restore:drill`.
27 +
28 +## Restore
29 +```bash
30 +openssl enc -d -aes-256-cbc -pbkdf2 -in ~/backups/wp-spboucher-ai/wp-<stamp>.dump.enc -out /tmp/wp.dump -pass env:BK # BK = BACKUP_PASSPHRASE
31 +createdb wp_spboucher_restored && pg_restore --no-owner -d wp_spboucher_restored /tmp/wp.dump
32 +openssl enc -d -aes-256-cbc -pbkdf2 -in ~/backups/wp-spboucher-ai/wp-files-<stamp>.tar.enc -out /tmp/files.tar -pass env:BK && tar -xf /tmp/files.tar -C ~/data/wp-spboucher-ai
33 +```
34 +
35 +## Rollback
36 +`mld deploy` keeps the previous release; `mld move`/`deploy` roll back automatically on failed health checks. Manual: `git checkout <sha> && pnpm build && pm2 restart wp-web wp-worker` on the node. Migrations are additive; never edit an applied migration (checksums are verified).
37 +
38 +## Security notes
39 +Admin: Argon2id passwords (policy ≥ 14 chars), TOTP, `__Host-` session cookie (HttpOnly/Strict/Secure), rotation every 12 h, idle 7 d / absolute 30 d, CSRF double-submit on every mutation, same-origin enforcement, login limiter (5 failures / 15 min, exponential lockout), audit log of every write. Public API: read-only role, 120 req/min per IP (60 for search/files). Uploads: PDF magic + poppler integrity, 150 MB max, content-addressed.
40 +
41 +## Known limits
42 +- Semantic search/relations depend on the optional OpenAI key; without it the platform falls back to lexical signals (labelled).
43 +- Rate limiting is in-process (single web process).
44 +- No official UQO logo is used (`site.institution.logo_authorized = false` until permission is confirmed).
added docs/verification.md +3 −0
@@ -0,0 +1,3 @@
1 +# Verification log
2 +
3 +Filled at each release. See the bottom of this file for the latest run.
added drizzle/0001_init.sql +279 −0
@@ -0,0 +1,279 @@
1 +-- wp.spboucher.ai — initial schema (PostgreSQL 17, pgvector, pg_trgm, unaccent, pgcrypto)
2 +CREATE EXTENSION IF NOT EXISTS vector;
3 +CREATE EXTENSION IF NOT EXISTS pg_trgm;
4 +CREATE EXTENSION IF NOT EXISTS unaccent;
5 +CREATE EXTENSION IF NOT EXISTS pgcrypto;
6 +
7 +-- Immutable helpers so that generated tsvector columns can use them.
8 +CREATE OR REPLACE FUNCTION f_unaccent(text) RETURNS text LANGUAGE sql IMMUTABLE PARALLEL SAFE STRICT
9 + AS $$ SELECT public.unaccent('public.unaccent', $1) $$;
10 +CREATE OR REPLACE FUNCTION jsonb_strings(j jsonb) RETURNS text LANGUAGE sql IMMUTABLE PARALLEL SAFE AS $$
11 + WITH RECURSIVE walk(v) AS (
12 + SELECT j
13 + UNION ALL
14 + SELECT CASE WHEN jsonb_typeof(v) = 'array' THEN e.value WHEN jsonb_typeof(v) = 'object' THEN o.value END
15 + FROM walk LEFT JOIN LATERAL jsonb_array_elements(CASE WHEN jsonb_typeof(v) = 'array' THEN v ELSE '[]'::jsonb END) e ON true
16 + LEFT JOIN LATERAL jsonb_each(CASE WHEN jsonb_typeof(v) = 'object' THEN v ELSE '{}'::jsonb END) o ON true
17 + WHERE jsonb_typeof(v) IN ('array', 'object')
18 + )
19 + SELECT coalesce(string_agg(v #>> '{}', ' '), '') FROM walk WHERE jsonb_typeof(v) = 'string'
20 +$$;
21 +CREATE OR REPLACE FUNCTION arr_text(a text[]) RETURNS text LANGUAGE sql IMMUTABLE PARALLEL SAFE AS $$ SELECT coalesce(array_to_string(a, ' '), '') $$;
22 +
23 +CREATE TABLE settings (key text PRIMARY KEY, value jsonb NOT NULL, updated_at timestamptz NOT NULL DEFAULT now());
24 +
25 +CREATE TABLE users (
26 + id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
27 + email text NOT NULL UNIQUE,
28 + display_name text NOT NULL,
29 + password_hash text NOT NULL,
30 + role text NOT NULL DEFAULT 'owner' CHECK (role IN ('owner', 'editor')),
31 + totp_secret text,
32 + totp_enabled boolean NOT NULL DEFAULT false,
33 + must_change_password boolean NOT NULL DEFAULT true,
34 + created_at timestamptz NOT NULL DEFAULT now(),
35 + last_login_at timestamptz
36 +);
37 +CREATE TABLE sessions (
38 + id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
39 + user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
40 + token_hash text NOT NULL UNIQUE,
41 + csrf_secret text NOT NULL,
42 + user_agent text, ip text,
43 + created_at timestamptz NOT NULL DEFAULT now(),
44 + last_seen_at timestamptz NOT NULL DEFAULT now(),
45 + expires_at timestamptz NOT NULL,
46 + absolute_expires_at timestamptz NOT NULL,
47 + rotated_at timestamptz NOT NULL DEFAULT now(),
48 + revoked_at timestamptz,
49 + mfa_passed boolean NOT NULL DEFAULT false
50 +);
51 +CREATE INDEX sessions_user_idx ON sessions(user_id);
52 +CREATE TABLE login_attempts (id bigserial PRIMARY KEY, ip text NOT NULL, email text, success boolean NOT NULL, created_at timestamptz NOT NULL DEFAULT now());
53 +CREATE INDEX login_attempts_ip_idx ON login_attempts(ip, created_at);
54 +CREATE INDEX login_attempts_email_idx ON login_attempts(email, created_at);
55 +
56 +CREATE TABLE audit_log (
57 + id bigserial PRIMARY KEY,
58 + actor_id uuid, actor_label text NOT NULL,
59 + action text NOT NULL, entity_type text NOT NULL, entity_id text,
60 + before jsonb, after jsonb, ip text,
61 + at timestamptz NOT NULL DEFAULT now()
62 +);
63 +CREATE INDEX audit_log_at_idx ON audit_log(at DESC);
64 +CREATE INDEX audit_log_entity_idx ON audit_log(entity_type, entity_id);
65 +
66 +CREATE TABLE series (
67 + id serial PRIMARY KEY, slug text NOT NULL UNIQUE, code text NOT NULL UNIQUE,
68 + name_en text NOT NULL, name_fr text NOT NULL, description_en text, description_fr text,
69 + issn text, sort int NOT NULL DEFAULT 0, is_active boolean NOT NULL DEFAULT true
70 +);
71 +CREATE TABLE authors (
72 + id serial PRIMARY KEY, slug text NOT NULL UNIQUE,
73 + given_name text NOT NULL, family_name text NOT NULL, display_name text NOT NULL,
74 + orcid text, email text, website text, github text, google_scholar text, ssrn_url text, repec text,
75 + affiliation_en text, affiliation_fr text, department_en text, department_fr text,
76 + bio_en text, bio_fr text, is_primary boolean NOT NULL DEFAULT false, sort int NOT NULL DEFAULT 0,
77 + created_at timestamptz NOT NULL DEFAULT now(), updated_at timestamptz NOT NULL DEFAULT now()
78 +);
79 +CREATE TABLE topics (
80 + id serial PRIMARY KEY, slug text NOT NULL UNIQUE,
81 + name_en text NOT NULL, name_fr text NOT NULL, description_en text, description_fr text,
82 + kind text NOT NULL DEFAULT 'topic' CHECK (kind IN ('domain', 'topic')),
83 + parent_id int REFERENCES topics(id) ON DELETE SET NULL,
84 + accent text, sort int NOT NULL DEFAULT 0, is_active boolean NOT NULL DEFAULT true
85 +);
86 +CREATE TABLE methods (
87 + id serial PRIMARY KEY, slug text NOT NULL UNIQUE,
88 + name_en text NOT NULL, name_fr text NOT NULL, description_en text, description_fr text,
89 + category text NOT NULL DEFAULT 'general', sort int NOT NULL DEFAULT 0
90 +);
91 +CREATE TABLE datasets (
92 + id serial PRIMARY KEY, slug text NOT NULL UNIQUE,
93 + name_en text NOT NULL, name_fr text NOT NULL, provider text, coverage text, frequency text,
94 + access_conditions text, license text, url text, description_en text, description_fr text,
95 + is_public boolean NOT NULL DEFAULT false, sort int NOT NULL DEFAULT 0
96 +);
97 +
98 +CREATE TABLE files (
99 + id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
100 + kind text NOT NULL CHECK (kind IN ('manuscript', 'supplement', 'preview', 'other')),
101 + storage_key text NOT NULL UNIQUE,
102 + original_name text, mime text NOT NULL, bytes bigint NOT NULL, sha256 text NOT NULL,
103 + pages int, pdf_meta jsonb, integrity_ok boolean, text_status text NOT NULL DEFAULT 'pending' CHECK (text_status IN ('pending', 'done', 'failed', 'skipped')),
104 + created_at timestamptz NOT NULL DEFAULT now()
105 +);
106 +CREATE INDEX files_sha_idx ON files(sha256);
107 +
108 +CREATE TABLE papers (
109 + id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
110 + series_id int NOT NULL REFERENCES series(id),
111 + number int NOT NULL,
112 + identifier text NOT NULL UNIQUE, -- permanent public identifier, e.g. WP38, FIRC0002
113 + slug text NOT NULL UNIQUE, -- URL slug, e.g. wp38
114 + internal_code text, -- internal project number / repository name (not proof of release)
115 + title text NOT NULL, subtitle text, title_fr text, subtitle_fr text,
116 + abstract jsonb NOT NULL DEFAULT '[]'::jsonb, -- array of paragraphs
117 + abstract_fr jsonb, structured_abstract jsonb,
118 + language text NOT NULL DEFAULT 'en',
119 + research_question text, research_question_fr text,
120 + hero_headline text, hero_subheadline text,
121 + contributions jsonb NOT NULL DEFAULT '[]'::jsonb, -- [{kind?, title, description}]
122 + methodology jsonb NOT NULL DEFAULT '[]'::jsonb, -- [{title, description}] (verified fields only)
123 + data_sections jsonb NOT NULL DEFAULT '[]'::jsonb, -- [{title, description}]
124 + findings jsonb NOT NULL DEFAULT '[]'::jsonb, -- [{value, label}]
125 + reproducibility jsonb NOT NULL DEFAULT '[]'::jsonb, -- [string]
126 + artifacts jsonb NOT NULL DEFAULT '[]'::jsonb, -- [{kind, label, url}]
127 + notice text, notice_fr text,
128 + keywords text[] NOT NULL DEFAULT '{}',
129 + jel text[] NOT NULL DEFAULT '{}',
130 + status text NOT NULL DEFAULT 'draft' CHECK (status IN ('draft', 'reserved', 'published', 'withdrawn', 'superseded')),
131 + publication_status text NOT NULL DEFAULT 'working_paper' CHECK (publication_status IN ('working_paper', 'under_review', 'forthcoming', 'published')),
132 + journal_ref text, doi text, ssrn_id text, external_links jsonb NOT NULL DEFAULT '[]'::jsonb,
133 + year int, first_released_on date, first_released_precision text NOT NULL DEFAULT 'month' CHECK (first_released_precision IN ('day', 'month', 'year')),
134 + last_revised_on date, last_revised_precision text NOT NULL DEFAULT 'month' CHECK (last_revised_precision IN ('day', 'month', 'year')),
135 + current_version_id uuid,
136 + featured boolean NOT NULL DEFAULT false, featured_rank int,
137 + primary_domain_id int REFERENCES topics(id) ON DELETE SET NULL,
138 + view_count bigint NOT NULL DEFAULT 0, download_count bigint NOT NULL DEFAULT 0,
139 + embedding vector(1536), lexvec vector(512), embedding_model text,
140 + fts tsvector GENERATED ALWAYS AS (
141 + setweight(to_tsvector('english', f_unaccent(coalesce(title, ''))), 'A') ||
142 + setweight(to_tsvector('english', f_unaccent(arr_text(keywords))), 'A') ||
143 + setweight(to_tsvector('english', f_unaccent(coalesce(subtitle, ''))), 'B') ||
144 + setweight(to_tsvector('english', f_unaccent(coalesce(research_question, ''))), 'B') ||
145 + setweight(to_tsvector('english', f_unaccent(jsonb_strings(abstract))), 'B') ||
146 + setweight(to_tsvector('english', f_unaccent(jsonb_strings(findings))), 'C') ||
147 + setweight(to_tsvector('english', f_unaccent(jsonb_strings(contributions))), 'C') ||
148 + setweight(to_tsvector('english', f_unaccent(jsonb_strings(methodology))), 'C') ||
149 + setweight(to_tsvector('english', f_unaccent(jsonb_strings(data_sections))), 'C') ||
150 + setweight(to_tsvector('english', f_unaccent(coalesce(identifier, '') || ' ' || arr_text(jel))), 'D')
151 + ) STORED,
152 + search_blob text GENERATED ALWAYS AS (f_unaccent(lower(coalesce(title, '') || ' ' || coalesce(subtitle, '') || ' ' || arr_text(keywords)))) STORED,
153 + created_at timestamptz NOT NULL DEFAULT now(), updated_at timestamptz NOT NULL DEFAULT now(), published_at timestamptz,
154 + UNIQUE (series_id, number)
155 +);
156 +CREATE INDEX papers_fts_idx ON papers USING gin(fts);
157 +CREATE INDEX papers_blob_trgm_idx ON papers USING gin(search_blob gin_trgm_ops);
158 +CREATE INDEX papers_status_idx ON papers(status, year DESC, number DESC);
159 +CREATE INDEX papers_keywords_idx ON papers USING gin(keywords);
160 +
161 +CREATE TABLE paper_authors (
162 + paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
163 + author_id int NOT NULL REFERENCES authors(id) ON DELETE RESTRICT,
164 + position int NOT NULL DEFAULT 1, affiliation_note text,
165 + PRIMARY KEY (paper_id, author_id)
166 +);
167 +CREATE INDEX paper_authors_author_idx ON paper_authors(author_id);
168 +CREATE TABLE paper_topics (
169 + paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
170 + topic_id int NOT NULL REFERENCES topics(id) ON DELETE CASCADE,
171 + assigned_by text NOT NULL DEFAULT 'editor' CHECK (assigned_by IN ('editor', 'author', 'inferred')),
172 + PRIMARY KEY (paper_id, topic_id)
173 +);
174 +CREATE INDEX paper_topics_topic_idx ON paper_topics(topic_id);
175 +CREATE TABLE paper_methods (
176 + paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
177 + method_id int NOT NULL REFERENCES methods(id) ON DELETE CASCADE,
178 + note text, assigned_by text NOT NULL DEFAULT 'editor' CHECK (assigned_by IN ('editor', 'author', 'inferred')),
179 + PRIMARY KEY (paper_id, method_id)
180 +);
181 +CREATE INDEX paper_methods_method_idx ON paper_methods(method_id);
182 +CREATE TABLE paper_datasets (
183 + paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
184 + dataset_id int NOT NULL REFERENCES datasets(id) ON DELETE CASCADE,
185 + role text, note text, assigned_by text NOT NULL DEFAULT 'editor' CHECK (assigned_by IN ('editor', 'author', 'inferred')),
186 + PRIMARY KEY (paper_id, dataset_id)
187 +);
188 +CREATE INDEX paper_datasets_dataset_idx ON paper_datasets(dataset_id);
189 +
190 +CREATE TABLE paper_versions (
191 + id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
192 + paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
193 + version_no int NOT NULL, label text NOT NULL,
194 + released_on date, released_precision text NOT NULL DEFAULT 'month' CHECK (released_precision IN ('day', 'month', 'year')),
195 + description text, description_fr text,
196 + file_id uuid REFERENCES files(id) ON DELETE SET NULL,
197 + pages int, sha256 text, bytes bigint,
198 + is_current boolean NOT NULL DEFAULT false,
199 + withdrawn boolean NOT NULL DEFAULT false, withdrawn_reason text,
200 + created_at timestamptz NOT NULL DEFAULT now(),
201 + UNIQUE (paper_id, version_no)
202 +);
203 +CREATE INDEX paper_versions_paper_idx ON paper_versions(paper_id, version_no DESC);
204 +ALTER TABLE papers ADD CONSTRAINT papers_current_version_fk FOREIGN KEY (current_version_id) REFERENCES paper_versions(id) ON DELETE SET NULL DEFERRABLE INITIALLY DEFERRED;
205 +
206 +CREATE TABLE paper_chunks (
207 + id bigserial PRIMARY KEY,
208 + paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
209 + version_id uuid REFERENCES paper_versions(id) ON DELETE CASCADE,
210 + chunk_index int NOT NULL, page_from int, page_to int,
211 + text text NOT NULL,
212 + fts tsvector GENERATED ALWAYS AS (to_tsvector('english', f_unaccent(text))) STORED,
213 + embedding vector(1536)
214 +);
215 +CREATE INDEX paper_chunks_paper_idx ON paper_chunks(paper_id, chunk_index);
216 +CREATE INDEX paper_chunks_fts_idx ON paper_chunks USING gin(fts);
217 +
218 +CREATE TABLE paper_relations (
219 + id bigserial PRIMARY KEY,
220 + from_paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
221 + to_paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
222 + kind text NOT NULL CHECK (kind IN ('topic', 'method', 'dataset', 'author', 'semantic', 'lexical', 'curated')),
223 + weight real NOT NULL DEFAULT 1, explanation text, explanation_fr text,
224 + inferred boolean NOT NULL DEFAULT false,
225 + created_at timestamptz NOT NULL DEFAULT now(),
226 + UNIQUE (from_paper_id, to_paper_id, kind), CHECK (from_paper_id <> to_paper_id)
227 +);
228 +CREATE INDEX paper_relations_from_idx ON paper_relations(from_paper_id, weight DESC);
229 +
230 +CREATE TABLE redirects (from_path text PRIMARY KEY, to_path text NOT NULL, status int NOT NULL DEFAULT 301, note text, created_at timestamptz NOT NULL DEFAULT now());
231 +
232 +CREATE TABLE events (
233 + id bigserial PRIMARY KEY,
234 + paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE,
235 + version_id uuid, kind text NOT NULL CHECK (kind IN ('view', 'download')),
236 + occurred_at timestamptz NOT NULL DEFAULT now(), day date NOT NULL DEFAULT current_date,
237 + visitor_hash text NOT NULL, referrer_host text, locale text, is_bot boolean NOT NULL DEFAULT false
238 +);
239 +CREATE INDEX events_day_idx ON events(day, paper_id);
240 +CREATE UNIQUE INDEX events_dedupe_idx ON events(paper_id, kind, day, visitor_hash);
241 +CREATE TABLE paper_stats_daily (paper_id uuid NOT NULL REFERENCES papers(id) ON DELETE CASCADE, day date NOT NULL, views int NOT NULL DEFAULT 0, downloads int NOT NULL DEFAULT 0, PRIMARY KEY (paper_id, day));
242 +
243 +CREATE TABLE search_log (id bigserial PRIMARY KEY, query text NOT NULL, mode text NOT NULL, results int NOT NULL, locale text, took_ms int, at timestamptz NOT NULL DEFAULT now());
244 +CREATE INDEX search_log_at_idx ON search_log(at DESC);
245 +
246 +CREATE TABLE ingest_candidates (
247 + id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
248 + source_path text NOT NULL, file_id uuid REFERENCES files(id) ON DELETE SET NULL, sha256 text NOT NULL,
249 + proposal jsonb NOT NULL DEFAULT '{}'::jsonb, flags jsonb NOT NULL DEFAULT '[]'::jsonb, duplicates jsonb NOT NULL DEFAULT '[]'::jsonb,
250 + status text NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'processing', 'ready', 'approved', 'rejected', 'merged')),
251 + paper_id uuid REFERENCES papers(id) ON DELETE SET NULL,
252 + reviewed_by uuid, reviewed_at timestamptz, review_note text,
253 + created_at timestamptz NOT NULL DEFAULT now(), updated_at timestamptz NOT NULL DEFAULT now()
254 +);
255 +CREATE UNIQUE INDEX ingest_candidates_sha_idx ON ingest_candidates(sha256) WHERE status IN ('queued', 'processing', 'ready');
256 +
257 +CREATE TABLE jobs (
258 + id bigserial PRIMARY KEY, kind text NOT NULL, payload jsonb NOT NULL DEFAULT '{}'::jsonb,
259 + status text NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'running', 'done', 'failed')),
260 + attempts int NOT NULL DEFAULT 0, max_attempts int NOT NULL DEFAULT 3,
261 + run_after timestamptz NOT NULL DEFAULT now(), locked_at timestamptz, last_error text,
262 + created_at timestamptz NOT NULL DEFAULT now(), finished_at timestamptz
263 +);
264 +CREATE INDEX jobs_queue_idx ON jobs(status, run_after) WHERE status IN ('queued', 'running');
265 +
266 +CREATE TABLE backups (id bigserial PRIMARY KEY, kind text NOT NULL, path text NOT NULL, bytes bigint, sha256 text, ok boolean NOT NULL, log text, remote_ok boolean, created_at timestamptz NOT NULL DEFAULT now());
267 +
268 +-- Public views: everything the read-only role may see.
269 +CREATE VIEW public_papers AS SELECT * FROM papers WHERE status IN ('published', 'withdrawn', 'superseded');
270 +CREATE VIEW public_paper_versions AS SELECT v.* FROM paper_versions v JOIN papers p ON p.id = v.paper_id WHERE p.status IN ('published', 'withdrawn', 'superseded');
271 +CREATE VIEW public_paper_chunks AS SELECT c.id, c.paper_id, c.version_id, c.chunk_index, c.page_from, c.page_to, c.text, c.fts, c.embedding FROM paper_chunks c JOIN papers p ON p.id = c.paper_id WHERE p.status = 'published';
272 +CREATE VIEW public_paper_relations AS SELECT r.* FROM paper_relations r JOIN papers a ON a.id = r.from_paper_id JOIN papers b ON b.id = r.to_paper_id WHERE a.status = 'published' AND b.status = 'published';
273 +CREATE VIEW public_settings AS SELECT key, value FROM settings WHERE key NOT LIKE 'private.%';
274 +CREATE VIEW public_files AS SELECT f.id, f.kind, f.storage_key, f.original_name, f.mime, f.bytes, f.sha256, f.pages, f.integrity_ok FROM files f WHERE EXISTS (SELECT 1 FROM paper_versions v JOIN papers p ON p.id = v.paper_id WHERE v.file_id = f.id AND p.status IN ('published', 'withdrawn', 'superseded') AND NOT v.withdrawn) OR f.kind = 'preview';
275 +
276 +CREATE OR REPLACE FUNCTION touch_updated_at() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN NEW.updated_at = now(); RETURN NEW; END $$;
277 +CREATE TRIGGER papers_touch BEFORE UPDATE ON papers FOR EACH ROW EXECUTE FUNCTION touch_updated_at();
278 +CREATE TRIGGER authors_touch BEFORE UPDATE ON authors FOR EACH ROW EXECUTE FUNCTION touch_updated_at();
279 +CREATE TRIGGER ingest_touch BEFORE UPDATE ON ingest_candidates FOR EACH ROW EXECUTE FUNCTION touch_updated_at();
added drizzle/0002_roles.sql +17 −0
@@ -0,0 +1,17 @@
1 +-- Privileges for the application role (read/write on everything except role management) and the read-only public role.
2 +DO $$ BEGIN
3 + IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'wp_app') THEN
4 + GRANT USAGE ON SCHEMA public TO wp_app;
5 + GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO wp_app;
6 + GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO wp_app;
7 + ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO wp_app;
8 + ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT USAGE, SELECT ON SEQUENCES TO wp_app;
9 + END IF;
10 + IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'wp_reader') THEN
11 + GRANT USAGE ON SCHEMA public TO wp_reader;
12 + GRANT SELECT ON series, authors, topics, methods, datasets, paper_authors, paper_topics, paper_methods, paper_datasets, redirects, paper_stats_daily TO wp_reader;
13 + GRANT SELECT ON public_papers, public_paper_versions, public_paper_chunks, public_paper_relations, public_settings, public_files TO wp_reader;
14 + GRANT INSERT ON events, search_log TO wp_reader;
15 + GRANT USAGE, SELECT ON SEQUENCE events_id_seq, search_log_id_seq TO wp_reader;
16 + END IF;
17 +END $$;
added infra/mld/wp-spboucher-ai.json +34 −0
@@ -0,0 +1,34 @@
1 +{
2 + "app": "wp-spboucher-ai",
3 + "label": "Research Working Papers — Simon-Pierre Boucher Research Observatory (wp.spboucher.ai)",
4 + "domain": "wp.spboucher.ai",
5 + "port": 8560,
6 + "health_path": "/api/health",
7 + "dir": "~/apps/wp-spboucher-ai",
8 + "extra_paths": [],
9 + "sync_excludes": [".git/", ".DS_Store", "node_modules/", ".next/", ".env", ".env.*", "test-results/", "playwright-report/", "tsconfig.tsbuildinfo", "next-env.d.ts", "docs/_verification-*.json"],
10 + "requires": { "runtimes": ["node", "pnpm", "pm2", "postgresql@17"], "ram_gb": 3, "ports": [8560] },
11 + "ram_mb_observed": 700,
12 + "size_mb": 25,
13 + "placement": { "pin": "M3U96b", "prefer": "M3U96b", "avoid": ["M1M32"], "reason": "PostgreSQL 17.11 + pgvector 0.8.7 + poppler installed and initialised on M3U96b (DB wp_spboucher, roles wp_app / wp_reader); 395 GB free for versioned manuscripts; only 2 other apps (toveli, autoscholar); wg1 peer of BHS64" },
14 + "processes": [
15 + { "name": "wp-web", "manager": "pm2", "script": "/opt/homebrew/bin/node", "args": ["node_modules/next/dist/bin/next", "start", "-p", "8560", "-H", "0.0.0.0"], "interpreter": null, "cwd": "{{HOME}}/apps/wp-spboucher-ai", "env": { "NODE_ENV": "production", "PORT": "8560", "NEXT_TELEMETRY_DISABLED": "1", "PATH": "/opt/homebrew/opt/postgresql@17/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin" }, "cron_restart": null, "autorestart": true, "max_memory_restart": "1500M", "kill_timeout": 8000 },
16 + { "name": "wp-worker", "manager": "pm2", "script": "/opt/homebrew/bin/node", "args": ["node_modules/tsx/dist/cli.mjs", "src/worker/index.ts"], "interpreter": null, "cwd": "{{HOME}}/apps/wp-spboucher-ai", "env": { "NODE_ENV": "production", "NEXT_TELEMETRY_DISABLED": "1", "PATH": "/opt/homebrew/opt/postgresql@17/bin:/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin" }, "cron_restart": null, "autorestart": true, "max_memory_restart": "1G", "kill_timeout": 30000 }
17 + ],
18 + "ngrok": null,
19 + "launchd": [],
20 + "env_overrides": {},
21 + "hooks": {
22 + "post_sync": [
23 + "set -o pipefail; export PATH=\"/opt/homebrew/bin:$PATH\"; test -f .env || { echo 'ERREUR: .env absent (secrets) — copier ~/Desktop/Cluster/secrets/wp-spboucher-ai/.env'; exit 1; }; chmod 600 .env",
24 + "set -o pipefail; export PATH=\"/opt/homebrew/bin:$PATH\"; pnpm install --frozen-lockfile --silent 2>&1 | grep -v '^\\s*at ' | tail -2; test -d node_modules/next && echo ' deps ok'",
25 + "set -o pipefail; export PATH=\"/opt/homebrew/opt/postgresql@17/bin:/opt/homebrew/bin:$PATH\"; node_modules/.bin/tsx scripts/migrate.ts && node_modules/.bin/tsx scripts/db-roles.ts && node_modules/.bin/tsx scripts/bootstrap-owner.ts && echo ' db ok'",
26 + "set -o pipefail; export PATH=\"/opt/homebrew/bin:$PATH\"; rm -rf .next; NEXT_TELEMETRY_DISABLED=1 node_modules/.bin/next build 2>&1 | grep -vE 'Warning: Dynamic|turbopackIgnore|^\\s*$' | tail -3 && test -f .next/BUILD_ID && echo ' build ok'",
27 + "mkdir -p ~/data/wp-spboucher-ai/files ~/data/wp-spboucher-ai/previews ~/data/wp-spboucher-ai/inbox ~/data/wp-spboucher-ai/tmp ~/backups/wp-spboucher-ai && chmod 700 ~/data/wp-spboucher-ai ~/backups/wp-spboucher-ai && echo ' storage ok'"
28 + ],
29 + "post_start": []
30 + },
31 + "ka_repo": false,
32 + "tunnel": { "domain": "wp.spboucher.ai", "gateway": "BHS64", "note": "DNS GoDaddy A wp.spboucher.ai → 51.161.112.61 vérifié le 2026-11-02 (préexistant, aucune modification) ; route Caddy posée par mld deploy" },
33 + "notes": "Next.js 16 + PostgreSQL 17/pgvector local (DB wp_spboucher ; rôles wp_app lecture/écriture, wp_reader lecture seule sur les vues public_*) + worker tsx (PDF → texte/chunks/aperçus, embeddings OpenAI optionnels, relations, stats quotidiennes 00:10, sauvegardes chiffrées 04:40 → ~/backups/wp-spboucher-ai + rsync M3U96a). Manuscrits : ~/data/wp-spboucher-ai (hors dépôt, adressés par SHA-256). Secrets uniquement dans ~/apps/wp-spboucher-ai/.env (600) ; copie ~/Desktop/Cluster/secrets/wp-spboucher-ai/.env."
34 +}
added next.config.ts +44 −0
@@ -0,0 +1,44 @@
1 +import type { NextConfig } from "next";
2 +
3 +const isProd = process.env.NODE_ENV === "production";
4 +const csp = [
5 + "default-src 'self'",
6 + isProd ? "script-src 'self' 'unsafe-inline'" : "script-src 'self' 'unsafe-inline' 'unsafe-eval'",
7 + "style-src 'self' 'unsafe-inline'",
8 + "img-src 'self' data: blob:",
9 + "font-src 'self' data:",
10 + "connect-src 'self'",
11 + "frame-src 'self'",
12 + "object-src 'self'",
13 + "base-uri 'self'",
14 + "form-action 'self'",
15 + "frame-ancestors 'none'",
16 + "upgrade-insecure-requests",
17 +].join("; ");
18 +
19 +const securityHeaders = [
20 + { key: "X-Content-Type-Options", value: "nosniff" },
21 + { key: "X-Frame-Options", value: "DENY" },
22 + { key: "Referrer-Policy", value: "strict-origin-when-cross-origin" },
23 + { key: "Permissions-Policy", value: "camera=(), microphone=(), geolocation=(), browsing-topics=()" },
24 + { key: "Strict-Transport-Security", value: "max-age=63072000; includeSubDomains" },
25 + { key: "Cross-Origin-Opener-Policy", value: "same-origin" },
26 + { key: "Content-Security-Policy", value: csp },
27 +];
28 +
29 +const nextConfig: NextConfig = {
30 + poweredByHeader: false,
31 + reactStrictMode: true,
32 + serverExternalPackages: ["postgres", "@node-rs/argon2"],
33 + experimental: { serverActions: { bodySizeLimit: "2mb" } },
34 + async headers() {
35 + return [
36 + { source: "/(.*)", headers: securityHeaders },
37 + { source: "/admin/:path*", headers: [{ key: "X-Robots-Tag", value: "noindex, nofollow" }, { key: "Cache-Control", value: "no-store" }] },
38 + { source: "/login", headers: [{ key: "X-Robots-Tag", value: "noindex, nofollow" }, { key: "Cache-Control", value: "no-store" }] },
39 + { source: "/api/admin/:path*", headers: [{ key: "X-Robots-Tag", value: "noindex" }, { key: "Cache-Control", value: "no-store" }] },
40 + { source: "/api/v1/:path*", headers: [{ key: "Access-Control-Allow-Origin", value: "*" }, { key: "Access-Control-Allow-Methods", value: "GET, OPTIONS" }] },
41 + ];
42 + },
43 +};
44 +export default nextConfig;
added package.json +54 −0
@@ -0,0 +1,54 @@
1 +{
2 + "name": "wp-spboucher-ai",
3 + "version": "1.0.0",
4 + "private": true,
5 + "type": "module",
6 + "description": "Research Working Papers \u2014 Simon-Pierre Boucher Research Observatory (wp.spboucher.ai)",
7 + "scripts": {
8 + "dev": "next dev -p 8560",
9 + "build": "next build",
10 + "start": "next start -p 8560 -H 0.0.0.0",
11 + "lint": "eslint .",
12 + "typecheck": "tsc --noEmit",
13 + "db:migrate": "tsx scripts/migrate.ts",
14 + "db:roles": "tsx scripts/db-roles.ts",
15 + "bootstrap": "tsx scripts/bootstrap-owner.ts",
16 + "seed:import": "tsx scripts/seed-import.ts",
17 + "ingest:scan": "tsx scripts/ingest-scan.ts",
18 + "relations:rebuild": "tsx scripts/relations-rebuild.ts",
19 + "worker": "tsx src/worker/index.ts",
20 + "backup": "tsx scripts/backup.ts",
21 + "restore:drill": "tsx scripts/restore-drill.ts",
22 + "test": "vitest run",
23 + "test:e2e": "playwright test",
24 + "test:security": "tsx scripts/security-audit.ts",
25 + "verify:deploy": "tsx scripts/verify-deploy.ts"
26 + },
27 + "dependencies": {
28 + "@node-rs/argon2": "^2.2.2",
29 + "d3-force": "^3.0.0",
30 + "date-fns": "^4.4.0",
31 + "dotenv": "^18.0.9",
32 + "lucide-react": "^1.66.0",
33 + "next": "16.4.3",
34 + "otplib": "^13.5.0",
35 + "postgres": "^4.0.0",
36 + "qrcode": "^1.5.4",
37 + "react": "19.3.1",
38 + "react-dom": "19.3.1",
39 + "tailwindcss": "^4.3.3",
40 + "@tailwindcss/postcss": "^4.3.3",
41 + "zod": "^4.6.7"
42 + },
43 + "devDependencies": {
44 + "@playwright/test": "^1.64.1",
45 + "@types/d3-force": "^3.0.10",
46 + "@types/node": "^26.6.6",
47 + "@types/qrcode": "^1.5.6",
48 + "@types/react": "^19.3.0",
49 + "@types/react-dom": "^19.3.0",
50 + "tsx": "^4.23.15",
51 + "typescript": "^7.0.2",
52 + "vitest": "~5.0.0"
53 + }
54 +}
added playwright.config.ts +10 −0
@@ -0,0 +1,10 @@
1 +import { defineConfig, devices } from "@playwright/test";
2 +const base = process.env.E2E_BASE_URL ?? "http://127.0.0.1:8560";
3 +export default defineConfig({
4 + testDir: "tests/e2e", timeout: 60_000, retries: 1, reporter: [["list"]], fullyParallel: false,
5 + use: { baseURL: base, trace: "retain-on-failure" },
6 + projects: [
7 + { name: "desktop", use: { ...devices["Desktop Chrome"] } },
8 + { name: "iphone", use: { ...devices["iPhone 14"] }, grepInvert: /@desktop-only/ },
9 + ],
10 +});
added pnpm-workspace.yaml +6 −0
@@ -0,0 +1,6 @@
1 +onlyBuiltDependencies:
2 + - "@node-rs/argon2"
3 + - esbuild
4 + - sharp
5 +minimumReleaseAge: 1440
6 +verifyDepsBeforeRun: false
added postcss.config.mjs +1 −0
@@ -0,0 +1 @@
1 +export default { plugins: { "@tailwindcss/postcss": {} } };
added scripts/_admin-sql.ts +8 −0
@@ -0,0 +1,8 @@
1 +import postgres from "postgres";
2 +/** Admin client for migrations/backups. Accepts libpq-style socket URLs such as `postgres://user@/db?host=/tmp` (Homebrew Postgres, trust over the unix socket). */
3 +export function adminSql() {
4 + const url = process.env.DATABASE_URL_ADMIN ?? process.env.DATABASE_URL; if (!url) throw new Error("DATABASE_URL_ADMIN missing");
5 + const m = /^postgres(?:ql)?:\/\/([^@\/:]*)(?::[^@\/]*)?@\/([^?]+)\?host=(\/.+)$/.exec(url);
6 + if (m) return postgres({ path: m[3].endsWith(".s.PGSQL.5432") ? m[3] : `${m[3]}/.s.PGSQL.5432`, database: m[2], username: decodeURIComponent(m[1]) || process.env.USER, max: 1, onnotice: () => {} });
7 + return postgres(url, { max: 1, onnotice: () => {} });
8 +}
added scripts/_env.ts +6 −0
@@ -0,0 +1,6 @@
1 +import { config } from "dotenv";
2 +import { existsSync } from "node:fs";
3 +import { join } from "node:path";
4 +const root = join(import.meta.dirname, "..");
5 +for (const f of [".env.local", ".env"]) if (existsSync(join(root, f))) config({ path: join(root, f), quiet: true } as never);
6 +export const ROOT = root;
added scripts/backup.ts +3 −0
@@ -0,0 +1,3 @@
1 +import "./_env";
2 +import { runBackup } from "../src/lib/jobs/backup";
3 +console.log(JSON.stringify(await runBackup(), null, 2)); process.exit(0);
added scripts/bootstrap-owner.ts +22 −0
@@ -0,0 +1,22 @@
1 +import "./_env";
2 +import postgres from "postgres";
3 +import { hashPassword } from "../src/lib/auth/password";
4 +/** Creates the owner account from OWNER_EMAIL / OWNER_INITIAL_PASSWORD if no user exists; idempotent. Also seeds default settings. */
5 +const sql = postgres(process.env.DATABASE_URL!, { max: 1, onnotice: () => {} });
6 +const email = process.env.OWNER_EMAIL, pw = process.env.OWNER_INITIAL_PASSWORD;
7 +const [{ n }] = await sql`SELECT count(*)::int AS n FROM users`;
8 +if (n === 0) {
9 + if (!email || !pw) { console.error("OWNER_EMAIL / OWNER_INITIAL_PASSWORD missing"); process.exit(1); }
10 + await sql`INSERT INTO users (email, display_name, password_hash, role, must_change_password) VALUES (${email.toLowerCase()}, ${"Owner"}, ${await hashPassword(pw)}, 'owner', true)`;
11 + await sql`INSERT INTO audit_log (actor_label, action, entity_type, entity_id, after) VALUES ('bootstrap', 'user.create', 'user', ${email.toLowerCase()}, ${sql.json({ role: "owner" })})`;
12 + console.log(`owner ${email} created (must change password at first login)`);
13 +} else console.log(`users present (${n}); nothing to do`);
14 +const defaults: Record<string, unknown> = {
15 + "site.attribution": { en: "This observatory is an independent publication portal maintained by Simon-Pierre Boucher, professor at the Université du Québec en Outaouais (Département des sciences administratives). It is not an official UQO institutional repository. Working papers are circulated for discussion and comment; they have not necessarily undergone formal peer review.", fr: "Cet observatoire est un portail de publication indépendant maintenu par Simon-Pierre Boucher, professeur à l'Université du Québec en Outaouais (Département des sciences administratives). Ce n'est pas un dépôt institutionnel officiel de l'UQO. Les cahiers de recherche circulent aux fins de discussion et de commentaires ; ils n'ont pas nécessairement fait l'objet d'une évaluation par les pairs." },
16 + "site.institution": { name_en: "Université du Québec en Outaouais", name_fr: "Université du Québec en Outaouais", department_en: "Département des sciences administratives", department_fr: "Département des sciences administratives", url: "https://uqo.ca", logo_authorized: false },
17 + "site.contact": { email: "simon-pierre.boucher@uqo.ca", website: "https://www.spboucher.ai" },
18 + "home.featured_limit": 4,
19 + "ingest.extra_dirs": [],
20 +};
21 +for (const [k, v] of Object.entries(defaults)) await sql`INSERT INTO settings (key, value) VALUES (${k}, ${sql.json(v as never)}) ON CONFLICT (key) DO NOTHING`;
22 +await sql.end();
added scripts/db-roles.ts +7 −0
@@ -0,0 +1,7 @@
1 +import "./_env";
2 +import { adminSql } from "./_admin-sql";
3 +/** Re-applies privileges (idempotent) — same statements as drizzle/0002_roles.sql — useful after creating new tables. */
4 +const sql = adminSql();
5 +const { readFileSync } = await import("node:fs"); const { join } = await import("node:path");
6 +await sql.unsafe(readFileSync(join(import.meta.dirname, "..", "drizzle", "0002_roles.sql"), "utf8"));
7 +console.log("roles ok"); await sql.end();
added scripts/ingest-scan.ts +4 −0
@@ -0,0 +1,4 @@
1 +import "./_env";
2 +import { ingestScan } from "../src/lib/jobs/handlers";
3 +import { sql } from "../src/db";
4 +console.log(await ingestScan({ dirs: process.argv.slice(2) })); await sql.end();
added scripts/migrate.ts +20 −0
@@ -0,0 +1,20 @@
1 +import "./_env";
2 +import { adminSql } from "./_admin-sql";
3 +import { readdirSync, readFileSync } from "node:fs";
4 +import { join } from "node:path";
5 +import { createHash } from "node:crypto";
6 +const sql = adminSql();
7 +const dir = join(import.meta.dirname, "..", "drizzle");
8 +const files = readdirSync(dir).filter((f) => /^\d{4}_.*\.sql$/.test(f)).sort();
9 +await sql`CREATE TABLE IF NOT EXISTS schema_migrations (name text PRIMARY KEY, checksum text NOT NULL, applied_at timestamptz NOT NULL DEFAULT now())`;
10 +const applied = new Map((await sql`SELECT name, checksum FROM schema_migrations`).map((r) => [r.name as string, r.checksum as string]));
11 +let n = 0;
12 +for (const f of files) {
13 + const body = readFileSync(join(dir, f), "utf8"); const sum = createHash("sha256").update(body).digest("hex");
14 + if (applied.has(f)) { if (applied.get(f) !== sum) console.warn(`! ${f} changed after being applied (checksum mismatch)`); continue; }
15 + console.log(`→ applying ${f}`);
16 + await sql.begin(async (tx) => { await tx.unsafe(body); await tx`INSERT INTO schema_migrations (name, checksum) VALUES (${f}, ${sum})`; });
17 + n++;
18 +}
19 +console.log(n ? `${n} migration(s) applied` : "schema up to date");
20 +await sql.end();
added scripts/relations-rebuild.ts +4 −0
@@ -0,0 +1,4 @@
1 +import "./_env";
2 +import { rebuildRelations } from "../src/lib/jobs/relations";
3 +import { sql } from "../src/db";
4 +console.log(await rebuildRelations()); await sql.end();
added scripts/restore-drill.ts +18 −0
@@ -0,0 +1,18 @@
1 +import "./_env";
2 +import { execFile } from "node:child_process";
3 +import { promisify } from "node:util";
4 +import { promises as fs } from "node:fs";
5 +import { join } from "node:path";
6 +import postgres from "postgres";
7 +const run = promisify(execFile);
8 +/** Restore drill: decrypts the latest backup, restores the dump into a scratch database, compares row counts with production, then drops the scratch database. */
9 +const dir = process.env.BACKUP_DIR!; const pass = process.env.BACKUP_PASSPHRASE!; const pgBin = process.env.PG_BIN ?? "/opt/homebrew/opt/postgresql@17/bin";
10 +const files = (await fs.readdir(dir)).filter((f) => /^wp-\d+T\d+\.dump\.enc$/.test(f)).sort(); const latest = files[files.length - 1]; if (!latest) { console.error("no backup found in " + dir); process.exit(1); }
11 +const tmp = join(dir, "restore-drill.dump"); await run("openssl", ["enc", "-d", "-aes-256-cbc", "-pbkdf2", "-in", join(dir, latest), "-out", tmp, "-pass", "env:WP_BK"], { env: { ...process.env, WP_BK: pass } });
12 +const scratch = "wp_restore_drill"; const admin = postgres({ path: "/tmp/.s.PGSQL.5432", database: "postgres", username: process.env.USER, max: 1, onnotice: () => {} });
13 +await admin.unsafe(`DROP DATABASE IF EXISTS ${scratch}`); await admin.unsafe(`CREATE DATABASE ${scratch}`); await admin.end();
14 +await run(join(pgBin, "pg_restore"), ["--no-owner", "--no-privileges", "-d", `postgres://${process.env.USER}@/${scratch}?host=/tmp`, tmp]);
15 +const a = postgres({ path: "/tmp/.s.PGSQL.5432", database: scratch, username: process.env.USER, max: 1, onnotice: () => {} }); const b = postgres(process.env.DATABASE_URL!, { max: 1, onnotice: () => {} });
16 +let ok = true; for (const t of ["papers", "paper_versions", "files", "authors", "topics", "paper_chunks", "audit_log"]) { const [[x], [y]] = await Promise.all([a.unsafe(`SELECT count(*)::int AS n FROM ${t}`), b.unsafe(`SELECT count(*)::int AS n FROM ${t}`)]); const same = Math.abs(x.n - y.n) <= Math.max(2, y.n * 0.02); console.log(`${same ? "OK " : "DIFF"} ${t}: backup ${x.n} / live ${y.n}`); if (!same && t !== "audit_log" && t !== "paper_chunks") ok = false; }
17 +await a.end(); await b.end(); const admin2 = postgres({ path: "/tmp/.s.PGSQL.5432", database: "postgres", username: process.env.USER, max: 1, onnotice: () => {} }); await admin2.unsafe(`DROP DATABASE ${scratch}`); await admin2.end(); await fs.unlink(tmp);
18 +console.log(ok ? `restore drill OK (${latest})` : "restore drill FAILED"); process.exit(ok ? 0 : 1);
added scripts/security-audit.ts +27 −0
@@ -0,0 +1,27 @@
1 +import "./_env";
2 +/** Black-box security checks against a running instance (default: APP_URL). Usage: tsx scripts/security-audit.ts [base-url] */
3 +const base = (process.argv[2] ?? process.env.APP_URL ?? "http://127.0.0.1:8560").replace(/\/$/, "");
4 +let pass = 0, fail = 0; const out: string[] = [];
5 +const check = (name: string, ok: boolean, detail = "") => { ok ? pass++ : fail++; out.push(`${ok ? "PASS" : "FAIL"} ${name}${detail ? " — " + detail : ""}`); };
6 +const get = (p: string, init: RequestInit = {}) => fetch(base + p, { redirect: "manual", ...init });
7 +const home = await get("/"); const h = home.headers;
8 +check("HTTP 200 on home", home.status === 200);
9 +for (const [k, v] of [["content-security-policy", "default-src 'self'"], ["x-content-type-options", "nosniff"], ["x-frame-options", "DENY"], ["referrer-policy", "strict-origin"], ["strict-transport-security", "max-age"], ["permissions-policy", "camera=()"]]) check(`header ${k}`, (h.get(k) ?? "").includes(v), h.get(k) ?? "missing");
10 +check("no x-powered-by", !h.get("x-powered-by"));
11 +for (const p of ["/admin", "/admin/papers", "/admin/settings", "/admin/system"]) { const r = await get(p); check(`${p} redirects to /login`, [302, 303, 307].includes(r.status) && (r.headers.get("location") ?? "").includes("/login"), String(r.status)); }
12 +check("/admin noindex", ((await get("/admin")).headers.get("x-robots-tag") ?? "").includes("noindex"));
13 +const up = await get("/api/admin/upload", { method: "POST" }); check("upload without session → 401", up.status === 401, String(up.status));
14 +const pv = await get("/api/admin/preview/" + "a".repeat(64)); check("admin preview without session → 401", pv.status === 401, String(pv.status));
15 +const bad = await get("/login", { method: "POST", headers: { origin: "https://evil.example", "content-type": "application/x-www-form-urlencoded" }, body: "email=a@b.c&password=x" }); check("cross-origin POST to login rejected", bad.status !== 200 || !(await bad.text()).includes("/admin"), String(bad.status));
16 +const ev = await get("/api/events", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ kind: "view", slug: "../etc" }) }); check("events endpoint validates input", ev.status === 204);
17 +const sqli = await get("/api/v1/papers?q=" + encodeURIComponent("' OR 1=1; --")); check("search survives injection attempt", sqli.status === 200);
18 +const tr = await get("/api/files/00000000-0000-0000-0000-000000000000/x.pdf"); check("unknown file id → 404", tr.status === 404);
19 +const tr2 = await get("/api/files/../../etc/passwd"); check("path traversal blocked", tr2.status === 404 || tr2.status === 400 || tr2.status === 308, String(tr2.status));
20 +const reserved = await get("/papers/wp39"); check("reserved identifier not public", reserved.status === 404, String(reserved.status));
21 +const api = await get("/api/v1/papers/wp39"); check("reserved identifier not in API", api.status === 404, String(api.status));
22 +let limited = false; for (let i = 0; i < 70; i++) { const r = await get("/api/v1/search?q=x" + i); if (r.status === 429) { limited = true; break; } } check("search rate limiting (60/min)", limited);
23 +const oai = await get("/api/oai?verb=GetRecord&identifier=oai:wp.spboucher.ai:wp39&metadataPrefix=oai_dc"); check("OAI hides reserved records", (await oai.text()).includes("idDoesNotExist"));
24 +const robots = await (await get("/robots.txt")).text(); check("robots disallows admin", robots.includes("Disallow: /admin"));
25 +const lg = await get("/login"); check("login no-store", (lg.headers.get("cache-control") ?? "").includes("no-store"));
26 +const body = await home.text(); check("no secret-looking strings in home HTML", !/sk-[A-Za-z0-9]{10,}|postgres:\/\/|SESSION_SECRET/.test(body));
27 +console.log(out.join("\n")); console.log(`\n${pass} passed, ${fail} failed (${base})`); process.exit(fail ? 1 : 0);
added scripts/seed-import.ts +105 −0
@@ -0,0 +1,105 @@
1 +import "./_env";
2 +import { readFileSync, existsSync, statSync } from "node:fs";
3 +import { join } from "node:path";
4 +import { sql } from "../src/db";
5 +import { makeIdentifier, parseVersionLabel } from "../src/lib/papers/identifiers";
6 +import { sha256File, storeFile, ensureStorage } from "../src/lib/pdf/process";
7 +import { lexvecFromMetadata } from "../src/lib/jobs/handlers";
8 +import { enqueue } from "../src/lib/jobs/queue";
9 +import { STORAGE_DIR } from "../src/lib/env";
10 +
11 +/**
12 + * Imports the verified catalog (content/seed/catalog.json — written from the manuscripts on 2026-11-01 for www.spboucher.ai)
13 + * and the editorial taxonomy (content/seed/taxonomy.json). Idempotent: papers are upserted by slug; relations/tags are replaced.
14 + * Manuscripts are looked up in $STORAGE_DIR/inbox/<slug>_uqo.pdf (or --pdf-dir). Status = published only with --publish, because the
15 + * records were already public on www.spboucher.ai; otherwise papers are imported as drafts for review in /admin.
16 + */
17 +const args = process.argv.slice(2); const publish = args.includes("--publish");
18 +const pdfDir = args.find((a) => a.startsWith("--pdf-dir="))?.slice(10) ?? join(STORAGE_DIR, "inbox");
19 +const root = join(import.meta.dirname, "..");
20 +const catalog = JSON.parse(readFileSync(join(root, "content/seed/catalog.json"), "utf8")) as { papers: any[]; details: any[] };
21 +const tx = JSON.parse(readFileSync(join(root, "content/seed/taxonomy.json"), "utf8")) as any;
22 +const MONTH: Record<string, number> = { january: 1, february: 2, march: 3, april: 4, may: 5, june: 6, july: 7, august: 8, september: 9, october: 10, november: 11, december: 12 };
23 +function monthDate(s?: string): string | null { if (!s) return null; const m = /^(\w+)\s+(\d{4})$/.exec(s.trim()); if (!m) return null; const mo = MONTH[m[1].toLowerCase()]; return mo ? `${m[2]}-${String(mo).padStart(2, "0")}-01` : null; }
24 +
25 +await ensureStorage();
26 +const log = (m: string) => console.log(m);
27 +// Series
28 +const seriesId: Record<string, number> = {};
29 +for (const s of tx.series) { const [r] = await sql`INSERT INTO series (slug, code, name_en, name_fr, description_en, description_fr, sort) VALUES (${s.slug}, ${s.code}, ${s.name_en}, ${s.name_fr}, ${s.description_en}, ${s.description_fr}, ${s.sort}) ON CONFLICT (slug) DO UPDATE SET name_en = EXCLUDED.name_en, name_fr = EXCLUDED.name_fr, description_en = EXCLUDED.description_en, description_fr = EXCLUDED.description_fr, sort = EXCLUDED.sort RETURNING id`; seriesId[s.code] = r.id; }
30 +// Authors
31 +const authorId: Record<string, number> = {}; const authorByName: Record<string, number> = {};
32 +for (const a of tx.authors) { const [r] = await sql`INSERT INTO authors (slug, given_name, family_name, display_name, is_primary, email, website, affiliation_en, affiliation_fr, department_en, department_fr, bio_en, bio_fr, sort) VALUES (${a.slug}, ${a.given_name}, ${a.family_name}, ${a.display_name}, ${a.is_primary}, ${a.email ?? null}, ${a.website ?? null}, ${a.affiliation_en ?? null}, ${a.affiliation_fr ?? null}, ${a.department_en ?? null}, ${a.department_fr ?? null}, ${a.bio_en ?? null}, ${a.bio_fr ?? null}, ${a.sort}) ON CONFLICT (slug) DO UPDATE SET display_name = EXCLUDED.display_name, is_primary = EXCLUDED.is_primary, email = EXCLUDED.email, website = EXCLUDED.website, affiliation_en = EXCLUDED.affiliation_en, affiliation_fr = EXCLUDED.affiliation_fr, department_en = EXCLUDED.department_en, department_fr = EXCLUDED.department_fr, bio_en = EXCLUDED.bio_en, bio_fr = EXCLUDED.bio_fr RETURNING id`; authorId[a.slug] = r.id; authorByName[a.display_name] = r.id; }
33 +// Topics (domains + topics)
34 +const topicId: Record<string, number> = {};
35 +let sort = 0;
36 +for (const d of tx.domains) { const [r] = await sql`INSERT INTO topics (slug, name_en, name_fr, description_en, description_fr, kind, accent, sort) VALUES (${d.slug}, ${d.name_en}, ${d.name_fr}, ${d.description_en ?? null}, ${d.description_fr ?? null}, 'domain', ${d.accent ?? null}, ${sort++}) ON CONFLICT (slug) DO UPDATE SET name_en = EXCLUDED.name_en, name_fr = EXCLUDED.name_fr, description_en = EXCLUDED.description_en, description_fr = EXCLUDED.description_fr, kind = 'domain', accent = EXCLUDED.accent RETURNING id`; topicId[d.slug] = r.id; }
37 +for (const t of tx.topics) { const [r] = await sql`INSERT INTO topics (slug, name_en, name_fr, kind, sort) VALUES (${t.slug}, ${t.name_en}, ${t.name_fr}, 'topic', ${sort++}) ON CONFLICT (slug) DO UPDATE SET name_en = EXCLUDED.name_en, name_fr = EXCLUDED.name_fr RETURNING id`; topicId[t.slug] = r.id; }
38 +const methodId: Record<string, number> = {};
39 +for (const m of tx.methods) { const [r] = await sql`INSERT INTO methods (slug, name_en, name_fr, category, sort) VALUES (${m.slug}, ${m.name_en}, ${m.name_fr}, ${m.category}, ${sort++}) ON CONFLICT (slug) DO UPDATE SET name_en = EXCLUDED.name_en, name_fr = EXCLUDED.name_fr, category = EXCLUDED.category RETURNING id`; methodId[m.slug] = r.id; }
40 +const datasetId: Record<string, number> = {};
41 +for (const d of tx.datasets) { const [r] = await sql`INSERT INTO datasets (slug, name_en, name_fr, provider, coverage, frequency, access_conditions, license, url, is_public, sort) VALUES (${d.slug}, ${d.name_en}, ${d.name_fr}, ${d.provider ?? null}, ${d.coverage ?? null}, ${d.frequency ?? null}, ${d.access_conditions ?? null}, ${d.license ?? null}, ${d.url ?? null}, ${d.is_public ?? false}, ${sort++}) ON CONFLICT (slug) DO UPDATE SET name_en = EXCLUDED.name_en, name_fr = EXCLUDED.name_fr, provider = EXCLUDED.provider, coverage = EXCLUDED.coverage, frequency = EXCLUDED.frequency, access_conditions = EXCLUDED.access_conditions, url = EXCLUDED.url, is_public = EXCLUDED.is_public RETURNING id`; datasetId[d.slug] = r.id; }
42 +log(`taxonomy: ${tx.domains.length} domains, ${tx.topics.length} topics, ${tx.methods.length} methods, ${tx.datasets.length} datasets`);
43 +
44 +let created = 0, updated = 0, withPdf = 0;
45 +for (const p of catalog.papers) {
46 + const d = catalog.details.find((x: any) => x.slug === p.slug) ?? {};
47 + const code = p.kind === "firc" ? "FIRC" : "WP"; const idp = makeIdentifier(code, p.number);
48 + const released = monthDate(p.date); const ver = parseVersionLabel(p.version ?? "1") ?? { versionNo: 1, label: "v1" };
49 + const ssrnId = p.ssrn ? (/abstract=(\d+)/.exec(p.ssrn)?.[1] ?? null) : null;
50 + const artifacts: unknown[] = []; if (p.repo) artifacts.push({ kind: "repository", label: "Source repository (git.spboucher.ai)", url: p.repo });
51 + const links: unknown[] = []; if (p.ssrn) links.push({ kind: "ssrn", label: "SSRN", url: p.ssrn });
52 + const status = publish ? "published" : "draft";
53 + const notice = p.status ?? null;
54 + const [existing] = await sql`SELECT id, status FROM papers WHERE slug = ${idp.slug}`;
55 + const row = {
56 + series_id: seriesId[code], number: p.number, identifier: idp.identifier, slug: idp.slug, internal_code: p.repo ? p.repo.split("/").pop() : null,
57 + title: p.title, subtitle: p.subtitle ?? null, abstract: sql.json(d.abstract ?? [p.description]), research_question: null, notice,
58 + hero_headline: d.hero?.headline ?? null, hero_subheadline: d.hero?.subheadline ?? null,
59 + contributions: sql.json(d.contributions ?? []), methodology: sql.json(d.methodology ?? []), data_sections: sql.json(d.data ?? []), findings: sql.json(d.findings ?? []), reproducibility: sql.json(d.reproducibility ?? []),
60 + artifacts: sql.json(artifacts), keywords: d.keywords ?? [], jel: d.jel ? String(d.jel).split(/,\s*/).filter(Boolean) : [],
61 + ssrn_id: ssrnId, external_links: sql.json(links), year: p.year ?? null,
62 + first_released_on: released, first_released_precision: "month", last_revised_on: released, last_revised_precision: "month",
63 + primary_domain_id: topicId[tx.domain_of[p.slug]] ?? null, featured: tx.featured.includes(p.slug), featured_rank: tx.featured.includes(p.slug) ? tx.featured.indexOf(p.slug) + 1 : null,
64 + };
65 + let paperId: string;
66 + if (existing) { await sql`UPDATE papers SET ${sql(row as never)} WHERE id = ${existing.id}`; paperId = existing.id; updated++; }
67 + else { const [r] = await sql`INSERT INTO papers ${sql({ ...row, status, published_at: status === "published" ? new Date() : null } as never)} RETURNING id`; paperId = r.id; created++; }
68 + // Authors (order from the catalog)
69 + await sql`DELETE FROM paper_authors WHERE paper_id = ${paperId}`;
70 + let pos = 1; for (const name of p.authors ?? ["Simon-Pierre Boucher"]) { const aid = authorByName[name]; if (!aid) throw new Error(`unknown author ${name}`); await sql`INSERT INTO paper_authors (paper_id, author_id, position) VALUES (${paperId}, ${aid}, ${pos++})`; }
71 + // Topics: primary domain + secondary domains/topics + topic map
72 + const tset = new Set<number>(); tset.add(topicId[tx.domain_of[p.slug]]);
73 + for (const s of tx.secondary_domains[p.slug] ?? []) tset.add(topicId[s]);
74 + for (const [ts, arr] of Object.entries(tx.topics_of as Record<string, string[]>)) if (arr.includes(p.slug)) tset.add(topicId[ts]);
75 + await sql`DELETE FROM paper_topics WHERE paper_id = ${paperId} AND assigned_by = 'editor'`;
76 + for (const tid of tset) await sql`INSERT INTO paper_topics (paper_id, topic_id, assigned_by) VALUES (${paperId}, ${tid}, 'editor') ON CONFLICT DO NOTHING`;
77 + await sql`DELETE FROM paper_methods WHERE paper_id = ${paperId} AND assigned_by = 'editor'`;
78 + for (const [ms, arr] of Object.entries(tx.methods_of as Record<string, string[]>)) if (arr.includes(p.slug)) await sql`INSERT INTO paper_methods (paper_id, method_id, assigned_by) VALUES (${paperId}, ${methodId[ms]}, 'editor') ON CONFLICT DO NOTHING`;
79 + await sql`DELETE FROM paper_datasets WHERE paper_id = ${paperId} AND assigned_by = 'editor'`;
80 + for (const [ds, arr] of Object.entries(tx.datasets_of as Record<string, string[]>)) if (arr.includes(p.slug)) await sql`INSERT INTO paper_datasets (paper_id, dataset_id, assigned_by) VALUES (${paperId}, ${datasetId[ds]}, 'editor') ON CONFLICT DO NOTHING`;
81 + // Manuscript → file + version 1 (label = manuscript's own version number)
82 + const pdfPath = join(pdfDir, `${p.slug}_uqo.pdf`);
83 + if (existsSync(pdfPath)) {
84 + const sha = await sha256File(pdfPath); const st = statSync(pdfPath);
85 + let [f] = await sql`SELECT id FROM files WHERE sha256 = ${sha}`;
86 + if (!f) { const key = await storeFile(pdfPath, sha); [f] = await sql`INSERT INTO files (kind, storage_key, original_name, mime, bytes, sha256) VALUES ('manuscript', ${key}, ${`${p.slug}_uqo.pdf`}, 'application/pdf', ${st.size}, ${sha}) RETURNING id`; }
87 + const [v] = await sql`INSERT INTO paper_versions (paper_id, version_no, label, released_on, released_precision, description, description_fr, file_id, pages, sha256, bytes, is_current)
88 + VALUES (${paperId}, ${1}, ${ver.label}, ${released}, 'month', ${"Current manuscript as published on www.spboucher.ai (revised edition after the 2026-10-17 scientific review for papers 1–17)."}, ${"Manuscrit courant tel que publié sur www.spboucher.ai (édition révisée après la révision scientifique du 2026-10-17 pour les cahiers 1–17)."}, ${f.id}, ${p.pageCount ?? null}, ${sha}, ${st.size}, true)
89 + ON CONFLICT (paper_id, version_no) DO UPDATE SET label = EXCLUDED.label, file_id = EXCLUDED.file_id, pages = EXCLUDED.pages, sha256 = EXCLUDED.sha256, bytes = EXCLUDED.bytes, released_on = EXCLUDED.released_on, is_current = true RETURNING id`;
90 + await sql`UPDATE papers SET current_version_id = ${v.id} WHERE id = ${paperId}`;
91 + await enqueue("process_pdf", { fileId: f.id, paperId, versionId: v.id }, { dedupe: true });
92 + withPdf++;
93 + } else log(`! no manuscript for ${p.slug} (${pdfPath})`);
94 + await lexvecFromMetadata(paperId);
95 +}
96 +// Reserved identifiers (never public)
97 +for (const r of tx.reserved) {
98 + const idp = makeIdentifier(r.code, r.number);
99 + await sql`INSERT INTO papers (series_id, number, identifier, slug, internal_code, title, status, notice) VALUES (${seriesId[r.code]}, ${r.number}, ${idp.identifier}, ${idp.slug}, ${r.internal_code}, ${r.title}, 'reserved', ${r.note})
100 + ON CONFLICT (slug) DO UPDATE SET title = EXCLUDED.title, internal_code = EXCLUDED.internal_code, notice = EXCLUDED.notice`;
101 +}
102 +await sql`INSERT INTO audit_log (actor_label, action, entity_type, entity_id, after) VALUES ('seed-import', ${publish ? "papers.seed_published" : "papers.seed_draft"}, 'catalog', 'content/seed/catalog.json', ${sql.json({ created, updated, withPdf, reserved: tx.reserved.length, note: publish ? "Records were already public on www.spboucher.ai; imported as published with owner approval given at deployment." : "Imported as drafts for review." })})`;
103 +await enqueue("relations", {}, { dedupe: true });
104 +log(`papers: ${created} created, ${updated} updated, ${withPdf} with manuscript, ${tx.reserved.length} reserved identifiers`);
105 +await sql.end();
added scripts/verify-deploy.ts +20 −0
@@ -0,0 +1,20 @@
1 +import "./_env";
2 +/** Post-deployment verification against the public URL: TLS, routes, API consistency with the database, discoverability endpoints. */
3 +const base = (process.argv[2] ?? process.env.APP_URL ?? "https://wp.spboucher.ai").replace(/\/$/, "");
4 +let pass = 0, fail = 0; const out: string[] = [];
5 +const check = (name: string, ok: boolean, detail = "") => { ok ? pass++ : fail++; out.push(`${ok ? "PASS" : "FAIL"} ${name}${detail ? " — " + detail : ""}`); };
6 +const t0 = Date.now(); const health = await fetch(base + "/api/health").then((r) => r.json()).catch((e) => ({ ok: false, error: String(e) })); check("health", health.ok === true, JSON.stringify(health)); check("health latency < 2 s", Date.now() - t0 < 2000);
7 +const stats = await fetch(base + "/api/v1/stats").then((r) => r.json());
8 +check("stats.papers equals health.papers (single source of truth)", stats.papers === health.papers, `${stats.papers} vs ${health.papers}`);
9 +const list = await fetch(base + "/api/v1/papers?per_page=100").then((r) => r.json()); check("list total equals stats.papers", list.total === stats.papers, `${list.total}`); check("every listed paper has a PDF url or is flagged", list.items.every((p: { has_pdf: boolean; pdf_url: string | null }) => !p.has_pdf || p.pdf_url), "");
10 +const first = list.items[0]; const full = await fetch(`${base}/api/v1/papers/${first.slug}`).then((r) => r.json()); check("detail endpoint", full.slug === first.slug);
11 +const pdf = full.pdf_url ? await fetch(full.pdf_url, { method: "HEAD" }) : null; check("current manuscript downloadable", !!pdf && pdf.status === 200 && (pdf.headers.get("content-type") ?? "").includes("pdf"), pdf ? `${pdf.status} ${pdf.headers.get("content-length")} bytes, sha ${pdf.headers.get("x-checksum-sha256")?.slice(0, 12)}` : "no pdf");
12 +for (const p of ["/", "/fr", "/papers", `/papers/${first.slug}`, `/fr/papers/${first.slug}`, "/topics", "/methods", "/datasets", "/authors", "/series", "/research-map", "/timeline", "/search?q=quebec", "/about", "/citation", "/api/docs", "/sitemap.xml", "/feed.xml", "/robots.txt", "/api/oai?verb=Identify", "/api/v1/openapi.json"]) { const r = await fetch(base + p); check(`GET ${p}`, r.status === 200, String(r.status)); }
13 +const html = await fetch(`${base}/papers/${first.slug}`).then((r) => r.text()); check("Google Scholar meta tags", html.includes('name="citation_title"') && html.includes('name="citation_pdf_url"')); check("JSON-LD ScholarlyArticle", html.includes('"@type":"ScholarlyArticle"')); check("hreflang alternates", html.includes('hrefLang="fr"') || html.includes('hreflang="fr"'));
14 +const fr = await fetch(`${base}/fr/papers/${first.slug}`).then((r) => r.text()); check("French render differs from English", fr.includes('lang="fr"') && fr.includes("Résumé"));
15 +const red = await fetch(`${base}/papers/${first.identifier}`, { redirect: "manual" }); check("identifier alias → 308 canonical", red.status === 308 && (red.headers.get("location") ?? "").endsWith(`/papers/${first.slug}`), String(red.status));
16 +check("404 for unknown paper", (await fetch(base + "/papers/wp99999")).status === 404); check("reserved WP39 hidden", (await fetch(base + "/papers/wp39")).status === 404);
17 +const admin = await fetch(base + "/admin", { redirect: "manual" }); check("admin protected", [302, 307].includes(admin.status));
18 +const sec = await fetch(base + "/").then((r) => r.headers); check("HSTS + CSP present", !!sec.get("strict-transport-security") && !!sec.get("content-security-policy"));
19 +const oai = await fetch(base + "/api/oai?verb=ListRecords&metadataPrefix=oai_dc").then((r) => r.text()); check("OAI ListRecords count matches", (oai.match(/<record>/g) ?? []).length === stats.papers, `${(oai.match(/<record>/g) ?? []).length}`);
20 +console.log(out.join("\n")); console.log(`\n${pass} passed, ${fail} failed (${base})`); process.exit(fail ? 1 : 0);
added src/app/(auth)/login/actions.ts +31 −0
@@ -0,0 +1,31 @@
1 +"use server";
2 +import { redirect } from "next/navigation";
3 +import { headers } from "next/headers";
4 +import { verifySync } from "otplib";
5 +import { sql } from "@/db";
6 +import { verifyPassword } from "@/lib/auth/password";
7 +import { clientIp, createSession, getSession, markMfaPassed, rotateSession, setSessionCookie } from "@/lib/auth/session";
8 +import { loginAllowed, recordAttempt } from "@/lib/auth/attempts";
9 +import { assertSameOrigin } from "@/lib/auth/guard";
10 +import { audit } from "@/lib/audit";
11 +export async function loginAction(f: FormData) {
12 + await assertSameOrigin();
13 + const email = String(f.get("email") ?? "").trim().toLowerCase(); const pw = String(f.get("password") ?? ""); const ip = clientIp(await headers());
14 + const allowed = await loginAllowed(ip, email); if (!allowed.ok) redirect(`/login?error=${encodeURIComponent(`Too many attempts. Try again in ${Math.ceil(allowed.retryAfterSec / 60)} min.`)}`);
15 + const [u] = await sql`SELECT * FROM users WHERE email = ${email}`;
16 + const ok = u ? await verifyPassword(u.password_hash, pw) : (await verifyPassword("$argon2id$v=19$m=65536,t=3,p=1$AAAAAAAAAAAAAAAAAAAAAA$AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", pw), false);
17 + await recordAttempt(ip, email, ok);
18 + if (!ok) { await audit({ label: email, ip }, "login.failed", "user", email, null, null); redirect("/login?error=Invalid+credentials"); }
19 + const token = await createSession(u.id, { mfaPassed: !u.totp_enabled }); await setSessionCookie(token);
20 + await sql`UPDATE users SET last_login_at = now() WHERE id = ${u.id}`; await audit({ id: u.id, label: email, ip }, "login.success", "user", u.id, null, { mfa: u.totp_enabled });
21 + redirect(u.totp_enabled ? "/login?step=totp" : "/admin");
22 +}
23 +export async function totpAction(f: FormData) {
24 + await assertSameOrigin(); const s = await getSession(); if (!s) redirect("/login");
25 + const [u] = await sql`SELECT totp_secret FROM users WHERE id = ${s.id}`; const code = String(f.get("code") ?? "").trim(); const ip = clientIp(await headers());
26 + const allowed = await loginAllowed(ip, s.email); if (!allowed.ok) redirect("/login?step=totp&error=Too+many+attempts");
27 + const ok = !!u?.totp_secret && verifySync({ token: code, secret: u.totp_secret }).valid; await recordAttempt(ip, s.email, ok);
28 + if (!ok) { await audit({ id: s.id, label: s.email, ip }, "login.totp_failed", "user", s.id, null, null); redirect("/login?step=totp&error=Invalid+code"); }
29 + await markMfaPassed(s.sessionId); const t = await rotateSession(s.sessionId); await setSessionCookie(t); await audit({ id: s.id, label: s.email, ip }, "login.totp_ok", "user", s.id, null, null);
30 + redirect("/admin");
31 +}
added src/app/(auth)/login/page.tsx +24 −0
@@ -0,0 +1,24 @@
1 +import type { Metadata } from "next";
2 +import { loginAction, totpAction } from "./actions";
3 +import { getSession } from "@/lib/auth/session";
4 +import { redirect } from "next/navigation";
5 +export const metadata: Metadata = { title: "Sign in", robots: { index: false } };
6 +export const dynamic = "force-dynamic";
7 +export default async function LoginPage({ searchParams }: { searchParams: Promise<{ step?: string; error?: string; next?: string }> }) {
8 + const sp = await searchParams; const s = await getSession();
9 + if (s && (!s.totpEnabled || s.mfaPassed)) redirect("/admin");
10 + const totp = sp.step === "totp" && s && s.totpEnabled && !s.mfaPassed;
11 + return (
12 + <main className="min-h-dvh flex items-center justify-center p-6">
13 + <form action={totp ? totpAction : loginAction} className="card p-6 w-full max-w-sm space-y-4">
14 + <div><p className="kicker">Administration</p><h1 className="text-2xl mt-1">{totp ? "Two-factor code" : "Sign in"}</h1></div>
15 + {sp.error && <p className="text-sm text-bad">{sp.error}</p>}
16 + {totp ? <label className="block text-sm"><span className="label">Authenticator code</span><input name="code" inputMode="numeric" autoComplete="one-time-code" pattern="[0-9]{6}" required autoFocus className="input mt-1 font-mono tracking-widest" /></label>
17 + : <><label className="block text-sm"><span className="label">E-mail</span><input name="email" type="email" autoComplete="username" required autoFocus className="input mt-1" /></label>
18 + <label className="block text-sm"><span className="label">Password</span><input name="password" type="password" autoComplete="current-password" required className="input mt-1" /></label></>}
19 + <button className="btn btn-primary w-full">{totp ? "Verify" : "Sign in"}</button>
20 + <p className="text-xs text-fg-3">Access is limited to the observatory's administrators. Attempts are logged.</p>
21 + </form>
22 + </main>
23 + );
24 +}
added src/app/(public)/[locale]/about/page.tsx +31 −0
@@ -0,0 +1,31 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { href, t } from "@/lib/i18n";
5 +import { siteSetting, stats } from "@/lib/papers/queries";
6 +export const revalidate = 300;
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.about.title, alternates: { canonical: "/about" } }; }
8 +export default async function AboutPage() {
9 + const locale = await getLocale(); const d = t(locale); const fr = locale === "fr";
10 + const [attr, inst, contact, s] = await Promise.all([siteSetting<{ en: string; fr: string }>("site.attribution"), siteSetting<{ name_en: string; name_fr: string; department_en?: string; department_fr?: string; url?: string }>("site.institution"), siteSetting<{ email?: string; website?: string }>("site.contact"), stats()]);
11 + return (<article className="container py-10 max-w-3xl prose-academic"><p className="kicker">{d.site.secondary}</p><h1 className="text-3xl sm:text-4xl mt-2 mb-6">{d.about.title}</h1>
12 + <p>{fr ? "Cet observatoire est le portail public de publication des cahiers de recherche associés à Simon-Pierre Boucher et à ses activités de recherche à l'Université du Québec en Outaouais. Il réunit une bibliothèque scientifique (manuscrits, métadonnées structurées, versions et outils de citation), un moteur de découverte (thèmes, méthodes, données, auteurs, chronologie) et une couche d'intelligence de recherche qui relie les cahiers entre eux." : "This observatory is the public publication portal for the research working papers associated with Simon-Pierre Boucher and his research activities at the Université du Québec en Outaouais. It combines a scientific library (manuscripts, structured metadata, versions and citation tools), a discovery engine (topics, methods, data, authors, timeline) and a research-intelligence layer that connects papers to one another."}</p>
13 + <h2 className="text-2xl mt-8 mb-3">{fr ? "Statut institutionnel" : "Institutional status"}</h2>
14 + <p>{attr ? (fr ? attr.fr : attr.en) : ""}</p>
15 + {inst && <p>{fr ? "Affiliation de l'auteur : " : "Author affiliation: "}{fr ? inst.name_fr : inst.name_en}{(fr ? inst.department_fr : inst.department_en) ? `, ${fr ? inst.department_fr : inst.department_en}` : ""}{inst.url ? <> (<a href={inst.url} rel="noopener" className="text-navy">{inst.url}</a>)</> : null}.</p>}
16 + <h2 className="text-2xl mt-8 mb-3">{fr ? "Principes éditoriaux" : "Editorial principles"}</h2>
17 + <ul className="list-disc pl-5 space-y-2 text-fg-2">
18 + <li>{fr ? "Les numéros de cahier sont permanents : ils ne sont jamais réattribués et les trous sont permis." : "Working paper numbers are permanent: they are never reassigned and gaps are allowed."}</li>
19 + <li>{fr ? "Chaque version publiée reste téléchargeable, sauf retrait pour motif légal ou de confidentialité ; l'historique est immuable." : "Every published version remains downloadable unless withdrawn for legal or privacy reasons; the history is immutable."}</li>
20 + <li>{fr ? "Les métadonnées (résumés, contributions, méthodes, données, résultats) sont rédigées à partir des manuscrits et approuvées par l'éditeur. Les liens inférés par similarité textuelle ou sémantique sont identifiés comme tels." : "Metadata (abstracts, contributions, methods, data, findings) is written from the manuscripts and approved by the editor. Links inferred from textual or semantic similarity are labelled as inferred."}</li>
21 + <li>{fr ? "Les statistiques affichées sont calculées en direct depuis la base de publications ; rien n'est estimé." : "Displayed statistics are computed live from the publication database; nothing is estimated."}</li>
22 + <li>{fr ? "Les cahiers circulent aux fins de discussion ; ils n'ont pas nécessairement fait l'objet d'une évaluation par les pairs. Le statut de publication en revue est indiqué sur chaque fiche lorsqu'il est connu." : "Papers circulate for discussion; they have not necessarily undergone peer review. Journal publication status is shown on each record when known."}</li>
23 + </ul>
24 + <h2 className="text-2xl mt-8 mb-3">{fr ? "Découvrabilité" : "Discoverability"}</h2>
25 + <p>{fr ? "Chaque fiche expose des balises Highwire Press (Google Scholar), Dublin Core et JSON-LD (ScholarlyArticle). Un point d'accès OAI-PMH, un fil RSS, un plan de site et une API publique sont disponibles." : "Each record exposes Highwire Press (Google Scholar), Dublin Core and JSON-LD (ScholarlyArticle) tags. An OAI-PMH endpoint, an RSS feed, a sitemap and a public API are available."} <Link href="/api/docs" className="text-navy">{d.nav.api}</Link> · <a href="/api/oai?verb=Identify" className="text-navy">OAI-PMH</a> · <a href="/feed.xml" className="text-navy">RSS</a>.</p>
26 + <h2 className="text-2xl mt-8 mb-3">{fr ? "La collection" : "The collection"}</h2>
27 + <p>{fr ? `${s.papers} cahiers publics, ${s.manuscripts} manuscrits téléchargeables, ${s.pages} pages, ${s.authors} auteur(s).` : `${s.papers} public working papers, ${s.manuscripts} downloadable manuscripts, ${s.pages} pages, ${s.authors} author(s).`}</p>
28 + {contact && <><h2 className="text-2xl mt-8 mb-3">Contact</h2><p>{contact.email && <a href={`mailto:${contact.email}`} className="text-navy">{contact.email}</a>}{contact.website && <> · <a href={contact.website} rel="me noopener" className="text-navy">{contact.website}</a></>}</p></>}
29 + <p className="mt-8"><Link href={href(locale, "/citation")} className="text-navy">{d.nav.citation} →</Link></p>
30 + </article>);
31 +}
added src/app/(public)/[locale]/api/docs/page.tsx +14 −0
@@ -0,0 +1,14 @@
1 +import type { Metadata } from "next";
2 +import { getLocale } from "@/lib/i18n/server";
3 +import { t } from "@/lib/i18n";
4 +import { openapi } from "@/lib/papers/openapi";
5 +export const revalidate = 3600;
6 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.api.title, alternates: { canonical: "/api/docs" } }; }
7 +export default async function ApiDocs() {
8 + const locale = await getLocale(); const d = t(locale); const spec = openapi();
9 + return (<div className="container py-10 max-w-4xl"><h1 className="text-3xl sm:text-4xl">{d.api.title}</h1><p className="meta mt-2">{spec.info.description} · <a href="/api/v1/openapi.json" className="text-navy">openapi.json</a></p>
10 + <p className="text-sm text-fg-2 mt-3">{locale === "fr" ? "Lecture seule, sans authentification, CORS ouvert, limite de 120 requêtes par minute et par adresse. Les réponses sont en JSON ; les citations en BibTeX ou RIS." : "Read-only, no authentication, open CORS, limited to 120 requests per minute per address. Responses are JSON; citations are BibTeX or RIS."}</p>
11 + <div className="mt-8 space-y-4">{Object.entries(spec.paths).map(([path, ops]) => Object.entries(ops as Record<string, { summary: string; description?: string; parameters?: { name: string; in: string; description?: string; schema?: { type?: string; enum?: string[] } }[] }>).map(([method, op]) => <section key={method + path} className="card p-4"><h2 className="font-mono text-base"><span className="chip chip-navy uppercase mr-2">{method}</span>{path}</h2><p className="text-sm text-fg mt-2 font-medium">{op.summary}</p>{op.description && <p className="text-sm text-fg-2 mt-1">{op.description}</p>}{op.parameters?.length ? <ul className="mt-2 text-sm grid gap-1">{op.parameters.map((p) => <li key={p.name}><code className="text-navy">{p.name}</code> <span className="text-fg-3">({p.in}{p.schema?.type ? `, ${p.schema.type}` : ""}{p.schema?.enum ? `: ${p.schema.enum.join(" | ")}` : ""})</span>{p.description ? ` — ${p.description}` : ""}</li>)}</ul> : null}</section>))}</div>
12 + <h2 className="text-2xl mt-10 mb-3">OAI-PMH</h2><p className="text-sm text-fg-2"><code>/api/oai?verb=Identify</code> · <code>?verb=ListRecords&metadataPrefix=oai_dc</code> · <code>?verb=GetRecord&identifier=oai:wp.spboucher.ai:wp38&metadataPrefix=oai_dc</code> · <code>?verb=ListSets</code></p>
13 + </div>);
14 +}
added src/app/(public)/[locale]/authors/[slug]/page.tsx +18 −0
@@ -0,0 +1,18 @@
1 +import type { Metadata } from "next";
2 +import { notFound } from "next/navigation";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { t } from "@/lib/i18n";
5 +import { getAuthor, listPapers } from "@/lib/papers/queries";
6 +import { PaperRow } from "@/components/paper-items";
7 +export const revalidate = 300;
8 +export async function generateMetadata({ params }: { params: Promise<{ slug: string }> }): Promise<Metadata> { const { slug } = await params; const a = await getAuthor(slug); return a ? { title: a.display_name, alternates: { canonical: `/authors/${slug}` } } : {}; }
9 +export default async function AuthorPage({ params }: { params: Promise<{ slug: string }> }) {
10 + const { slug } = await params; const locale = await getLocale(); const d = t(locale); const a = await getAuthor(slug); if (!a) notFound();
11 + const res = await listPapers({ author: [slug], sort: "recent", perPage: 100 });
12 + const links: [string, string | null][] = [[d.authors.website, a.website], [d.authors.orcid, a.orcid ? `https://orcid.org/${a.orcid}` : null], [d.authors.scholar, a.google_scholar], [d.authors.ssrn, a.ssrn_url], [d.authors.github, a.github]];
13 + return (<div className="container py-10"><div className="grid gap-8 lg:grid-cols-[1fr_18rem]"><div><p className="kicker">{d.papers.author}</p><h1 className="text-3xl sm:text-4xl mt-2">{a.display_name}</h1>
14 + {(locale === "fr" ? a.affiliation_fr : a.affiliation_en) && <p className="text-fg-2 mt-2">{locale === "fr" ? a.affiliation_fr : a.affiliation_en}{(locale === "fr" ? a.department_fr : a.department_en) ? ` · ${locale === "fr" ? a.department_fr : a.department_en}` : ""}</p>}
15 + {(locale === "fr" ? a.bio_fr : a.bio_en) && <p className="mt-4 text-fg-2 leading-relaxed max-w-3xl">{locale === "fr" ? a.bio_fr : a.bio_en}</p>}
16 + <p className="meta mt-6">{res.total} {res.total === 1 ? d.topics.paper : d.authors.papers}</p><div className="card px-5 mt-3">{res.items.map((p) => <PaperRow key={p.id} p={p} locale={locale} />)}</div></div>
17 + <aside className="card p-4 text-sm self-start"><ul className="space-y-2">{links.filter(([, v]) => v).map(([k, v]) => <li key={k}><span className="label block">{k}</span><a href={v!} rel="me noopener" className="text-navy break-all">{v}</a></li>)}{a.email && <li><span className="label block">E-mail</span><a href={`mailto:${a.email}`} className="text-navy">{a.email}</a></li>}</ul></aside></div></div>);
18 +}
added src/app/(public)/[locale]/authors/page.tsx +12 −0
@@ -0,0 +1,12 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { href, t } from "@/lib/i18n";
5 +import { authorsWithCounts } from "@/lib/papers/queries";
6 +export const revalidate = 300;
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.authors.title, description: d.authors.lead, alternates: { canonical: "/authors" } }; }
8 +export default async function AuthorsPage() {
9 + const locale = await getLocale(); const d = t(locale); const rows = await authorsWithCounts();
10 + return (<div className="container py-10"><h1 className="text-3xl sm:text-4xl">{d.authors.title}</h1><p className="meta mt-2">{d.authors.lead}</p>
11 + <ul className="grid gap-4 sm:grid-cols-2 lg:grid-cols-3 mt-8">{rows.map((a) => <li key={a.slug} className="card p-5"><h2 className="text-xl"><Link href={href(locale, `/authors/${a.slug}`)} className="text-fg no-underline hover:underline">{a.display_name}</Link></h2>{(locale === "fr" ? a.affiliation_fr : a.affiliation_en) && <p className="text-sm text-fg-2 mt-1">{locale === "fr" ? a.affiliation_fr : a.affiliation_en}</p>}<p className="meta mt-2">{a.n} {a.n === 1 ? d.topics.paper : d.authors.papers}</p></li>)}</ul></div>);
12 +}
added src/app/(public)/[locale]/citation/page.tsx +21 −0
@@ -0,0 +1,21 @@
1 +import type { Metadata } from "next";
2 +import { getLocale } from "@/lib/i18n/server";
3 +import { t } from "@/lib/i18n";
4 +import { APP_URL } from "@/lib/env";
5 +export const revalidate = 3600;
6 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.citation.title, alternates: { canonical: "/citation" } }; }
7 +export default async function CitationPage() {
8 + const locale = await getLocale(); const d = t(locale); const fr = locale === "fr";
9 + return (<article className="container py-10 max-w-3xl prose-academic"><h1 className="text-3xl sm:text-4xl mb-6">{d.citation.title}</h1>
10 + <h2 className="text-2xl mt-6 mb-3">{fr ? "Comment citer un cahier" : "How to cite a working paper"}</h2>
11 + <p>{fr ? "Chaque fiche propose une citation APA et Chicago, ainsi que des exports BibTeX et RIS. Citez l'identifiant permanent (p. ex. WP38), la série, l'institution, la version et l'URL canonique :" : "Every record offers APA and Chicago citations plus BibTeX and RIS exports. Cite the permanent identifier (e.g. WP38), the series, the institution, the version and the canonical URL:"}</p>
12 + <pre className="card p-4 text-sm overflow-x-auto"><code>{`Boucher, S.-P. (2026). Title of the paper (UQO Working Papers WP38, v1.0). Université du Québec en Outaouais. ${APP_URL}/papers/wp38`}</code></pre>
13 + <p>{fr ? "Les URL /papers/wpNN sont stables ; une version précise est adressable par /papers/wpNN/vK. Les exports machine : " : "URLs of the form /papers/wpNN are stable; a specific version is addressable at /papers/wpNN/vK. Machine exports: "}<code>/api/v1/papers/wp38/citation.bib</code>, <code>/api/v1/papers/wp38/citation.ris</code>.</p>
14 + <h2 className="text-2xl mt-8 mb-3">{fr ? "Réutilisation" : "Reuse"}</h2>
15 + <p>{fr ? "Les manuscrits demeurent la propriété intellectuelle de leurs auteurs. Ils peuvent être téléchargés, lus et cités librement aux fins de recherche et d'enseignement. Toute reproduction substantielle, traduction ou redistribution commerciale requiert l'accord écrit des auteurs. Les métadonnées des fiches (titres, résumés, mots-clés, classification) peuvent être réutilisées avec attribution." : "Manuscripts remain the intellectual property of their authors. They may be downloaded, read and cited freely for research and teaching. Substantial reproduction, translation or commercial redistribution requires the written consent of the authors. Record metadata (titles, abstracts, keywords, classification) may be reused with attribution."}</p>
16 + <h2 className="text-2xl mt-8 mb-3">{fr ? "Données et code" : "Data and code"}</h2>
17 + <p>{fr ? "Lorsqu'un dépôt de code ou des notes de reproductibilité sont indiqués sur la fiche, ils s'appliquent au manuscrit cité. Les jeux de données propriétaires ne sont pas redistribués ; les conditions d'accès figurent dans le catalogue des données." : "Where a code repository or reproducibility notes are listed on a record, they apply to the cited manuscript. Proprietary datasets are not redistributed; access conditions are stated in the data catalog."}</p>
18 + <h2 className="text-2xl mt-8 mb-3">{fr ? "Moissonnage" : "Harvesting"}</h2>
19 + <p>OAI-PMH: <code>{APP_URL}/api/oai?verb=ListRecords&metadataPrefix=oai_dc</code> · RSS: <code>{APP_URL}/feed.xml</code> · API: <code>{APP_URL}/api/v1/papers</code>.</p>
20 + </article>);
21 +}
added src/app/(public)/[locale]/datasets/[slug]/page.tsx +17 −0
@@ -0,0 +1,17 @@
1 +import type { Metadata } from "next";
2 +import { notFound } from "next/navigation";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { t } from "@/lib/i18n";
5 +import { getDataset, listPapers } from "@/lib/papers/queries";
6 +import { PaperRow } from "@/components/paper-items";
7 +export const revalidate = 300;
8 +export async function generateMetadata({ params }: { params: Promise<{ slug: string }> }): Promise<Metadata> { const { slug } = await params; const x = await getDataset(slug); const locale = await getLocale(); return x ? { title: locale === "fr" ? x.name_fr : x.name_en, alternates: { canonical: `/datasets/${slug}` } } : {}; }
9 +export default async function DatasetPage({ params }: { params: Promise<{ slug: string }> }) {
10 + const { slug } = await params; const locale = await getLocale(); const d = t(locale); const x = await getDataset(slug); if (!x) notFound();
11 + const res = await listPapers({ dataset: [slug], sort: "recent", perPage: 100 });
12 + const rows: [string, string | null][] = [[d.datasets.provider, x.provider], [d.datasets.coverage, x.coverage], [d.datasets.frequency, x.frequency], [d.datasets.access, x.access_conditions], [d.datasets.license, x.license]];
13 + return (<div className="container py-10"><p className="kicker">{d.papers.dataset}</p><h1 className="text-3xl sm:text-4xl mt-2">{locale === "fr" ? x.name_fr : x.name_en}</h1>
14 + <dl className="card p-5 mt-6 grid gap-3 sm:grid-cols-2 text-sm max-w-3xl">{rows.filter(([, v]) => v).map(([k, v]) => <div key={k}><dt className="label">{k}</dt><dd className="text-fg-2">{v}</dd></div>)}{x.url && <div><dt className="label">URL</dt><dd><a href={x.url} rel="noopener" className="text-navy">{x.url}</a></dd></div>}<div><dt className="label">{d.datasets.access}</dt><dd><span className={`chip ${x.is_public ? "" : "chip-burgundy"}`}>{x.is_public ? d.datasets.public : d.datasets.restricted}</span></dd></div></dl>
15 + {(locale === "fr" ? x.description_fr : x.description_en) && <p className="mt-4 text-fg-2 max-w-3xl">{locale === "fr" ? x.description_fr : x.description_en}</p>}
16 + <p className="meta mt-6">{res.total} {res.total === 1 ? d.topics.paper : d.topics.papers}</p><div className="card px-5 mt-3">{res.items.map((p) => <PaperRow key={p.id} p={p} locale={locale} />)}</div></div>);
17 +}
added src/app/(public)/[locale]/datasets/page.tsx +17 −0
@@ -0,0 +1,17 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { href, t } from "@/lib/i18n";
5 +import { datasetsWithCounts } from "@/lib/papers/queries";
6 +export const revalidate = 300;
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.datasets.title, description: d.datasets.lead, alternates: { canonical: "/datasets" } }; }
8 +export default async function DatasetsPage() {
9 + const locale = await getLocale(); const d = t(locale); const rows = await datasetsWithCounts();
10 + return (
11 + <div className="container py-10">
12 + <h1 className="text-3xl sm:text-4xl">{d.datasets.title}</h1><p className="meta mt-2 max-w-3xl">{d.datasets.lead}</p>
13 + <div className="overflow-x-auto card mt-8"><table className="data"><thead><tr><th>{d.papers.dataset}</th><th>{d.datasets.provider}</th><th>{d.datasets.coverage}</th><th>{d.datasets.frequency}</th><th>{d.datasets.access}</th><th>{d.topics.papers}</th></tr></thead>
14 + <tbody>{rows.map((x) => <tr key={x.slug}><td><Link href={href(locale, `/datasets/${x.slug}`)} className="text-fg no-underline hover:underline font-medium">{locale === "fr" ? x.name_fr : x.name_en}</Link></td><td className="text-fg-2">{x.provider ?? "—"}</td><td className="text-fg-2">{x.coverage ?? "—"}</td><td className="text-fg-2">{x.frequency ?? "—"}</td><td><span className={`chip ${x.is_public ? "" : "chip-burgundy"}`}>{x.is_public ? d.datasets.public : d.datasets.restricted}</span></td><td className="whitespace-nowrap">{(x.papers as { slug: string; identifier: string }[]).map((p) => <Link key={p.slug} href={href(locale, `/papers/${p.slug}`)} className="chip no-underline font-mono mr-1">{p.identifier}</Link>)}</td></tr>)}</tbody></table></div>
15 + </div>
16 + );
17 +}
added src/app/(public)/[locale]/layout.tsx +18 −0
@@ -0,0 +1,18 @@
1 +import { notFound } from "next/navigation";
2 +import { SiteHeader } from "@/components/site-header";
3 +import { SiteFooter } from "@/components/site-footer";
4 +import { siteSetting } from "@/lib/papers/queries";
5 +import { t, type Locale } from "@/lib/i18n";
6 +export function generateStaticParams() { return [{ locale: "en" }, { locale: "fr" }]; }
7 +export default async function PublicLayout({ children, params }: { children: React.ReactNode; params: Promise<{ locale: string }> }) {
8 + const { locale: raw } = await params; if (raw !== "en" && raw !== "fr") notFound(); const locale = raw as Locale; const d = t(locale);
9 + const [attribution, institution] = await Promise.all([siteSetting<{ en: string; fr: string }>("site.attribution"), siteSetting<{ name_en: string; name_fr: string; department_en?: string; department_fr?: string; url?: string }>("site.institution")]);
10 + return (
11 + <>
12 + <a href="#main" className="skip-link">{d.nav.skip}</a>
13 + <SiteHeader locale={locale} />
14 + <main id="main" className="flex-1">{children}</main>
15 + <SiteFooter locale={locale} attribution={attribution} institution={institution} />
16 + </>
17 + );
18 +}
added src/app/(public)/[locale]/methods/[slug]/page.tsx +13 −0
@@ -0,0 +1,13 @@
1 +import type { Metadata } from "next";
2 +import { notFound } from "next/navigation";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { t } from "@/lib/i18n";
5 +import { getMethod, listPapers } from "@/lib/papers/queries";
6 +import { PaperRow } from "@/components/paper-items";
7 +export const revalidate = 300;
8 +export async function generateMetadata({ params }: { params: Promise<{ slug: string }> }): Promise<Metadata> { const { slug } = await params; const x = await getMethod(slug); const locale = await getLocale(); return x ? { title: locale === "fr" ? x.name_fr : x.name_en, alternates: { canonical: `/methods/${slug}` } } : {}; }
9 +export default async function MethodPage({ params }: { params: Promise<{ slug: string }> }) {
10 + const { slug } = await params; const locale = await getLocale(); const d = t(locale); const x = await getMethod(slug); if (!x) notFound();
11 + const res = await listPapers({ method: [slug], sort: "recent", perPage: 100 });
12 + return (<div className="container py-10"><p className="kicker">{d.papers.method} · {x.category}</p><h1 className="text-3xl sm:text-4xl mt-2">{locale === "fr" ? x.name_fr : x.name_en}</h1>{(locale === "fr" ? x.description_fr : x.description_en) && <p className="meta mt-2 max-w-3xl">{locale === "fr" ? x.description_fr : x.description_en}</p>}<p className="meta mt-3">{res.total} {res.total === 1 ? d.topics.paper : d.topics.papers}</p><div className="card px-5 mt-6">{res.items.map((p) => <PaperRow key={p.id} p={p} locale={locale} />)}</div></div>);
13 +}
added src/app/(public)/[locale]/methods/page.tsx +17 −0
@@ -0,0 +1,17 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { href, t } from "@/lib/i18n";
5 +import { methodsWithCounts } from "@/lib/papers/queries";
6 +export const revalidate = 300;
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.methods.title, description: d.methods.lead, alternates: { canonical: "/methods" } }; }
8 +export default async function MethodsPage() {
9 + const locale = await getLocale(); const d = t(locale); const rows = await methodsWithCounts();
10 + const cats = [...new Set(rows.map((r) => r.category))];
11 + return (
12 + <div className="container py-10">
13 + <h1 className="text-3xl sm:text-4xl">{d.methods.title}</h1><p className="meta mt-2 max-w-3xl">{d.methods.lead}</p>
14 + {cats.map((c) => <section key={c} className="mt-10"><h2 className="label mb-3">{c}</h2><ul className="grid gap-3 sm:grid-cols-2 lg:grid-cols-3">{rows.filter((r) => r.category === c).map((m) => <li key={m.slug} className="card p-4"><Link href={href(locale, `/methods/${m.slug}`)} className="text-fg no-underline hover:underline font-medium">{locale === "fr" ? m.name_fr : m.name_en}</Link><p className="meta mt-1">{m.n} {m.n === 1 ? d.topics.paper : d.topics.papers}</p><p className="mt-2 flex flex-wrap gap-1">{(m.papers as { slug: string; identifier: string }[]).slice(0, 8).map((p) => <Link key={p.slug} href={href(locale, `/papers/${p.slug}`)} className="chip no-underline font-mono">{p.identifier}</Link>)}</p></li>)}</ul></section>)}
15 + </div>
16 + );
17 +}
added src/app/(public)/[locale]/page.tsx +69 −0
@@ -0,0 +1,69 @@
1 +import Link from "next/link";
2 +import { ArrowRight, Search } from "lucide-react";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { fmtDate, href, t } from "@/lib/i18n";
5 +import { domainsWithCounts, graph, listPapers, stats } from "@/lib/papers/queries";
6 +import { PaperCard, PaperRow } from "@/components/paper-items";
7 +import { StatTiles } from "@/components/stat-tiles";
8 +import { Constellation, type GEdge, type GNode } from "@/components/constellation";
9 +export const revalidate = 300;
10 +
11 +export default async function Home() {
12 + const locale = await getLocale(); const d = t(locale);
13 + const [s, featured, latest, domains, g] = await Promise.all([stats(), listPapers({ featured: true, sort: "recent", perPage: 4 }), listPapers({ sort: "revised", perPage: 6 }), domainsWithCounts(), graph(["topic", "method", "dataset", "author", "semantic", "curated"])]);
14 + const featuredItems = [...featured.items].sort((a, b) => (a.featured_rank ?? 99) - (b.featured_rank ?? 99));
15 + return (
16 + <>
17 + <section className="border-b border-line bg-bg-2/60">
18 + <div className="container py-14 sm:py-20 grid gap-8 lg:grid-cols-[1.4fr_1fr] items-end">
19 + <div>
20 + <p className="kicker mb-3">{d.site.secondary} · {d.site.affiliation}</p>
21 + <h1 className="text-4xl sm:text-5xl leading-[1.08] text-fg">{d.home.heroTitle}</h1>
22 + <p className="mt-5 text-lg text-fg-2 max-w-2xl leading-relaxed">{d.home.heroLead}</p>
23 + <form action={href(locale, "/search")} method="get" role="search" className="mt-7 flex gap-2 max-w-2xl">
24 + <label htmlFor="home-q" className="sr-only">{d.nav.search}</label>
25 + <input id="home-q" name="q" className="input" placeholder={d.home.searchPlaceholder} autoComplete="off" />
26 + <button type="submit" className="btn btn-primary shrink-0"><Search size={16} aria-hidden /> <span className="hidden sm:inline">{d.search.go}</span></button>
27 + </form>
28 + <div className="mt-5 flex flex-wrap gap-2">
29 + <Link href={href(locale, "/papers")} className="btn btn-primary no-underline">{d.home.explore} <ArrowRight size={16} aria-hidden /></Link>
30 + <Link href={href(locale, "/topics")} className="btn no-underline">{d.home.browseTopics}</Link>
31 + </div>
32 + </div>
33 + <div className="text-sm text-fg-3 lg:text-right"><p className="serif text-fg text-lg italic">{d.site.tagline}</p></div>
34 + </div>
35 + </section>
36 +
37 + <section className="container py-12">
38 + <div className="flex items-end justify-between gap-4 mb-5"><div><h2 className="text-2xl">{d.home.stats}</h2><p className="meta mt-1">{d.home.statsLead}</p></div></div>
39 + <StatTiles s={s} locale={locale} />
40 + </section>
41 +
42 + {featuredItems.length > 0 && (
43 + <section className="container py-6">
44 + <div className="flex items-end justify-between gap-4 mb-5"><div><h2 className="text-2xl">{d.home.featured}</h2><p className="meta mt-1">{d.home.featuredLead}</p></div><Link href={href(locale, "/papers")} className="text-sm text-navy no-underline hover:underline whitespace-nowrap">{d.home.viewAll} →</Link></div>
45 + <div className="grid gap-4 md:grid-cols-2">{featuredItems.map((p) => <PaperCard key={p.id} p={p} locale={locale} />)}</div>
46 + </section>
47 + )}
48 +
49 + <section className="container py-10 grid gap-10 lg:grid-cols-[1.5fr_1fr]">
50 + <div>
51 + <div className="flex items-end justify-between gap-4 mb-2"><div><h2 className="text-2xl">{d.home.latest}</h2><p className="meta mt-1">{d.home.latestLead}</p></div><Link href={href(locale, "/papers?sort=revised")} className="text-sm text-navy no-underline hover:underline whitespace-nowrap">{d.home.viewAll} →</Link></div>
52 + <div className="card px-5">{latest.items.map((p) => <PaperRow key={p.id} p={p} locale={locale} compact />)}</div>
53 + </div>
54 + <div>
55 + <h2 className="text-2xl">{d.home.domains}</h2><p className="meta mt-1 mb-4">{d.home.domainsLead}</p>
56 + <ul className="grid gap-2">
57 + {domains.map((dm) => <li key={dm.slug}><Link href={href(locale, `/topics/${dm.slug}`)} className="card px-4 py-3 flex items-center gap-3 no-underline hover:shadow-[var(--shadow-lift)] transition-shadow"><span className="w-2.5 h-2.5 rounded-full shrink-0" style={{ background: dm.accent ?? "var(--navy)" }} /><span className="flex-1 text-fg">{locale === "fr" ? dm.name_fr : dm.name_en}</span><span className="text-xs text-fg-3 font-mono">{dm.n}</span></Link></li>)}
58 + </ul>
59 + </div>
60 + </section>
61 +
62 + <section className="container py-10">
63 + <div className="flex items-end justify-between gap-4 mb-5"><div><h2 className="text-2xl">{d.home.constellation}</h2><p className="meta mt-1 max-w-3xl">{d.home.constellationLead}</p></div><Link href={href(locale, "/research-map")} className="text-sm text-navy no-underline hover:underline whitespace-nowrap">{d.home.openMap} →</Link></div>
64 + <Constellation locale={locale} nodes={g.nodes as unknown as GNode[]} edges={g.edges as unknown as GEdge[]} compact />
65 + </section>
66 + <p className="container text-xs text-fg-3 pb-4">{d.stats.latest}: {s.latest ? fmtDate(locale, s.latest, "month") : "—"}</p>
67 + </>
68 + );
69 +}
added src/app/(public)/[locale]/papers/[id]/[version]/page.tsx +33 −0
@@ -0,0 +1,33 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { notFound, permanentRedirect } from "next/navigation";
4 +import { getLocale } from "@/lib/i18n/server";
5 +import { fmtDate, href, t } from "@/lib/i18n";
6 +import { getPaper } from "@/lib/papers/queries";
7 +import { parseIdentifier } from "@/lib/papers/identifiers";
8 +import { AuthorsLine } from "@/components/paper-items";
9 +export async function generateMetadata({ params }: { params: Promise<{ id: string; version: string }> }): Promise<Metadata> { const { id, version } = await params; const p = await getPaper(id); return p ? { title: `${p.identifier} ${version} · ${p.title}`, alternates: { canonical: `/papers/${p.slug}` }, robots: { index: false, follow: true } } : {}; }
10 +export default async function VersionPage({ params }: { params: Promise<{ id: string; version: string }> }) {
11 + const { id, version } = await params; const locale = await getLocale(); const d = t(locale);
12 + const m = /^v(\d+)$/i.exec(version); if (!m) notFound();
13 + const parsed = parseIdentifier(id); if (parsed && parsed.slug !== id) permanentRedirect(`/papers/${parsed.slug}/${version.toLowerCase()}`);
14 + const p = await getPaper(id); if (!p) notFound();
15 + const v = p.versions.find((x) => x.version_no === Number(m[1])); if (!v) notFound();
16 + return (
17 + <article className="container py-10 max-w-3xl">
18 + <p className="kicker"><Link href={href(locale, `/papers/${p.slug}`)} className="no-underline hover:underline">{p.identifier}</Link> · {v.label}{v.is_current ? ` · ${d.paper.versionTable.current}` : ""}</p>
19 + <h1 className="mt-3 text-3xl leading-tight">{p.title}</h1>
20 + <p className="mt-3 text-fg-2"><AuthorsLine authors={p.authors} locale={locale} /></p>
21 + <dl className="mt-6 grid gap-3 sm:grid-cols-2 text-sm card p-5">
22 + <div><dt className="label">{d.paper.versionTable.version}</dt><dd>{v.label} (#{v.version_no})</dd></div>
23 + <div><dt className="label">{d.paper.versionTable.date}</dt><dd>{fmtDate(locale, v.released_on, v.released_precision as "day" | "month" | "year") || "—"}</dd></div>
24 + <div className="sm:col-span-2"><dt className="label">{d.paper.versionTable.description}</dt><dd className="text-fg-2">{locale === "fr" ? v.description_fr ?? v.description : v.description}</dd></div>
25 + <div><dt className="label">{d.paper.pagesLabel}</dt><dd>{v.pages ?? "—"}</dd></div>
26 + <div><dt className="label">{d.paper.sha}</dt><dd className="font-mono text-xs break-all">{v.sha256 ?? "—"}</dd></div>
27 + </dl>
28 + {v.withdrawn ? <p className="card p-4 mt-6 text-burgundy">{d.paper.versionTable.withdrawn}{v.withdrawn_reason ? ` — ${v.withdrawn_reason}` : ""}</p>
29 + : v.file_id ? <a href={`/api/files/${v.file_id}/${p.slug}-${v.label}.pdf`} className="btn btn-primary mt-6 no-underline">{d.paper.download} ({v.label})</a> : null}
30 + <p className="mt-8"><Link href={href(locale, `/papers/${p.slug}`)} className="text-navy no-underline hover:underline">← {d.misc.back}</Link></p>
31 + </article>
32 + );
33 +}
added src/app/(public)/[locale]/papers/[id]/page.tsx +127 −0
@@ -0,0 +1,127 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { notFound, permanentRedirect } from "next/navigation";
4 +import { BookOpen, Download, History } from "lucide-react";
5 +import { getLocale } from "@/lib/i18n/server";
6 +import { fmtDate, fmtInt, href, t, type Locale } from "@/lib/i18n";
7 +import { findRedirect, getPaper } from "@/lib/papers/queries";
8 +import { parseIdentifier } from "@/lib/papers/identifiers";
9 +import { citationApa, citationChicago } from "@/lib/papers/citation";
10 +import { citable, jsonLd, scholarMeta } from "@/lib/papers/meta";
11 +import type { PaperFull } from "@/lib/papers/types";
12 +import { AuthorsLine, StatusChips } from "@/components/paper-items";
13 +import { CiteMenu } from "@/components/cite-menu";
14 +import { ViewBeacon } from "@/components/view-beacon";
15 +import { APP_URL } from "@/lib/env";
16 +export const revalidate = 120;
17 +
18 +async function load(id: string) {
19 + const parsed = parseIdentifier(id); const p = await getPaper(id);
20 + if (!p) { const r = await findRedirect(`/papers/${id}`); if (r) permanentRedirect(r.to_path); notFound(); }
21 + if (parsed && parsed.slug !== id) permanentRedirect(`/papers/${parsed.slug}`);
22 + return p;
23 +}
24 +export async function generateMetadata({ params }: { params: Promise<{ id: string }> }): Promise<Metadata> {
25 + const { id } = await params; const p = await getPaper(id); if (!p) return {};
26 + const other = scholarMeta(p);
27 + return { title: `${p.identifier} · ${p.title}`, description: p.abstract[0]?.slice(0, 300), alternates: { canonical: `/papers/${p.slug}`, languages: { en: `/papers/${p.slug}`, fr: `/fr/papers/${p.slug}` } }, openGraph: { type: "article", title: p.title, description: p.abstract[0]?.slice(0, 300), url: `${APP_URL}/papers/${p.slug}`, images: [`/api/preview/${p.slug}.png`] }, other };
28 +}
29 +
30 +function Section({ id, title, children }: { id: string; title: string; children: React.ReactNode }) { return <section id={id} className="mt-10"><h2 className="text-xl sm:text-2xl mb-3">{title}</h2>{children}</section>; }
31 +function Pairs({ items, locale }: { items: { title: string; description: string; kind?: string }[]; locale: Locale }) {
32 + return <dl className="grid gap-3 sm:grid-cols-2">{items.map((c, i) => <div key={i} className="card p-4"><dt className="font-medium text-fg">{c.kind ? <span className="label block mb-1">{c.kind}</span> : null}{c.title}</dt><dd className="text-sm text-fg-2 mt-1 leading-relaxed">{c.description}</dd></div>)}</dl>;
33 +}
34 +const lname = (x: { name_en: string; name_fr: string }, locale: Locale) => (locale === "fr" ? x.name_fr : x.name_en);
35 +
36 +export default async function PaperPage({ params }: { params: Promise<{ id: string }> }) {
37 + const { id } = await params; const locale = await getLocale(); const d = t(locale); const p: PaperFull = await load(id);
38 + const c = citable(p, locale); const apa = citationApa(c), chicago = citationChicago(c);
39 + const pdfUrl = p.has_pdf && p.file_id ? `/api/files/${p.file_id}/${p.slug}-${p.version_label ?? "current"}.pdf` : null;
40 + const abstract = locale === "fr" && p.abstract_fr?.length ? p.abstract_fr : p.abstract; const frOnly = locale === "fr" && !p.abstract_fr?.length;
41 + const ld = jsonLd(p);
42 + return (
43 + <article className="container py-10">
44 + <script type="application/ld+json" dangerouslySetInnerHTML={{ __html: JSON.stringify(ld) }} />
45 + <ViewBeacon slug={p.slug} />
46 + {p.status === "withdrawn" && <p className="card p-4 mb-6 border-burgundy text-burgundy">{d.paper.withdrawnNotice}</p>}
47 + {p.status === "superseded" && <p className="card p-4 mb-6">{d.paper.supersededNotice}</p>}
48 + <header>
49 + <p className="kicker">{locale === "fr" ? p.series_name_fr : p.series_name_en} · <span className="font-mono">{p.identifier}</span>{p.version_label ? ` · ${p.version_label}` : ""}</p>
50 + <h1 className="mt-3 text-3xl sm:text-4xl leading-tight text-fg">{locale === "fr" && p.title_fr ? p.title_fr : p.title}</h1>
51 + {p.subtitle && <p className="mt-2 text-lg text-fg-2 serif">{p.subtitle}</p>}
52 + <p className="mt-4 text-fg-2"><AuthorsLine authors={p.authors} locale={locale} /></p>
53 + <dl className="mt-5 grid gap-x-6 gap-y-2 grid-cols-2 sm:grid-cols-4 text-sm">
54 + <div><dt className="label">{d.paper.year}</dt><dd>{p.year ?? "—"}</dd></div>
55 + <div><dt className="label">{d.paper.released}</dt><dd>{fmtDate(locale, p.first_released_on, p.first_released_precision) || "—"}</dd></div>
56 + <div><dt className="label">{d.paper.revised}</dt><dd>{fmtDate(locale, p.last_revised_on, p.last_revised_precision) || "—"}</dd></div>
57 + <div><dt className="label">{d.paper.pubStatus}</dt><dd>{d.misc.pubStatus[p.publication_status as keyof typeof d.misc.pubStatus]}{p.journal_ref ? ` — ${p.journal_ref}` : ""}</dd></div>
58 + </dl>
59 + <div className="mt-3"><StatusChips p={p} locale={locale} /></div>
60 + <div className="mt-6 flex flex-wrap gap-2">
61 + {pdfUrl ? <a href={pdfUrl} className="btn btn-primary no-underline"><Download size={16} aria-hidden /> {d.paper.download}{p.pages ? ` (${p.pages} ${d.papers.pages})` : ""}</a> : <span className="btn opacity-60 cursor-not-allowed">{d.paper.notAvailable}</span>}
62 + {pdfUrl && <Link href={href(locale, `/papers/${p.slug}/read`)} className="btn no-underline"><BookOpen size={16} aria-hidden /> {d.paper.read}</Link>}
63 + <CiteMenu locale={locale} slug={p.slug} apa={apa} chicago={chicago} shareUrl={`${APP_URL}/papers/${p.slug}`} />
64 + {p.versions.length > 1 && <a href="#versions" className="btn no-underline"><History size={16} aria-hidden /> {d.paper.previousVersions}</a>}
65 + </div>
66 + </header>
67 +
68 + <div className="grid gap-10 lg:grid-cols-[1fr_19rem] mt-10">
69 + <div className="min-w-0">
70 + {p.hero_headline && <p className="serif text-xl text-fg leading-snug border-l-2 border-burgundy pl-4">{p.hero_headline}</p>}
71 + {p.hero_subheadline && <p className="text-fg-2 mt-2 pl-4 text-sm leading-relaxed">{p.hero_subheadline}</p>}
72 + <Section id="abstract" title={d.paper.abstract}>
73 + {frOnly && <p className="meta mb-2 italic">{d.paper.frOnly}</p>}
74 + <div className="prose-academic">{abstract.map((para, i) => <p key={i}>{para}</p>)}</div>
75 + </Section>
76 + {(locale === "fr" ? p.research_question_fr ?? p.research_question : p.research_question) && <Section id="question" title={d.paper.question}><p className="prose-academic">{locale === "fr" ? p.research_question_fr ?? p.research_question : p.research_question}</p></Section>}
77 + {p.findings.length > 0 && <Section id="findings" title={d.paper.findings}><ul className="grid gap-3 sm:grid-cols-2 lg:grid-cols-3">{p.findings.map((f, i) => <li key={i} className="card p-4"><p className="serif text-xl text-navy">{f.value}</p><p className="text-sm text-fg-2 mt-1 leading-snug">{f.label}</p></li>)}</ul></Section>}
78 + {p.contributions.length > 0 && <Section id="contributions" title={d.paper.contributions}><Pairs items={p.contributions} locale={locale} /></Section>}
79 + {p.methodology.length > 0 && <Section id="methodology" title={d.paper.methodology}><details open className="card p-4"><summary className="font-medium">{d.paper.methodology} ({p.methodology.length})</summary><div className="mt-3 grid gap-3">{p.methodology.map((m, i) => <div key={i}><p className="font-medium text-fg">{m.title}</p><p className="text-sm text-fg-2 mt-0.5 leading-relaxed">{m.description}</p></div>)}</div></details></Section>}
80 + {p.data_sections.length > 0 && <Section id="data" title={d.paper.data}><Pairs items={p.data_sections} locale={locale} />
81 + {p.datasets.length > 0 && <p className="meta mt-3">{d.paper.datasetsTag}: {p.datasets.map((x, i) => <span key={x.slug}>{i > 0 && ", "}<Link href={href(locale, `/datasets/${x.slug}`)} className="text-navy no-underline hover:underline">{lname(x, locale)}</Link>{x.assigned_by === "inferred" ? ` (${d.paper.inferred})` : ""}</span>)}</p>}
82 + </Section>}
83 + {p.reproducibility.length > 0 && <Section id="reproducibility" title={d.paper.reproducibility}><ul className="list-disc pl-5 space-y-1 text-fg-2 text-sm leading-relaxed">{p.reproducibility.map((r, i) => <li key={i}>{r}</li>)}</ul></Section>}
84 + <Section id="artifacts" title={d.paper.artifacts}>
85 + <ul className="grid gap-2 sm:grid-cols-2 text-sm">
86 + {pdfUrl && <li className="card px-4 py-3"><a href={pdfUrl} className="text-navy no-underline hover:underline">{d.paper.download}</a><span className="meta block">PDF · {p.version_label} · SHA-256 <code className="text-xs">{p.sha256?.slice(0, 16)}…</code></span></li>}
87 + {p.artifacts.map((a, i) => <li key={i} className="card px-4 py-3"><a href={a.url} rel="noopener" className="text-navy no-underline hover:underline">{a.label}</a><span className="meta block">{a.kind}</span></li>)}
88 + {p.ssrn_id && <li className="card px-4 py-3"><a href={`https://ssrn.com/abstract=${p.ssrn_id}`} rel="noopener" className="text-navy no-underline hover:underline">SSRN {p.ssrn_id}</a><span className="meta block">{d.paper.ssrn}</span></li>}
89 + {p.doi && <li className="card px-4 py-3"><a href={`https://doi.org/${p.doi}`} rel="noopener" className="text-navy no-underline hover:underline">{p.doi}</a><span className="meta block">{d.paper.doi}</span></li>}
90 + {p.external_links.filter((l) => l.kind !== "ssrn").map((l, i) => <li key={"x" + i} className="card px-4 py-3"><a href={l.url} rel="noopener" className="text-navy no-underline hover:underline">{l.label}</a><span className="meta block">{l.kind}</span></li>)}
91 + </ul>
92 + </Section>
93 + {p.related.length > 0 && <Section id="related" title={d.paper.related}>
94 + <ul className="grid gap-2">{p.related.map((r) => <li key={r.slug} className="card px-4 py-3"><Link href={href(locale, `/papers/${r.slug}`)} className="text-fg no-underline hover:underline"><span className="font-mono text-navy text-xs mr-2">{r.identifier}</span>{r.title}</Link><p className="meta mt-1">{d.paper.whyRelated} {(locale === "fr" ? r.explanations_fr : r.explanations).filter(Boolean).join(" · ")}{r.inferred ? ` — ${d.paper.inferred}` : ""}</p></li>)}</ul>
95 + </Section>}
96 + <Section id="versions" title={d.paper.versions}>
97 + <div className="overflow-x-auto card"><table className="data"><thead><tr><th>{d.paper.versionTable.version}</th><th>{d.paper.versionTable.date}</th><th>{d.paper.versionTable.description}</th><th>{d.paper.versionTable.file}</th></tr></thead>
98 + <tbody>{p.versions.map((v) => <tr key={v.id}><td className="whitespace-nowrap"><Link href={href(locale, `/papers/${p.slug}/v${v.version_no}`)} className="text-navy no-underline hover:underline">{v.label}</Link>{v.is_current && <span className="chip chip-navy ml-2">{d.paper.versionTable.current}</span>}{v.withdrawn && <span className="chip chip-burgundy ml-2">{d.paper.versionTable.withdrawn}</span>}</td><td className="whitespace-nowrap">{fmtDate(locale, v.released_on, v.released_precision as "day" | "month" | "year") || "—"}</td><td className="text-fg-2">{locale === "fr" ? v.description_fr ?? v.description : v.description}{v.withdrawn && v.withdrawn_reason ? ` — ${v.withdrawn_reason}` : ""}</td><td className="whitespace-nowrap">{v.file_id && !v.withdrawn ? <a href={`/api/files/${v.file_id}/${p.slug}-${v.label}.pdf`} className="text-navy no-underline hover:underline">PDF{v.pages ? ` · ${v.pages} p.` : ""}</a> : "—"}</td></tr>)}</tbody></table></div>
99 + </Section>
100 + </div>
101 + <aside className="space-y-6 text-sm">
102 + <div className="card p-4">
103 + <p className="label mb-2">{d.paper.keywords}</p><p className="flex flex-wrap gap-1.5">{p.keywords.map((k) => <Link key={k} href={href(locale, `/papers?keyword=${encodeURIComponent(k)}`)} className="chip no-underline">{k}</Link>)}</p>
104 + {p.jel.length > 0 && <><p className="label mt-4 mb-1">{d.paper.jel}</p><p className="font-mono text-fg-2">{p.jel.join(", ")}</p></>}
105 + </div>
106 + <div className="card p-4">
107 + <p className="label mb-2">{d.paper.topics}</p><p className="flex flex-wrap gap-1.5">{p.topics.map((x) => <Link key={x.slug} href={href(locale, `/topics/${x.slug}`)} className={`chip no-underline ${x.kind === "domain" ? "chip-navy" : ""}`}>{lname(x, locale)}{x.assigned_by === "inferred" ? "*" : ""}</Link>)}</p>
108 + {p.methods.length > 0 && <><p className="label mt-4 mb-2">{d.paper.methodsTag}</p><p className="flex flex-wrap gap-1.5">{p.methods.map((x) => <Link key={x.slug} href={href(locale, `/methods/${x.slug}`)} className="chip no-underline">{lname(x, locale)}</Link>)}</p></>}
109 + {p.topics.some((x) => x.assigned_by === "inferred") && <p className="meta mt-2">* {d.paper.inferred}</p>}
110 + </div>
111 + <div className="card p-4">
112 + <p className="label mb-2">{d.paper.authors}</p>
113 + <ul className="space-y-2">{p.authors.map((a) => <li key={a.slug}><Link href={href(locale, `/authors/${a.slug}`)} className="text-fg no-underline hover:underline">{a.display_name}</Link></li>)}</ul>
114 + <p className="label mt-4 mb-1">{d.paper.affiliation}</p><p className="text-fg-2">{d.site.affiliation}</p>
115 + </div>
116 + <div className="card p-4 text-fg-2">
117 + <p><span className="label">{d.paper.identifier}</span><br /><span className="font-mono">{p.identifier}</span></p>
118 + <p className="mt-2"><span className="label">{d.paper.series}</span><br /><Link href={href(locale, "/series")} className="no-underline hover:underline">{locale === "fr" ? p.series_name_fr : p.series_name_en}</Link></p>
119 + {p.internal_code && <p className="mt-2"><span className="label">Internal code</span><br /><span className="font-mono text-xs">{p.internal_code}</span></p>}
120 + <p className="mt-2"><span className="label">{d.paper.viewsLabel} / {d.paper.downloadsLabel}</span><br />{fmtInt(locale, p.view_count)} / {fmtInt(locale, p.download_count)}</p>
121 + <p className="mt-2 text-xs">{d.papers.popularityNote}</p>
122 + </div>
123 + </aside>
124 + </div>
125 + </article>
126 + );
127 +}
added src/app/(public)/[locale]/papers/[id]/read/page.tsx +20 −0
@@ -0,0 +1,20 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { notFound } from "next/navigation";
4 +import { getLocale } from "@/lib/i18n/server";
5 +import { href, t } from "@/lib/i18n";
6 +import { getPaper } from "@/lib/papers/queries";
7 +export async function generateMetadata({ params }: { params: Promise<{ id: string }> }): Promise<Metadata> { const { id } = await params; const p = await getPaper(id); return p ? { title: `${p.identifier} · ${p.title} — read`, robots: { index: false } } : {}; }
8 +export default async function ReadPage({ params }: { params: Promise<{ id: string }> }) {
9 + const { id } = await params; const locale = await getLocale(); const d = t(locale); const p = await getPaper(id);
10 + if (!p || !p.has_pdf || !p.file_id) notFound();
11 + const url = `/api/files/${p.file_id}/${p.slug}-${p.version_label ?? "current"}.pdf?inline=1`;
12 + return (
13 + <div className="container py-6">
14 + <div className="flex flex-wrap items-center gap-3 mb-3"><Link href={href(locale, `/papers/${p.slug}`)} className="btn btn-sm no-underline">← {p.identifier}</Link><h1 className="text-lg truncate">{p.title}</h1><a href={url.replace("?inline=1", "")} className="btn btn-sm ml-auto no-underline">{d.paper.download}</a></div>
15 + <object data={url} type="application/pdf" className="w-full card" style={{ height: "calc(100dvh - 10rem)" }} aria-label={d.paper.readerTitle}>
16 + <p className="p-6 text-fg-2">{d.paper.readerFallback} <a href={url.replace("?inline=1", "")} className="text-navy">{d.paper.download}</a></p>
17 + </object>
18 + </div>
19 + );
20 +}
added src/app/(public)/[locale]/papers/page.tsx +75 −0
@@ -0,0 +1,75 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { href, t } from "@/lib/i18n";
5 +import { facets, listPapers } from "@/lib/papers/queries";
6 +import type { ListParams } from "@/lib/papers/types";
7 +import { parseListParams, type SP } from "@/lib/papers/params";
8 +import { PaperCard, PaperRow, PaperTable } from "@/components/paper-items";
9 +import { DisplayToggle, type DisplayMode } from "@/components/display-toggle";
10 +import { Pagination } from "@/components/pagination";
11 +import { Suspense } from "react";
12 +export const dynamic = "force-dynamic";
13 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.papers.title, description: d.papers.lead, alternates: { canonical: "/papers" } }; }
14 +
15 +export default async function PapersPage({ searchParams }: { searchParams: Promise<SP> }) {
16 + const sp = await searchParams; const locale = await getLocale(); const d = t(locale);
17 + const params = parseListParams(sp); const view = (["list", "compact", "cards", "table"].includes(String(sp.view)) ? sp.view : "list") as DisplayMode;
18 + const [res, f] = await Promise.all([listPapers(params), facets()]);
19 + const qs = new URLSearchParams(); for (const [k, v] of Object.entries(sp)) if (v !== undefined && k !== "page") (Array.isArray(v) ? v : [v]).forEach((x) => qs.append(k, x));
20 + const has = (k: keyof ListParams, v: string) => ((params[k] as string[] | number[] | undefined) ?? []).map(String).includes(v);
21 + const active = Object.entries({ year: params.year, domain: params.domain, topic: params.topic, author: params.author, keyword: params.keyword, method: params.method, dataset: params.dataset, pub: params.pubStatus, series: params.series, lang: params.lang }).filter(([, v]) => v && v.length).length + (params.pdf ? 1 : 0) + (params.q ? 1 : 0);
22 + const name = (x: { name_en: string; name_fr: string }) => (locale === "fr" ? x.name_fr : x.name_en);
23 + const Group = ({ title, k, opts }: { title: string; k: string; opts: { v: string; label: string; n: number }[] }) => opts.length ? (
24 + <details open={!!(sp[k] && String(sp[k]).length)} className="border-b border-line py-2">
25 + <summary className="flex justify-between items-center text-sm font-medium py-1">{title}<span className="text-fg-3 text-xs">{opts.length}</span></summary>
26 + <ul className="mt-1 max-h-56 overflow-auto pr-1 space-y-0.5 text-sm">{opts.map((o) => <li key={o.v}><label className="flex items-center gap-2 py-0.5 cursor-pointer"><input type="checkbox" name={k} value={o.v} defaultChecked={has(k === "pub" ? "pubStatus" : (k as keyof ListParams), o.v)} className="accent-[var(--navy)]" /><span className="flex-1 truncate text-fg-2">{o.label}</span><span className="text-xs text-fg-3 font-mono">{o.n}</span></label></li>)}</ul>
27 + </details>) : null;
28 + return (
29 + <div className="container py-10">
30 + <header className="mb-6"><h1 className="text-3xl sm:text-4xl">{d.papers.title}</h1><p className="meta mt-2 max-w-3xl">{d.papers.lead}</p></header>
31 + <div className="grid gap-8 lg:grid-cols-[17rem_1fr]">
32 + <aside className="lg:sticky lg:top-20 self-start">
33 + <form method="get" action={href(locale, "/papers")} className="card p-4">
34 + <input type="hidden" name="view" value={view} /><input type="hidden" name="sort" value={params.sort} />
35 + <label htmlFor="q" className="label">{d.nav.search}</label>
36 + <input id="q" name="q" defaultValue={params.q ?? ""} className="input mt-1 mb-3" placeholder={d.search.placeholder} />
37 + <p className="label mb-1">{d.papers.filters}{active ? ` (${active})` : ""}</p>
38 + <Group title={d.papers.field} k="domain" opts={f.domains.map((x) => ({ v: x.slug, label: name(x), n: x.n }))} />
39 + <Group title={d.papers.topic} k="topic" opts={f.topics.map((x) => ({ v: x.slug, label: name(x), n: x.n }))} />
40 + <Group title={d.papers.year} k="year" opts={f.years.map((x) => ({ v: String(x.year), label: String(x.year), n: x.n }))} />
41 + <Group title={d.papers.author} k="author" opts={f.authors.map((x) => ({ v: x.slug, label: x.name, n: x.n }))} />
42 + <Group title={d.papers.method} k="method" opts={f.methods.map((x) => ({ v: x.slug, label: name(x), n: x.n }))} />
43 + <Group title={d.papers.dataset} k="dataset" opts={f.datasets.map((x) => ({ v: x.slug, label: name(x), n: x.n }))} />
44 + <Group title={d.papers.keyword} k="keyword" opts={f.keywords.map((x) => ({ v: x.keyword, label: x.keyword, n: x.n }))} />
45 + <Group title={d.papers.series} k="series" opts={f.series.map((x) => ({ v: x.slug, label: name(x), n: x.n }))} />
46 + <Group title={d.papers.pubStatus} k="pub" opts={(["working_paper", "under_review", "forthcoming", "published"] as const).map((v) => ({ v, label: d.misc.pubStatus[v], n: 0 })).filter(() => true)} />
47 + <label className="flex items-center gap-2 text-sm py-2"><input type="checkbox" name="pdf" value="1" defaultChecked={params.pdf} className="accent-[var(--navy)]" />{d.papers.onlyPdf}</label>
48 + <div className="flex gap-2 mt-2"><button type="submit" className="btn btn-primary btn-sm flex-1">{d.search.go}</button><Link href={href(locale, "/papers")} className="btn btn-sm no-underline">{d.papers.clear}</Link></div>
49 + </form>
50 + </aside>
51 + <section>
52 + <div className="flex flex-wrap items-center gap-3 mb-4">
53 + <p className="text-sm text-fg-2"><strong className="text-fg">{res.total}</strong> {res.total === 1 ? d.papers.result : d.papers.results}</p>
54 + <form method="get" action={href(locale, "/papers")} className="ml-auto flex items-center gap-2 text-sm">
55 + {[...qs.entries()].filter(([k]) => k !== "sort" && k !== "view").map(([k, v], i) => <input key={i} type="hidden" name={k} value={v} />)}
56 + <input type="hidden" name="view" value={view} />
57 + <label htmlFor="sort" className="text-fg-3">{d.papers.sort}</label>
58 + <select id="sort" name="sort" defaultValue={params.sort} className="input !w-auto !py-1.5 !text-sm">
59 + {(["recent", "revised", "number", "title", "year", "views", "downloads"] as const).map((s) => <option key={s} value={s}>{d.papers[`sort${s[0].toUpperCase()}${s.slice(1)}` as keyof typeof d.papers] as string}</option>)}
60 + </select>
61 + <button type="submit" className="btn btn-sm">OK</button>
62 + </form>
63 + <Suspense><DisplayToggle locale={locale} current={view} /></Suspense>
64 + </div>
65 + {res.items.length === 0 ? <p className="card p-8 text-center text-fg-2">{d.papers.noResults}</p>
66 + : view === "cards" ? <div className="grid gap-4 md:grid-cols-2">{res.items.map((p) => <PaperCard key={p.id} p={p} locale={locale} />)}</div>
67 + : view === "table" ? <PaperTable items={res.items} locale={locale} />
68 + : <div className="card px-5">{res.items.map((p) => <PaperRow key={p.id} p={p} locale={locale} compact={view === "compact"} />)}</div>}
69 + <Pagination locale={locale} page={res.page} total={res.total} perPage={res.perPage} base={href(locale, "/papers")} params={qs} />
70 + {(params.sort === "views" || params.sort === "downloads" || view === "table") && <p className="text-xs text-fg-3 mt-4">{d.papers.popularityNote}</p>}
71 + </section>
72 + </div>
73 + </div>
74 + );
75 +}
added src/app/(public)/[locale]/research-map/page.tsx +12 −0
@@ -0,0 +1,12 @@
1 +import type { Metadata } from "next";
2 +import { getLocale } from "@/lib/i18n/server";
3 +import { t } from "@/lib/i18n";
4 +import { graph } from "@/lib/papers/queries";
5 +import { Constellation, type GEdge, type GNode } from "@/components/constellation";
6 +export const revalidate = 300;
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.map.title, description: d.map.lead, alternates: { canonical: "/research-map" } }; }
8 +export default async function MapPage() {
9 + const locale = await getLocale(); const d = t(locale); const g = await graph();
10 + return (<div className="container py-10"><h1 className="text-3xl sm:text-4xl">{d.map.title}</h1><p className="meta mt-2 max-w-3xl">{d.map.lead}</p><div className="mt-6"><Constellation locale={locale} nodes={g.nodes as unknown as GNode[]} edges={g.edges as unknown as GEdge[]} height={640} /></div>
11 + <p className="text-xs text-fg-3 mt-3">{d.home.constellationLead}</p></div>);
12 +}
added src/app/(public)/[locale]/search/page.tsx +21 −0
@@ -0,0 +1,21 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { href, t } from "@/lib/i18n";
5 +import { logSearch, search, type SearchMode } from "@/lib/papers/search";
6 +export const dynamic = "force-dynamic";
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.search.title, description: d.search.lead, alternates: { canonical: "/search" } }; }
8 +export default async function SearchPage({ searchParams }: { searchParams: Promise<{ q?: string; mode?: string }> }) {
9 + const sp = await searchParams; const locale = await getLocale(); const d = t(locale);
10 + const q = (sp.q ?? "").trim(); const mode = (["hybrid", "lexical", "semantic"].includes(sp.mode ?? "") ? sp.mode : "hybrid") as SearchMode;
11 + const res = q ? await search(q, mode, 25) : null; if (res) await logSearch(q, res.mode, res.hits.length, locale, res.tookMs);
12 + return (<div className="container py-10 max-w-4xl"><h1 className="text-3xl sm:text-4xl">{d.search.title}</h1><p className="meta mt-2">{d.search.lead}</p>
13 + <form method="get" action={href(locale, "/search")} role="search" className="mt-6 card p-4 grid gap-3 sm:grid-cols-[1fr_auto_auto]">
14 + <label htmlFor="q" className="sr-only">{d.nav.search}</label><input id="q" name="q" defaultValue={q} className="input" placeholder={d.home.searchPlaceholder} autoFocus />
15 + <select name="mode" defaultValue={mode} aria-label={d.search.mode} className="input !w-auto"><option value="hybrid">{d.search.hybrid}</option><option value="lexical">{d.search.lexical}</option><option value="semantic">{d.search.semantic}</option></select>
16 + <button type="submit" className="btn btn-primary">{d.search.go}</button></form>
17 + {res && (<section className="mt-6"><p className="meta mb-3">{res.hits.length} {res.hits.length === 1 ? d.papers.result : d.papers.results} · {d.search.took} {res.tookMs} ms · {res.semanticUsed ? d.search.semantic : d.search.lexical}{res.mode === "hybrid" && res.semanticUsed ? ` + ${d.search.lexical}` : ""}</p>
18 + {res.hits.length === 0 && <p className="card p-8 text-center text-fg-2">{d.search.none}</p>}
19 + <ol className="space-y-3">{res.hits.map((h) => <li key={h.slug} className="card p-4"><p className="text-xs flex flex-wrap gap-2 items-center"><span className="font-mono text-navy">{h.identifier}</span><span className="text-fg-3">{h.year ?? ""}</span>{h.signals.map((s) => <span key={s} className="chip">{d.search.fieldHit} {s}</span>)}</p><h2 className="text-lg mt-1"><Link href={href(locale, `/papers/${h.slug}`)} className="text-fg no-underline hover:underline">{h.title}</Link></h2><p className="meta">{h.authors.join(", ")}</p>{h.snippets.map((s, i) => <p key={i} className="text-sm text-fg-2 mt-2 leading-relaxed"><span dangerouslySetInnerHTML={{ __html: s.text.replace(/<(?!\/?mark>)[^>]*>/g, "") }} />{s.page ? <span className="text-fg-3"> — p. {s.page}</span> : null}</p>)}</li>)}</ol></section>)}
20 + </div>);
21 +}
added src/app/(public)/[locale]/series/page.tsx +16 −0
@@ -0,0 +1,16 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { href, t } from "@/lib/i18n";
5 +import { seriesWithCounts } from "@/lib/papers/queries";
6 +export const revalidate = 300;
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.series.title, description: d.series.lead, alternates: { canonical: "/series" } }; }
8 +export default async function SeriesPage() {
9 + const locale = await getLocale(); const d = t(locale); const rows = await seriesWithCounts();
10 + return (
11 + <div className="container py-10">
12 + <h1 className="text-3xl sm:text-4xl">{d.series.title}</h1><p className="meta mt-2 max-w-3xl">{d.series.lead}</p>
13 + <ul className="grid gap-4 md:grid-cols-2 mt-8">{rows.map((s) => <li key={s.slug} className="card p-5"><p className="kicker">{s.code}</p><h2 className="text-xl mt-1"><Link href={href(locale, `/papers?series=${s.code}`)} className="no-underline hover:underline text-fg">{locale === "fr" ? s.name_fr : s.name_en}</Link></h2><p className="text-sm text-fg-2 mt-2 leading-relaxed">{locale === "fr" ? s.description_fr : s.description_en}</p><p className="meta mt-3">{s.n} {d.topics.papers}{s.min_no ? ` · ${s.code}${String(s.min_no).padStart(2, "0")} – ${s.code}${String(s.max_no).padStart(2, "0")}` : ""}</p></li>)}</ul>
14 + </div>
15 + );
16 +}
added src/app/(public)/[locale]/timeline/page.tsx +18 −0
@@ -0,0 +1,18 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { fmtDate, href, t } from "@/lib/i18n";
5 +import { timeline } from "@/lib/papers/queries";
6 +export const revalidate = 300;
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.timeline.title, description: d.timeline.lead, alternates: { canonical: "/timeline" } }; }
8 +export default async function TimelinePage() {
9 + const locale = await getLocale(); const d = t(locale); const rows = await timeline();
10 + const byMonth = new Map<string, typeof rows>(); for (const r of rows) { const k = (r.vdate ?? r.released ?? "unknown").slice(0, 7); (byMonth.get(k) ?? byMonth.set(k, []).get(k)!).push(r); }
11 + const months = [...byMonth.keys()].sort().reverse(); const max = Math.max(1, ...months.map((m) => byMonth.get(m)!.length));
12 + return (<div className="container py-10"><h1 className="text-3xl sm:text-4xl">{d.timeline.title}</h1><p className="meta mt-2">{d.timeline.lead}</p>
13 + <svg viewBox={`0 0 ${Math.max(400, months.length * 64)} 160`} role="img" aria-label={d.timeline.title} className="w-full max-w-4xl mt-8 card p-2 bg-bg-2">
14 + {[...months].reverse().map((m, i) => { const n = byMonth.get(m)!.length; const h = (n / max) * 100; const rel = byMonth.get(m)!.filter((r) => r.version_no === 1).length; return <g key={m} transform={`translate(${i * 64 + 12},0)`}><rect x={8} y={120 - h} width={40} height={h} fill="var(--navy)" opacity={0.85} /><rect x={8} y={120 - (rel / max) * 100} width={40} height={(rel / max) * 100} fill="var(--burgundy)" opacity={0.6} /><text x={28} y={135} textAnchor="middle" fontSize={10} fill="var(--fg-3)" fontFamily="var(--font-mono)">{m}</text><text x={28} y={112 - h} textAnchor="middle" fontSize={11} fill="var(--fg)">{n}</text></g>; })}
15 + </svg>
16 + <p className="meta mt-2"><span className="inline-block w-3 h-3 bg-navy align-middle mr-1" /> {d.timeline.releases} + {d.timeline.revisions} · <span className="inline-block w-3 h-3 bg-burgundy align-middle mr-1" /> {d.timeline.releases}</p>
17 + <ol className="mt-10 relative border-l border-line-2 pl-6 space-y-8">{months.map((m) => <li key={m}><h2 className="text-xl -ml-6 pl-6 relative before:absolute before:left-[-5px] before:top-2.5 before:w-2.5 before:h-2.5 before:rounded-full before:bg-navy">{fmtDate(locale, m + "-01", "month")}</h2><ul className="mt-3 space-y-2">{byMonth.get(m)!.map((r) => <li key={r.slug + r.version_no} className="flex gap-3 text-sm"><span className="font-mono text-navy w-20 shrink-0">{r.identifier}</span><span className="chip shrink-0">{r.version_no === 1 ? d.home.newRelease : d.home.revision} · {r.label}</span><Link href={href(locale, `/papers/${r.slug}`)} className="text-fg no-underline hover:underline">{r.title}</Link></li>)}</ul></li>)}</ol></div>);
18 +}
added src/app/(public)/[locale]/topics/[slug]/page.tsx +21 −0
@@ -0,0 +1,21 @@
1 +import type { Metadata } from "next";
2 +import { notFound } from "next/navigation";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { t } from "@/lib/i18n";
5 +import { getTopic, listPapers } from "@/lib/papers/queries";
6 +import { PaperRow } from "@/components/paper-items";
7 +export const revalidate = 300;
8 +export async function generateMetadata({ params }: { params: Promise<{ slug: string }> }): Promise<Metadata> { const { slug } = await params; const x = await getTopic(slug); const locale = await getLocale(); return x ? { title: locale === "fr" ? x.name_fr : x.name_en, description: (locale === "fr" ? x.description_fr : x.description_en) ?? undefined, alternates: { canonical: `/topics/${slug}` } } : {}; }
9 +export default async function TopicPage({ params }: { params: Promise<{ slug: string }> }) {
10 + const { slug } = await params; const locale = await getLocale(); const d = t(locale); const x = await getTopic(slug); if (!x) notFound();
11 + const res = await listPapers({ topic: [slug], sort: "recent", perPage: 100 });
12 + return (
13 + <div className="container py-10">
14 + <p className="kicker">{x.kind === "domain" ? d.paper.domain : d.papers.topic}</p>
15 + <h1 className="text-3xl sm:text-4xl mt-2">{locale === "fr" ? x.name_fr : x.name_en}</h1>
16 + {(locale === "fr" ? x.description_fr : x.description_en) && <p className="meta mt-2 max-w-3xl">{locale === "fr" ? x.description_fr : x.description_en}</p>}
17 + <p className="meta mt-3">{res.total} {res.total === 1 ? d.topics.paper : d.topics.papers}</p>
18 + <div className="card px-5 mt-6">{res.items.map((p) => <PaperRow key={p.id} p={p} locale={locale} />)}</div>
19 + </div>
20 + );
21 +}
added src/app/(public)/[locale]/topics/page.tsx +20 −0
@@ -0,0 +1,20 @@
1 +import type { Metadata } from "next";
2 +import Link from "next/link";
3 +import { getLocale } from "@/lib/i18n/server";
4 +import { href, t } from "@/lib/i18n";
5 +import { topicsWithCounts } from "@/lib/papers/queries";
6 +export const revalidate = 300;
7 +export async function generateMetadata(): Promise<Metadata> { const d = t(await getLocale()); return { title: d.topics.title, description: d.topics.lead, alternates: { canonical: "/topics" } }; }
8 +export default async function TopicsPage() {
9 + const locale = await getLocale(); const d = t(locale); const rows = await topicsWithCounts();
10 + const domains = rows.filter((r) => r.kind === "domain"), topics = rows.filter((r) => r.kind === "topic" && r.n > 0);
11 + return (
12 + <div className="container py-10">
13 + <h1 className="text-3xl sm:text-4xl">{d.topics.title}</h1><p className="meta mt-2 max-w-3xl">{d.topics.lead}</p>
14 + <h2 className="text-2xl mt-10 mb-4">{d.topics.domains}</h2>
15 + <ul className="grid gap-4 sm:grid-cols-2 lg:grid-cols-3">{domains.map((x) => <li key={x.slug}><Link href={href(locale, `/topics/${x.slug}`)} className="card p-5 block h-full no-underline hover:shadow-[var(--shadow-lift)] transition-shadow"><span className="flex items-center gap-2"><span className="w-3 h-3 rounded-full" style={{ background: x.accent ?? "var(--navy)" }} /><span className="text-lg text-fg serif">{locale === "fr" ? x.name_fr : x.name_en}</span></span><span className="block text-sm text-fg-2 mt-2 leading-relaxed">{locale === "fr" ? x.description_fr : x.description_en}</span><span className="block meta mt-3">{x.n} {x.n === 1 ? d.topics.paper : d.topics.papers}</span></Link></li>)}</ul>
16 + <h2 className="text-2xl mt-12 mb-4">{d.topics.all}</h2>
17 + <ul className="flex flex-wrap gap-2">{topics.map((x) => <li key={x.slug}><Link href={href(locale, `/topics/${x.slug}`)} className="chip no-underline !text-sm !py-1.5 !px-3">{locale === "fr" ? x.name_fr : x.name_en} <span className="font-mono text-fg-3">{x.n}</span></Link></li>)}</ul>
18 + </div>
19 + );
20 +}
added src/app/admin/account/page.tsx +20 −0
@@ -0,0 +1,20 @@
1 +import { generateSecret, generateURI } from "otplib";
2 +import QRCode from "qrcode";
3 +import { sql } from "@/db";
4 +import { requireUserPage } from "@/lib/auth/guard";
5 +import { csrfToken } from "@/lib/auth/session";
6 +import { accountAction } from "@/lib/admin/actions";
7 +import { Csrf, Field, Notice, PageTitle } from "@/components/admin-ui";
8 +export default async function Account({ searchParams }: { searchParams: Promise<Record<string, string | undefined>> }) {
9 + const sp = await searchParams; const u = await requireUserPage(); const csrf = csrfToken(u);
10 + const sessions = await sql`SELECT id, ip, user_agent, created_at, last_seen_at FROM sessions WHERE user_id = ${u.id} AND revoked_at IS NULL AND expires_at > now() ORDER BY last_seen_at DESC`;
11 + const secret = u.totpEnabled ? null : generateSecret(); const qr = secret ? await QRCode.toDataURL(generateURI({ issuer: "wp.spboucher.ai", label: u.email, secret })) : null;
12 + return (<div><PageTitle title="Account" /><Notice sp={sp} />{sp.must_change && <p className="card p-3 mb-4 text-warn border-warn">You must change the initial password before using the administration.</p>}
13 + <div className="grid lg:grid-cols-2 gap-6">
14 + <form action={accountAction} className="card p-4 space-y-3"><h2 className="text-lg">Change password</h2><Csrf token={csrf} /><input type="hidden" name="op" value="password" /><Field label="Current password" name="current" type="password" required /><Field label="New password (≥ 14 chars, upper, lower, digit)" name="new" type="password" required /><Field label="Confirm" name="confirm" type="password" required /><button className="btn btn-primary">Update password</button></form>
15 + <section className="card p-4 space-y-3"><h2 className="text-lg">Two-factor authentication (TOTP)</h2>
16 + {u.totpEnabled ? <form action={accountAction} className="space-y-3"><p className="text-ok text-sm">Enabled.</p><Csrf token={csrf} /><input type="hidden" name="op" value="totp_disable" /><Field label="Current password to disable" name="current" type="password" required /><button className="btn btn-sm text-bad">Disable</button></form>
17 + : <form action={accountAction} className="space-y-3"><p className="meta">Scan with an authenticator app, then enter the 6-digit code.</p>{qr && <img src={qr} alt="TOTP QR code" width={180} height={180} className="border border-line" />}<p className="font-mono text-xs break-all">{secret}</p><Csrf token={csrf} /><input type="hidden" name="op" value="totp_enable" /><input type="hidden" name="secret" value={secret ?? ""} /><Field label="Code" name="code" required /><button className="btn btn-primary btn-sm">Enable</button></form>}</section>
18 + <section className="card p-4 lg:col-span-2"><h2 className="text-lg mb-2">Active sessions</h2><table className="data"><thead><tr><th>IP</th><th>Client</th><th>Created</th><th>Last seen</th></tr></thead><tbody>{sessions.map((s) => <tr key={s.id}><td>{s.ip}{s.id === u.sessionId ? " (this)" : ""}</td><td className="text-xs text-fg-2">{s.user_agent}</td><td className="text-xs">{new Date(s.created_at).toLocaleString("en-CA")}</td><td className="text-xs">{new Date(s.last_seen_at).toLocaleString("en-CA")}</td></tr>)}</tbody></table><form action={accountAction} className="mt-3"><Csrf token={csrf} /><input type="hidden" name="op" value="revoke_others" /><button className="btn btn-sm">Sign out other sessions</button></form></section>
19 + </div></div>);
20 +}
added src/app/admin/analytics/page.tsx +17 −0
@@ -0,0 +1,17 @@
1 +import { sql } from "@/db";
2 +import { PageTitle } from "@/components/admin-ui";
3 +export default async function Analytics() {
4 + const [top, searches, referrers, byDay] = await Promise.all([
5 + sql`SELECT p.identifier, p.title, p.view_count::int AS views, p.download_count::int AS downloads, (SELECT count(*)::int FROM events e WHERE e.paper_id = p.id AND e.kind = 'view' AND NOT e.is_bot AND e.day > current_date - 7) AS views_7d FROM papers p WHERE p.status = 'published' ORDER BY p.view_count DESC, p.download_count DESC LIMIT 25`,
6 + sql`SELECT query, mode, count(*)::int AS n, avg(results)::int AS avg_results, avg(took_ms)::int AS ms FROM search_log WHERE at > now() - interval '90 days' GROUP BY query, mode ORDER BY n DESC LIMIT 40`,
7 + sql`SELECT referrer_host, count(*)::int AS n FROM events WHERE NOT is_bot AND referrer_host IS NOT NULL AND day > current_date - 30 GROUP BY referrer_host ORDER BY n DESC LIMIT 20`,
8 + sql`SELECT day::text, count(*) FILTER (WHERE kind = 'view')::int AS views, count(*) FILTER (WHERE kind = 'download')::int AS downloads, count(*) FILTER (WHERE is_bot)::int AS bots FROM events WHERE day > current_date - 30 GROUP BY day ORDER BY day DESC`,
9 + ]);
10 + return (<div><PageTitle title="Analytics" /><p className="meta mb-4">Views = distinct daily visitors per paper page (sha256 of IP + user agent + daily salt, not reversible; bots excluded). Downloads = manuscript requests. Raw events are kept 400 days; daily aggregates are kept indefinitely. No cookies, no third-party analytics.</p>
11 + <div className="grid xl:grid-cols-2 gap-6">
12 + <section className="card p-4"><h2 className="text-lg mb-2">Most viewed papers (all time)</h2><table className="data"><thead><tr><th>ID</th><th>Title</th><th>Views</th><th>7 d</th><th>Downloads</th></tr></thead><tbody>{top.map((p) => <tr key={p.identifier}><td className="font-mono text-navy">{p.identifier}</td><td className="text-fg-2">{p.title}</td><td>{p.views}</td><td>{p.views_7d}</td><td>{p.downloads}</td></tr>)}</tbody></table></section>
13 + <section className="card p-4"><h2 className="text-lg mb-2">Daily events (raw, last 30 days)</h2><table className="data"><thead><tr><th>Day</th><th>Views</th><th>Downloads</th><th>Bot hits</th></tr></thead><tbody>{byDay.map((d) => <tr key={d.day}><td>{d.day}</td><td>{d.views}</td><td>{d.downloads}</td><td className="text-fg-3">{d.bots}</td></tr>)}</tbody></table></section>
14 + <section className="card p-4"><h2 className="text-lg mb-2">Searches (90 days)</h2><table className="data"><thead><tr><th>Query</th><th>Mode</th><th>Count</th><th>Avg results</th><th>ms</th></tr></thead><tbody>{searches.map((s, i) => <tr key={i}><td>{s.query}</td><td>{s.mode}</td><td>{s.n}</td><td>{s.avg_results}</td><td>{s.ms}</td></tr>)}</tbody></table></section>
15 + <section className="card p-4"><h2 className="text-lg mb-2">Referrers (30 days)</h2><table className="data"><thead><tr><th>Host</th><th>Visits</th></tr></thead><tbody>{referrers.map((r) => <tr key={r.referrer_host}><td>{r.referrer_host}</td><td>{r.n}</td></tr>)}</tbody></table></section>
16 + </div></div>);
17 +}
added src/app/admin/audit/page.tsx +9 −0
@@ -0,0 +1,9 @@
1 +import { sql } from "@/db";
2 +import { PageTitle } from "@/components/admin-ui";
3 +export default async function Audit({ searchParams }: { searchParams: Promise<Record<string, string | undefined>> }) {
4 + const sp = await searchParams; const page = Math.max(1, Number(sp.page ?? 1) || 1); const q = sp.q || null;
5 + const rows = await sql`SELECT * FROM audit_log WHERE (${q}::text IS NULL OR action ILIKE '%' || ${q ?? ""} || '%' OR actor_label ILIKE '%' || ${q ?? ""} || '%' OR entity_id ILIKE '%' || ${q ?? ""} || '%') ORDER BY at DESC LIMIT 100 OFFSET ${(page - 1) * 100}`;
6 + return (<div><PageTitle title="Audit log" /><form className="flex gap-2 mb-4"><input name="q" defaultValue={sp.q ?? ""} placeholder="filter action / actor / entity" className="input !w-72" /><button className="btn btn-sm">Filter</button></form>
7 + <div className="card overflow-x-auto"><table className="data"><thead><tr><th>When</th><th>Actor</th><th>IP</th><th>Action</th><th>Entity</th><th>Before</th><th>After</th></tr></thead><tbody>{rows.map((a) => <tr key={a.id}><td className="whitespace-nowrap text-xs">{new Date(a.at).toLocaleString("en-CA")}</td><td className="text-xs">{a.actor_label}</td><td className="text-xs">{a.ip ?? ""}</td><td className="font-mono text-xs">{a.action}</td><td className="text-xs">{a.entity_type} {a.entity_id ?? ""}</td><td className="text-xs text-fg-3 max-w-xs truncate">{a.before ? JSON.stringify(a.before) : ""}</td><td className="text-xs text-fg-3 max-w-xs truncate">{a.after ? JSON.stringify(a.after) : ""}</td></tr>)}</tbody></table></div>
8 + <p className="mt-3 text-sm flex gap-3">{page > 1 && <a href={`?page=${page - 1}&q=${sp.q ?? ""}`} className="text-navy">← Newer</a>}{rows.length === 100 && <a href={`?page=${page + 1}&q=${sp.q ?? ""}`} className="text-navy">Older →</a>}</p></div>);
9 +}
added src/app/admin/ingest/[id]/page.tsx +39 −0
@@ -0,0 +1,39 @@
1 +import { notFound } from "next/navigation";
2 +import { sql } from "@/db";
3 +import { requireUserPage } from "@/lib/auth/guard";
4 +import { csrfToken } from "@/lib/auth/session";
5 +import { ingestAction } from "@/lib/admin/actions";
6 +import { Csrf, Field, Notice, PageTitle, Select } from "@/components/admin-ui";
7 +export default async function Candidate({ params, searchParams }: { params: Promise<{ id: string }>; searchParams: Promise<Record<string, string | undefined>> }) {
8 + const { id } = await params; const sp = await searchParams; const u = await requireUserPage(); const csrf = csrfToken(u);
9 + const [c] = await sql`SELECT c.*, f.pages, f.integrity_ok, f.pdf_meta, f.sha256 AS fsha FROM ingest_candidates c LEFT JOIN files f ON f.id = c.file_id WHERE c.id = ${id}`; if (!c) notFound();
10 + const pr = (c.proposal ?? {}) as Record<string, any>; const [authors, series, papers] = await Promise.all([sql`SELECT id, display_name FROM authors ORDER BY sort`, sql`SELECT code, name_en FROM series ORDER BY sort`, sql`SELECT id, identifier, title FROM papers ORDER BY number DESC`]);
11 + const guessAuthors = new Set((authors as unknown as { id: number; display_name: string }[]).filter((a) => (pr.authors ?? []).some((n: string) => n.toLowerCase().includes(a.display_name.split(" ").pop()!.toLowerCase()))).map((a) => a.id));
12 + const MONTH: Record<string, number> = { january: 1, february: 2, march: 3, april: 4, may: 5, june: 6, july: 7, august: 8, september: 9, october: 10, november: 11, december: 12, janvier: 1, février: 2, mars: 3, avril: 4, mai: 5, juin: 6, juillet: 7, août: 8, septembre: 9, octobre: 10, novembre: 11, décembre: 12 };
13 + const dm = pr.dateText ? /^(\S+)\s+(\d{4})$/.exec(pr.dateText) : null; const dateGuess = dm && MONTH[dm[1].toLowerCase()] ? `${dm[2]}-${String(MONTH[dm[1].toLowerCase()]).padStart(2, "0")}` : "";
14 + return (<div><PageTitle title={`Candidate · ${String(c.source_path).split("/").pop()}`} /><Notice sp={sp} />
15 + <div className="grid lg:grid-cols-[1fr_22rem] gap-6">
16 + <div className="space-y-6">
17 + <section className="card p-4 text-sm"><h2 className="text-lg mb-2">Extraction report <span className={`chip ${c.status === "ready" ? "chip-navy" : ""}`}>{c.status}</span></h2>
18 + <dl className="grid sm:grid-cols-2 gap-2"><div><dt className="label">SHA-256</dt><dd className="font-mono text-xs break-all">{c.sha256}</dd></div><div><dt className="label">Integrity</dt><dd>{c.integrity_ok === null ? "pending" : c.integrity_ok ? "OK" : <span className="text-bad">FAILED</span>} · {c.pages ?? "?"} pages</dd></div><div><dt className="label">PDF metadata</dt><dd className="text-fg-2">{pr.pdfMeta ? `${pr.pdfMeta.title ?? "—"} / ${pr.pdfMeta.author ?? "—"} / ${pr.pdfMeta.creationDate ?? "—"}` : "—"}</dd></div><div><dt className="label">Confidence</dt><dd className="font-mono text-xs">{pr.confidence ? Object.entries(pr.confidence).map(([k, v]) => `${k}:${Number(v).toFixed(1)}`).join(" ") : "—"}</dd></div></dl>
19 + {(c.flags as { code: string; message: string }[]).length > 0 && <ul className="mt-3 space-y-1">{(c.flags as { code: string; message: string }[]).map((f) => <li key={f.code} className="text-warn">⚠ {f.message}</li>)}</ul>}
20 + {(c.duplicates as { identifier: string; title: string; reasons: string[]; score: number; paperId: string }[]).length > 0 && <div className="mt-3"><p className="label">Possible duplicates</p><ul className="space-y-1">{(c.duplicates as { identifier: string; title: string; reasons: string[]; score: number }[]).map((d) => <li key={d.identifier}><span className="font-mono text-navy">{d.identifier}</span> {d.title} — {d.reasons.join(", ")} ({(d.score * 100).toFixed(0)}%)</li>)}</ul></div>}
21 + {pr.abstract && <details className="mt-3"><summary className="label">Proposed abstract</summary><p className="text-fg-2 mt-1 leading-relaxed">{pr.abstract}</p></details>}
22 + <p className="meta mt-3">The proposal below is a suggestion extracted from the title page. Verify every field against the manuscript. The release date must come from the document, never from the file timestamp. Author consent to publication is not inferred.</p>
23 + </section>
24 + {c.status === "ready" && c.integrity_ok && (<>
25 + <form action={ingestAction} className="card p-4 space-y-4"><h2 className="text-lg">Approve as a new working paper (draft)</h2><Csrf token={csrf} /><input type="hidden" name="candidate_id" value={c.id} /><input type="hidden" name="op" value="approve" />
26 + <div className="grid sm:grid-cols-4 gap-3"><Select label="Series" name="series_code" value={pr.wp?.seriesCode ?? "WP"} options={series.map((s) => ({ v: s.code, label: `${s.code} — ${s.name_en}` }))} /><Field label="Number" name="number" type="number" value={pr.wp?.number} required /><Field label="Version label" name="version_label" value={pr.version ? `v${pr.version}` : "v1"} /><Field label="Release date (YYYY-MM[-DD])" name="released_on" value={dateGuess} hint={pr.dateText ? `detected: ${pr.dateText}` : "not detected"} /></div>
27 + <Field label="Title" name="title" value={pr.title} required /><Field label="Abstract" name="abstract" value={pr.abstract} rows={6} /><div className="grid sm:grid-cols-3 gap-3"><Field label="Keywords (one per line)" name="keywords" value={(pr.keywords ?? []).join("\n")} rows={4} /><Field label="JEL" name="jel" value={(pr.jel ?? []).join(", ")} /><Field label="Year" name="year" type="number" value={dm ? Number(dm[2]) : ""} /></div>
28 + <div><p className="label mb-1">Authors (detected: {(pr.authors ?? []).join("; ") || "none"})</p><div className="grid sm:grid-cols-3 gap-1">{authors.map((a) => <label key={a.id} className="text-sm flex gap-2"><input type="checkbox" name="author_ids" value={a.id} defaultChecked={guessAuthors.has(a.id)} />{a.display_name}</label>)}</div><p className="text-xs text-fg-3 mt-1">Unknown authors must first be created in Taxonomy.</p></div>
29 + <button className="btn btn-primary">Create draft paper</button></form>
30 + <form action={ingestAction} className="card p-4 space-y-3"><h2 className="text-lg">Attach as a new version of an existing paper</h2><Csrf token={csrf} /><input type="hidden" name="candidate_id" value={c.id} /><input type="hidden" name="op" value="attach" />
31 + <Select label="Target paper" name="target_paper_id" value={(c.duplicates as { paperId?: string }[])[0]?.paperId ?? ""} options={papers.map((p) => ({ v: p.id, label: `${p.identifier} — ${p.title.slice(0, 80)}` }))} />
32 + <div className="grid sm:grid-cols-3 gap-3"><Field label="Version label" name="version_label" value={pr.version ? `v${pr.version}` : ""} /><Field label="Release date" name="released_on" value={dateGuess} /><Field label="Description" name="description" value="Revised manuscript" /></div>
33 + <label className="flex gap-2 text-sm"><input type="checkbox" name="make_current" defaultChecked /> Make it the current version (re-indexes full text)</label><button className="btn">Attach version</button></form>
34 + </>)}
35 + <div className="flex gap-2"><form action={ingestAction} className="flex gap-2"><Csrf token={csrf} /><input type="hidden" name="candidate_id" value={c.id} /><input type="hidden" name="op" value="reject" /><input name="note" placeholder="reason" className="input !w-56" /><button className="btn btn-sm text-bad">Reject</button></form><form action={ingestAction}><Csrf token={csrf} /><input type="hidden" name="candidate_id" value={c.id} /><input type="hidden" name="op" value="reprocess" /><button className="btn btn-sm">Re-run extraction</button></form></div>
36 + </div>
37 + <aside className="card p-3 self-start">{c.fsha ? <img src={`/api/admin/preview/${c.fsha}`} alt="First page preview" className="w-full border border-line" /> : null}<p className="meta mt-2 break-all">{c.source_path}</p></aside>
38 + </div></div>);
39 +}
added src/app/admin/ingest/page.tsx +15 −0
@@ -0,0 +1,15 @@
1 +import Link from "next/link";
2 +import { sql } from "@/db";
3 +import { requireUserPage } from "@/lib/auth/guard";
4 +import { csrfToken } from "@/lib/auth/session";
5 +import { ingestAction } from "@/lib/admin/actions";
6 +import { Csrf, Notice, PageTitle } from "@/components/admin-ui";
7 +import { STORAGE_DIR } from "@/lib/env";
8 +export default async function Ingest({ searchParams }: { searchParams: Promise<Record<string, string | undefined>> }) {
9 + const sp = await searchParams; const u = await requireUserPage();
10 + const rows = await sql`SELECT c.*, f.pages, f.integrity_ok FROM ingest_candidates c LEFT JOIN files f ON f.id = c.file_id ORDER BY (c.status IN ('ready','queued','processing')) DESC, c.created_at DESC LIMIT 200`;
11 + const [last] = await sql`SELECT value FROM settings WHERE key = 'ingest.last_scan'`; const [dirs] = await sql`SELECT value FROM settings WHERE key = 'ingest.extra_dirs'`;
12 + return (<div><PageTitle title="Manuscript ingestion"><form action={ingestAction}><Csrf token={csrfToken(u)} /><input type="hidden" name="op" value="scan" /><button className="btn btn-primary btn-sm">Scan directories now</button></form></PageTitle><Notice sp={sp} />
13 + <p className="meta mb-4">Scanned directories: <code>{STORAGE_DIR}/inbox</code>{((dirs?.value as string[]) ?? []).map((d) => <span key={d}>, <code>{d}</code></span>)} (configure in Settings). Last scan: {last ? `${last.value.at} — ${last.value.found} PDF found, ${last.value.queued} new` : "never"}. Every candidate is checksummed, integrity-checked, text-extracted and checked for duplicates; metadata is a proposal that you must verify before approval. Nothing is published automatically.</p>
14 + <div className="card overflow-x-auto"><table className="data"><thead><tr><th>File</th><th>Status</th><th>Proposed title</th><th>WP no.</th><th>Flags</th><th>Duplicates</th><th>Created</th></tr></thead><tbody>{rows.map((c) => { const pr = c.proposal as Record<string, any>; return <tr key={c.id}><td className="text-xs"><Link href={`/admin/ingest/${c.id}`} className="text-navy no-underline hover:underline">{String(c.source_path).split("/").pop()}</Link><br /><span className="text-fg-3">{c.pages ?? "?"} p.</span></td><td><span className={`chip ${c.status === "ready" ? "chip-navy" : c.status === "rejected" ? "chip-burgundy" : ""}`}>{c.status}</span></td><td className="text-fg-2">{pr?.title ?? "—"}</td><td className="font-mono">{pr?.wp ? `${pr.wp.seriesCode}${pr.wp.number}` : "—"}</td><td className="text-xs">{(c.flags as { code: string }[]).map((f) => <span key={f.code} className="chip mr-1">{f.code}</span>)}</td><td className="text-xs">{(c.duplicates as { identifier: string; score: number }[]).slice(0, 2).map((d) => <span key={d.identifier} className="chip chip-burgundy mr-1">{d.identifier} {(d.score * 100).toFixed(0)}%</span>)}</td><td className="text-xs whitespace-nowrap">{String(c.created_at).slice(0, 16)}</td></tr>; })}</tbody></table>{rows.length === 0 && <p className="p-6 text-center meta">Queue is empty. Drop PDFs in the inbox directory and scan.</p>}</div></div>);
15 +}
added src/app/admin/layout.tsx +22 −0
@@ -0,0 +1,22 @@
1 +import Link from "next/link";
2 +import { requireUserPage } from "@/lib/auth/guard";
3 +import { csrfToken, maybeRotate } from "@/lib/auth/session";
4 +import { accountAction } from "@/lib/admin/actions";
5 +export const dynamic = "force-dynamic";
6 +const NAV: [string, string][] = [["/admin", "Dashboard"], ["/admin/papers", "Papers"], ["/admin/ingest", "Ingestion"], ["/admin/taxonomy", "Taxonomy"], ["/admin/analytics", "Analytics"], ["/admin/audit", "Audit log"], ["/admin/system", "System"], ["/admin/settings", "Settings"], ["/admin/account", "Account"]];
7 +export default async function AdminLayout({ children }: { children: React.ReactNode }) {
8 + const u = await requireUserPage(); await maybeRotate(u);
9 + return (
10 + <div className="min-h-dvh grid lg:grid-cols-[14rem_1fr]">
11 + <aside className="border-b lg:border-b-0 lg:border-r border-line bg-bg-3/50 p-4 lg:sticky lg:top-0 lg:h-dvh overflow-auto">
12 + <Link href="/" className="serif text-lg text-fg no-underline">Research Working Papers</Link><p className="text-xs text-fg-3 mb-4">Administration · {u.email}</p>
13 + <nav className="flex lg:flex-col flex-wrap gap-1 text-sm">{NAV.map(([h, l]) => <Link key={h} href={h} className="px-2.5 py-1.5 rounded-md text-fg-2 no-underline hover:bg-bg-2 hover:text-fg">{l}</Link>)}
14 + <form action={accountAction} className="mt-2"><input type="hidden" name="_csrf" value={csrfToken(u)} /><input type="hidden" name="op" value="logout" /><button className="px-2.5 py-1.5 rounded-md text-fg-3 hover:bg-bg-2 hover:text-fg w-full text-left">Sign out</button></form>
15 + </nav>
16 + {u.mustChangePassword && <p className="mt-4 text-xs text-bad">Change your initial password in Account.</p>}
17 + {!u.totpEnabled && <p className="mt-2 text-xs text-warn">Two-factor authentication is off. Enable it in Account.</p>}
18 + </aside>
19 + <main className="p-4 sm:p-8 min-w-0">{children}</main>
20 + </div>
21 + );
22 +}
added src/app/admin/page.tsx +17 −0
@@ -0,0 +1,17 @@
1 +import Link from "next/link";
2 +import { dashboard } from "@/lib/admin/queries";
3 +import { PageTitle } from "@/components/admin-ui";
4 +export default async function Dashboard() {
5 + const d = await dashboard(); const q = Object.fromEntries(d.queue.map((x) => [x.status, x.n])); const j = Object.fromEntries(d.jobs.map((x) => [x.status, x.n]));
6 + const tile = (l: string, v: string | number, href?: string) => <div className="card px-4 py-3"><p className="label">{l}</p><p className="serif text-2xl mt-1">{href ? <Link href={href} className="no-underline text-fg hover:underline">{v}</Link> : v}</p></div>;
7 + return (<div><PageTitle title="Dashboard"><Link href="/admin/papers/new" className="btn btn-primary btn-sm no-underline">New paper</Link><Link href="/admin/ingest" className="btn btn-sm no-underline">Ingestion queue</Link></PageTitle>
8 + <div className="grid grid-cols-2 md:grid-cols-4 gap-3">{tile("Published", d.counts.published, "/admin/papers?status=published")}{tile("Drafts", d.counts.draft, "/admin/papers?status=draft")}{tile("Reserved", d.counts.reserved, "/admin/papers?status=reserved")}{tile("Withdrawn", d.counts.withdrawn, "/admin/papers?status=withdrawn")}
9 + {tile("Ingest: ready for review", q.ready ?? 0, "/admin/ingest")}{tile("Ingest: processing", (q.queued ?? 0) + (q.processing ?? 0))}{tile("Jobs failed (7 d)", j.failed ?? 0, "/admin/system")}{tile("Jobs queued", (j.queued ?? 0) + (j.running ?? 0), "/admin/system")}
10 + {tile("Embeddings", `${d.ai.embedded}/${d.ai.total} papers · ${d.ai.chunks_embedded}/${d.ai.chunks} chunks`)}{tile("Last backup", d.backup ? `${d.backup.ok ? "OK" : "FAILED"} · ${new Date(d.backup.created_at).toLocaleString("en-CA")}${d.backup.remote_ok === false ? " · remote failed" : ""}` : "none", "/admin/system")}</div>
11 + <div className="grid lg:grid-cols-2 gap-6 mt-8">
12 + <section className="card p-4"><h2 className="text-lg mb-2">Views and downloads, last 30 days</h2><svg viewBox="0 0 600 140" className="w-full"><g>{d.views.map((v, i) => { const max = Math.max(1, ...d.views.map((x) => x.views + x.downloads)); const w = 600 / Math.max(1, d.views.length); return <g key={v.day} transform={`translate(${i * w},0)`}><rect x={1} y={120 - (v.views / max) * 110} width={w - 2} height={(v.views / max) * 110} fill="var(--navy)" /><rect x={1} y={120 - ((v.views + v.downloads) / max) * 110} width={w - 2} height={(v.downloads / max) * 110} fill="var(--burgundy)" opacity={0.7} /><title>{v.day}: {v.views} views, {v.downloads} downloads</title></g>; })}</g></svg><p className="meta">{d.views.length ? `${d.views.reduce((a, x) => a + x.views, 0)} views · ${d.views.reduce((a, x) => a + x.downloads, 0)} downloads` : "No aggregated data yet (daily job runs after midnight)."}</p></section>
13 + <section className="card p-4"><h2 className="text-lg mb-2">Top searches, 30 days</h2>{d.searches.length ? <table className="data"><thead><tr><th>Query</th><th>Count</th><th>Avg results</th></tr></thead><tbody>{d.searches.map((s) => <tr key={s.query}><td>{s.query}</td><td>{s.n}</td><td>{s.avg_results}</td></tr>)}</tbody></table> : <p className="meta">No searches yet.</p>}</section>
14 + </div>
15 + <section className="card p-4 mt-6"><h2 className="text-lg mb-2">Recent audit entries</h2><table className="data"><thead><tr><th>When</th><th>Actor</th><th>Action</th><th>Entity</th></tr></thead><tbody>{d.audit.map((a) => <tr key={a.id}><td className="whitespace-nowrap">{new Date(a.at).toLocaleString("en-CA")}</td><td>{a.actor_label}</td><td className="font-mono text-xs">{a.action}</td><td className="text-fg-2">{a.entity_type} {a.entity_id ?? ""}</td></tr>)}</tbody></table><p className="mt-2 text-sm"><Link href="/admin/audit" className="text-navy">Full audit log →</Link></p></section>
16 + </div>);
17 +}
added src/app/admin/papers/[id]/page.tsx +36 −0
@@ -0,0 +1,36 @@
1 +import Link from "next/link";
2 +import { notFound } from "next/navigation";
3 +import { adminPaper, taxonomy } from "@/lib/admin/queries";
4 +import { requireUserPage } from "@/lib/auth/guard";
5 +import { csrfToken } from "@/lib/auth/session";
6 +import { deletePaperAction, redirectAction, relationAction, setStatusAction, updatePaperAction, versionAction } from "@/lib/admin/actions";
7 +import { Csrf, Field, Notice, PageTitle } from "@/components/admin-ui";
8 +import { PaperForm } from "../paper-form";
9 +import { UploadForm } from "./upload-form";
10 +export default async function EditPaper({ params, searchParams }: { params: Promise<{ id: string }>; searchParams: Promise<Record<string, string | undefined>> }) {
11 + const { id } = await params; const sp = await searchParams; const u = await requireUserPage(); const [p, tax] = await Promise.all([adminPaper(id), taxonomy()]); if (!p) notFound(); const csrf = csrfToken(u);
12 + return (<div>
13 + <PageTitle title={`${p.identifier} · ${p.status}`}>{p.status === "published" && <Link href={`/papers/${p.slug}`} className="btn btn-sm no-underline" target="_blank">View public page ↗</Link>}</PageTitle><Notice sp={sp} />
14 + <section className="card p-4 mb-6 flex flex-wrap gap-2 items-center">
15 + <span className="label mr-2">Status</span>
16 + {(["draft", "reserved", "published", "withdrawn", "superseded"] as const).filter((st) => st !== p.status).map((st) => <form key={st} action={setStatusAction} className="inline-flex gap-1 items-center"><Csrf token={csrf} /><input type="hidden" name="id" value={p.id} /><input type="hidden" name="status" value={st} />{(st === "withdrawn" || st === "superseded") && <input name="reason" placeholder="reason (public)" className="input !w-44 !py-1 !text-xs" />}<button className={`btn btn-sm ${st === "published" ? "btn-primary" : ""}`} {...(st === "published" && !p.versions.some((v: Record<string, any>) => v.file_id) ? { title: "Upload a manuscript first (allowed without, but no PDF will be offered)" } : {})}>{st === "published" ? "Publish" : `Mark ${st}`}</button></form>)}
17 + {p.status !== "published" && <form action={deletePaperAction} className="ml-auto"><Csrf token={csrf} /><input type="hidden" name="id" value={p.id} /><button className="btn btn-sm text-bad">Delete draft</button></form>}
18 + <p className="w-full text-xs text-fg-3 mt-1">Publishing requires explicit approval here; nothing is published automatically. Withdrawn papers keep their record and identifier.</p>
19 + </section>
20 + <PaperForm p={p} tax={tax} csrf={csrf} action={updatePaperAction} />
21 + <section id="versions" className="card p-4 mt-8"><h2 className="text-lg mb-3">Versions</h2>
22 + <table className="data"><thead><tr><th>#</th><th>Label</th><th>Date</th><th>Description</th><th>File</th><th>Actions</th></tr></thead><tbody>{p.versions.map((v: Record<string, any>) => <tr key={v.id}><td>{v.version_no}{v.is_current && <span className="chip chip-navy ml-1">current</span>}{v.withdrawn && <span className="chip chip-burgundy ml-1">withdrawn</span>}</td>
23 + <td colSpan={3}><form action={versionAction} className="grid sm:grid-cols-[6rem_8rem_1fr_auto] gap-2 items-end"><Csrf token={csrf} /><input type="hidden" name="version_id" value={v.id} /><input type="hidden" name="op" value="update" /><input name="label" defaultValue={v.label} className="input !py-1 !text-xs" aria-label="label" /><input name="released_on" defaultValue={v.released_on ? (v.released_precision === "month" ? String(v.released_on).slice(0, 7) : v.released_precision === "year" ? String(v.released_on).slice(0, 4) : String(v.released_on).slice(0, 10)) : ""} className="input !py-1 !text-xs" aria-label="date" /><input name="description" defaultValue={v.description ?? ""} className="input !py-1 !text-xs" aria-label="description" /><button className="btn btn-sm">Save</button></form></td>
24 + <td className="text-xs">{v.file_id ? <>{v.pages ?? "?"} p. · {v.integrity_ok === false ? <span className="text-bad">integrity failed</span> : v.integrity_ok ? "ok" : "pending"} · text {v.text_status}<br /><code>{String(v.sha256 ?? "").slice(0, 12)}</code> · <a href={`/api/files/${v.file_id}/${p.slug}-${v.label}.pdf`} className="text-navy">PDF</a></> : "—"}</td>
25 + <td className="whitespace-nowrap"><div className="flex gap-1">{!v.is_current && <form action={versionAction}><Csrf token={csrf} /><input type="hidden" name="version_id" value={v.id} /><input type="hidden" name="op" value="make_current" /><button className="btn btn-sm">Make current</button></form>}
26 + {v.withdrawn ? <form action={versionAction}><Csrf token={csrf} /><input type="hidden" name="version_id" value={v.id} /><input type="hidden" name="op" value="restore" /><button className="btn btn-sm">Restore</button></form> : <form action={versionAction} className="flex gap-1"><Csrf token={csrf} /><input type="hidden" name="version_id" value={v.id} /><input type="hidden" name="op" value="withdraw" /><input name="reason" placeholder="legal/privacy reason" className="input !w-36 !py-1 !text-xs" /><button className="btn btn-sm text-bad">Withdraw</button></form>}</div></td></tr>)}</tbody></table>
27 + <h3 className="text-base mt-6 mb-2">Upload a new version (PDF)</h3><UploadForm paperId={p.id} csrf={csrf} nextNo={(p.versions[0]?.version_no ?? 0) + 1} />
28 + </section>
29 + <section id="relations" className="card p-4 mt-8 grid md:grid-cols-2 gap-6">
30 + <div><h2 className="text-lg mb-3">Relations ({p.relations.length})</h2><ul className="text-sm space-y-1 max-h-72 overflow-auto">{p.relations.map((r: Record<string, any>) => <li key={r.id} className="flex gap-2 items-start"><span className="chip shrink-0">{r.kind}{r.inferred ? " · inferred" : ""}</span><span className="text-fg-2"><span className="font-mono text-navy">{r.identifier}</span> {r.title} <span className="text-fg-3">— {r.explanation}</span></span>{r.kind === "curated" && <form action={relationAction}><Csrf token={csrf} /><input type="hidden" name="paper_id" value={p.id} /><input type="hidden" name="op" value="remove" /><input type="hidden" name="relation_id" value={r.id} /><button className="text-bad text-xs">remove</button></form>}</li>)}</ul></div>
31 + <form action={relationAction} className="space-y-2"><h3 className="text-base">Add a curated relation (both directions)</h3><Csrf token={csrf} /><input type="hidden" name="paper_id" value={p.id} /><input type="hidden" name="op" value="add" /><Field label="Target identifier or slug" name="to_slug" required /><Field label="Explanation (EN)" name="explanation" /><Field label="Explanation (FR)" name="explanation_fr" /><button className="btn btn-sm">Add</button></form>
32 + </section>
33 + <section className="card p-4 mt-8"><h2 className="text-lg mb-3">Redirects to this paper</h2><ul className="text-sm">{p.redirects.map((r: Record<string, any>) => <li key={r.from_path} className="flex gap-2 items-center"><code>{r.from_path}</code> → <code>{r.to_path}</code> ({r.status})<form action={redirectAction}><Csrf token={csrf} /><input type="hidden" name="op" value="remove" /><input type="hidden" name="from_path" value={r.from_path} /><input type="hidden" name="back" value={`/admin/papers/${p.id}`} /><button className="text-bad text-xs ml-2">remove</button></form></li>)}</ul>
34 + <form action={redirectAction} className="flex flex-wrap gap-2 items-end mt-3"><Csrf token={csrf} /><input type="hidden" name="op" value="add" /><input type="hidden" name="to_path" value={`/papers/${p.slug}`} /><input type="hidden" name="back" value={`/admin/papers/${p.id}`} /><Field label="Historical path (e.g. /research/old-slug)" name="from_path" required /><Field label="Note" name="note" /><button className="btn btn-sm">Add redirect</button></form></section>
35 + </div>);
36 +}
added src/app/admin/papers/[id]/upload-form.tsx +20 −0
@@ -0,0 +1,20 @@
1 +"use client";
2 +import { useState } from "react";
3 +export function UploadForm({ paperId, csrf, nextNo }: { paperId: string; csrf: string; nextNo: number }) {
4 + const [msg, setMsg] = useState<string | null>(null); const [busy, setBusy] = useState(false);
5 + const onSubmit = async (e: React.FormEvent<HTMLFormElement>) => {
6 + e.preventDefault(); setBusy(true); setMsg(null); const fd = new FormData(e.currentTarget);
7 + try { const r = await fetch("/api/admin/upload", { method: "POST", body: fd, headers: { "x-csrf-token": csrf } }); const j = await r.json(); if (!r.ok) throw new Error(j.error ?? r.statusText); setMsg(`Uploaded ${j.label} (sha256 ${String(j.sha256).slice(0, 12)}…). Processing queued.`); setTimeout(() => location.reload(), 1200); }
8 + catch (err) { setMsg("Error: " + (err as Error).message); } finally { setBusy(false); }
9 + };
10 + return (<form onSubmit={onSubmit} className="grid sm:grid-cols-[1fr_8rem_10rem_1fr_auto] gap-2 items-end text-sm">
11 + <input type="hidden" name="paper_id" value={paperId} />
12 + <label className="block"><span className="label">PDF</span><input type="file" name="file" accept="application/pdf" required className="input mt-1 !py-1.5" /></label>
13 + <label className="block"><span className="label">Label</span><input name="label" defaultValue={`v${nextNo}`} className="input mt-1" /></label>
14 + <label className="block"><span className="label">Date (YYYY-MM[-DD])</span><input name="released_on" className="input mt-1" /></label>
15 + <label className="block"><span className="label">Description</span><input name="description" className="input mt-1" placeholder="Revised manuscript" /></label>
16 + <label className="flex items-center gap-2 pb-2"><input type="checkbox" name="make_current" defaultChecked /> current</label>
17 + <button className="btn btn-primary btn-sm sm:col-span-5 justify-self-start" disabled={busy}>{busy ? "Uploading…" : "Upload version"}</button>
18 + {msg && <p className="sm:col-span-5 text-xs text-fg-2">{msg}</p>}
19 + </form>);
20 +}
added src/app/admin/papers/new/page.tsx +7 −0
@@ -0,0 +1,7 @@
1 +import { taxonomy } from "@/lib/admin/queries";
2 +import { requireUserPage } from "@/lib/auth/guard";
3 +import { csrfToken } from "@/lib/auth/session";
4 +import { createPaperAction } from "@/lib/admin/actions";
5 +import { PageTitle } from "@/components/admin-ui";
6 +import { PaperForm } from "../paper-form";
7 +export default async function NewPaper() { const u = await requireUserPage(); const tax = await taxonomy(); return <div><PageTitle title="New working paper" /><p className="meta mb-4">Created as a draft. Upload a manuscript and publish explicitly from the paper page.</p><PaperForm p={null} tax={tax} csrf={csrfToken(u)} action={createPaperAction} /></div>; }
added src/app/admin/papers/page.tsx +10 −0
@@ -0,0 +1,10 @@
1 +import Link from "next/link";
2 +import { adminPaperList } from "@/lib/admin/queries";
3 +import { Notice, PageTitle } from "@/components/admin-ui";
4 +export default async function AdminPapers({ searchParams }: { searchParams: Promise<Record<string, string | undefined>> }) {
5 + const sp = await searchParams; const rows = await adminPaperList(sp.q || undefined, sp.status || undefined);
6 + return (<div><PageTitle title={`Papers (${rows.length})`}><Link href="/admin/papers/new" className="btn btn-primary btn-sm no-underline">New paper</Link></PageTitle><Notice sp={sp} />
7 + <form className="flex flex-wrap gap-2 mb-4"><input name="q" defaultValue={sp.q ?? ""} placeholder="Search title or identifier" className="input !w-64" /><select name="status" defaultValue={sp.status ?? ""} className="input !w-auto"><option value="">All statuses</option>{["published", "draft", "reserved", "withdrawn", "superseded"].map((s) => <option key={s} value={s}>{s}</option>)}</select><button className="btn btn-sm">Filter</button></form>
8 + <div className="card overflow-x-auto"><table className="data"><thead><tr><th>ID</th><th>Title</th><th>Authors</th><th>Status</th><th>Version</th><th>PDF</th><th>Views / DL</th><th>Updated</th></tr></thead>
9 + <tbody>{rows.map((p) => <tr key={p.id}><td className="font-mono text-navy whitespace-nowrap"><Link href={`/admin/papers/${p.id}`} className="no-underline hover:underline">{p.identifier}</Link></td><td><Link href={`/admin/papers/${p.id}`} className="text-fg no-underline hover:underline">{p.title}</Link>{p.featured ? <span className="chip chip-burgundy ml-2">featured</span> : null}</td><td className="text-fg-2 text-xs">{p.authors}</td><td><span className={`chip ${p.status === "published" ? "chip-navy" : p.status === "withdrawn" ? "chip-burgundy" : ""}`}>{p.status}</span></td><td>{p.version_label ?? "—"}</td><td>{p.has_pdf ? "✓" : "—"}</td><td className="whitespace-nowrap">{p.view_count} / {p.download_count}</td><td className="whitespace-nowrap text-xs">{p.updated_at.slice(0, 16)}</td></tr>)}</tbody></table></div></div>);
10 +}
added src/app/admin/papers/paper-form.tsx +53 −0
@@ -0,0 +1,53 @@
1 +import { Check, Csrf, Field, Select } from "@/components/admin-ui";
2 +import type { taxonomy } from "@/lib/admin/queries";
3 +type Tax = Awaited<ReturnType<typeof taxonomy>>;
4 +const dateStr = (d: string | null | undefined, p: string | undefined) => (!d ? "" : p === "year" ? String(d).slice(0, 4) : p === "month" ? String(d).slice(0, 7) : String(d).slice(0, 10));
5 +export function PaperForm({ p, tax, csrf, action }: { p: Record<string, any> | null; tax: Tax; csrf: string; action: (f: FormData) => Promise<void> }) {
6 + const sel = (k: string) => new Set<number>(((p?.[k] as { id: number }[]) ?? []).map((x) => x.id));
7 + const j = (v: unknown) => (v ? JSON.stringify(v, null, 1) : "");
8 + return (
9 + <form action={action} className="space-y-6">
10 + <Csrf token={csrf} />{p && <input type="hidden" name="id" value={p.id} />}
11 + <section className="card p-4 grid gap-4 sm:grid-cols-4">
12 + <Select label="Series" name="series_code" value={p?.series_code ?? "WP"} options={tax.series.map((s) => ({ v: s.code, label: `${s.code} — ${s.name_en}` }))} />
13 + <Field label="Number" name="number" type="number" value={p?.number} required hint="Permanent. Never renumber; gaps are fine." />
14 + <Field label="Internal code" name="internal_code" value={p?.internal_code} hint="e.g. repository name; not proof of release" />
15 + <Field label="Language" name="language" value={p?.language ?? "en"} />
16 + <Field label="Title" name="title" value={p?.title} required className="sm:col-span-4" />
17 + <Field label="Subtitle" name="subtitle" value={p?.subtitle} className="sm:col-span-4" />
18 + <Field label="Title (FR)" name="title_fr" value={p?.title_fr} className="sm:col-span-2" /><Field label="Subtitle (FR)" name="subtitle_fr" value={p?.subtitle_fr} className="sm:col-span-2" />
19 + <Field label="Abstract (paragraphs separated by a blank line)" name="abstract" value={(p?.abstract as string[] | undefined)?.join("\n\n")} rows={10} className="sm:col-span-4" />
20 + <Field label="Abstract (FR)" name="abstract_fr" value={(p?.abstract_fr as string[] | undefined)?.join("\n\n")} rows={6} className="sm:col-span-4" />
21 + <Field label="Research question" name="research_question" value={p?.research_question} rows={2} className="sm:col-span-2" /><Field label="Research question (FR)" name="research_question_fr" value={p?.research_question_fr} rows={2} className="sm:col-span-2" />
22 + <Field label="Headline (one-sentence result)" name="hero_headline" value={p?.hero_headline} className="sm:col-span-2" /><Field label="Sub-headline" name="hero_subheadline" value={p?.hero_subheadline} rows={2} className="sm:col-span-2" />
23 + <Field label="Notice (e.g. Working draft)" name="notice" value={p?.notice} className="sm:col-span-2" /><Field label="Notice (FR)" name="notice_fr" value={p?.notice_fr} className="sm:col-span-2" />
24 + </section>
25 + <section className="card p-4 grid gap-4 sm:grid-cols-4">
26 + <Field label="Year" name="year" type="number" value={p?.year} />
27 + <Field label="First release (YYYY, YYYY-MM or YYYY-MM-DD)" name="first_released_on" value={dateStr(p?.first_released_on, p?.first_released_precision)} hint="Precision is kept as entered; never guess a day." />
28 + <Field label="Latest revision" name="last_revised_on" value={dateStr(p?.last_revised_on, p?.last_revised_precision)} />
29 + <Select label="Publication status" name="publication_status" value={p?.publication_status ?? "working_paper"} options={[{ v: "working_paper", label: "Working paper" }, { v: "under_review", label: "Under review" }, { v: "forthcoming", label: "Forthcoming" }, { v: "published", label: "Published in a journal" }]} />
30 + <Field label="Journal reference" name="journal_ref" value={p?.journal_ref} className="sm:col-span-2" /><Field label="DOI" name="doi" value={p?.doi} /><Field label="SSRN abstract id" name="ssrn_id" value={p?.ssrn_id} />
31 + <Field label="Keywords (one per line)" name="keywords" value={(p?.keywords as string[] | undefined)?.join("\n")} rows={5} className="sm:col-span-2" /><Field label="JEL codes (comma-separated)" name="jel" value={(p?.jel as string[] | undefined)?.join(", ")} className="sm:col-span-2" />
32 + <Select label="Primary domain" name="primary_domain_id" value={p?.primary_domain_id ?? ""} options={[{ v: "", label: "—" }, ...tax.topics.filter((t) => t.kind === "domain").map((t) => ({ v: t.id, label: t.name_en }))]} />
33 + <div className="flex items-end gap-4"><Check label="Featured on home" name="featured" checked={p?.featured} /><Field label="Rank" name="featured_rank" type="number" value={p?.featured_rank} /></div>
34 + </section>
35 + <section className="card p-4 grid gap-4 md:grid-cols-2">
36 + <div><p className="label mb-2">Authors (order = position)</p><div className="grid gap-1 max-h-48 overflow-auto">{tax.authors.map((a) => <Check key={a.id} label={a.display_name} name="author_ids" checked={sel("authors").has(a.id)} />)}</div><input type="hidden" name="author_ids_hint" value="" /></div>
37 + <div><p className="label mb-2">Topics and domains</p><div className="grid gap-1 max-h-48 overflow-auto">{tax.topics.map((t) => <Check key={t.id} label={`${t.kind === "domain" ? "◆ " : ""}${t.name_en}`} name="topic_ids" checked={sel("topics").has(t.id)} />)}</div></div>
38 + <div><p className="label mb-2">Methods</p><div className="grid gap-1 max-h-48 overflow-auto">{tax.methods.map((m) => <Check key={m.id} label={m.name_en} name="method_ids" checked={sel("methods").has(m.id)} />)}</div></div>
39 + <div><p className="label mb-2">Datasets</p><div className="grid gap-1 max-h-48 overflow-auto">{tax.datasets.map((d) => <Check key={d.id} label={d.name_en} name="dataset_ids" checked={sel("datasets").has(d.id)} />)}</div></div>
40 + </section>
41 + <section className="card p-4 grid gap-4 md:grid-cols-2">
42 + <Field label='Findings — JSON [{"value","label"}]' name="findings" value={j(p?.findings)} rows={8} />
43 + <Field label='Contributions — JSON [{"kind?","title","description"}]' name="contributions" value={j(p?.contributions)} rows={8} />
44 + <Field label='Methodology — JSON [{"title","description"}] (verified fields only)' name="methodology" value={j(p?.methodology)} rows={8} />
45 + <Field label='Data sections — JSON [{"title","description"}]' name="data_sections" value={j(p?.data_sections)} rows={8} />
46 + <Field label="Reproducibility notes (one per line)" name="reproducibility" value={(p?.reproducibility as string[] | undefined)?.join("\n")} rows={5} />
47 + <Field label='Artifacts — JSON [{"kind","label","url"}] (code, appendix, slides…)' name="artifacts" value={j(p?.artifacts)} rows={5} />
48 + <Field label='External links — JSON [{"kind","label","url"}]' name="external_links" value={j(p?.external_links)} rows={4} />
49 + </section>
50 + <div className="flex gap-2"><button className="btn btn-primary">{p ? "Save changes" : "Create draft"}</button></div>
51 + </form>
52 + );
53 +}
added src/app/admin/settings/page.tsx +17 −0
@@ -0,0 +1,17 @@
1 +import { sql } from "@/db";
2 +import { requireUserPage } from "@/lib/auth/guard";
3 +import { csrfToken } from "@/lib/auth/session";
4 +import { settingsAction } from "@/lib/admin/actions";
5 +import { Check, Csrf, Field, Notice, PageTitle } from "@/components/admin-ui";
6 +export default async function Settings({ searchParams }: { searchParams: Promise<Record<string, string | undefined>> }) {
7 + const sp = await searchParams; const u = await requireUserPage(); const rows = await sql`SELECT key, value FROM settings WHERE key NOT LIKE 'private.%'`; const g = (k: string) => (rows.find((r) => r.key === k)?.value ?? {}) as Record<string, any>;
8 + const attr = g("site.attribution"), inst = g("site.institution"), contact = g("site.contact");
9 + return (<div><PageTitle title="Settings" /><Notice sp={sp} />
10 + <form action={settingsAction} className="space-y-6 max-w-4xl"><Csrf token={csrfToken(u)} />
11 + <section className="card p-4 space-y-3"><h2 className="text-lg">Institutional attribution block</h2><p className="meta">Shown in the footer and on the About page. Keep the distinction between the author's affiliation and an official institutional repository.</p><Field label="Attribution (EN)" name="attribution_en" value={attr.en} rows={4} /><Field label="Attribution (FR)" name="attribution_fr" value={attr.fr} rows={4} />
12 + <div className="grid sm:grid-cols-2 gap-3"><Field label="Institution name (EN)" name="inst_name_en" value={inst.name_en} /><Field label="Institution name (FR)" name="inst_name_fr" value={inst.name_fr} /><Field label="Department (EN)" name="inst_dept_en" value={inst.department_en} /><Field label="Department (FR)" name="inst_dept_fr" value={inst.department_fr} /><Field label="Institution URL" name="inst_url" value={inst.url} /></div>
13 + <Check label="Official logo use authorized (confirm permission and visual-identity rules before enabling; the logo file must be added to /public/brand)" name="logo_authorized" checked={inst.logo_authorized} /></section>
14 + <section className="card p-4 grid sm:grid-cols-3 gap-3"><h2 className="text-lg sm:col-span-3">Contact and home</h2><Field label="Contact e-mail" name="contact_email" value={contact.email} /><Field label="Website" name="contact_website" value={contact.website} /><Field label="Featured papers on home (max)" name="featured_limit" type="number" value={g("home.featured_limit") as unknown as number} /></section>
15 + <section className="card p-4"><h2 className="text-lg">Ingestion directories</h2><Field label="Additional absolute directories to scan for PDFs (one per line; the storage inbox is always scanned)" name="ingest_dirs" value={((rows.find((r) => r.key === "ingest.extra_dirs")?.value as string[]) ?? []).join("\n")} rows={3} /></section>
16 + <button className="btn btn-primary">Save settings</button></form></div>);
17 +}
added src/app/admin/system/page.tsx +17 −0
@@ -0,0 +1,17 @@
1 +import { sql } from "@/db";
2 +import { requireUserPage } from "@/lib/auth/guard";
3 +import { csrfToken } from "@/lib/auth/session";
4 +import { systemAction } from "@/lib/admin/actions";
5 +import { Csrf, Notice, PageTitle } from "@/components/admin-ui";
6 +import { embeddingsEnabled, verifyEmbeddingModel } from "@/lib/ai/embeddings";
7 +import { STORAGE_DIR } from "@/lib/env";
8 +export default async function System({ searchParams }: { searchParams: Promise<Record<string, string | undefined>> }) {
9 + const sp = await searchParams; const u = await requireUserPage(); const csrf = csrfToken(u);
10 + const [jobs, backups, [storage], ai] = await Promise.all([sql`SELECT * FROM jobs ORDER BY created_at DESC LIMIT 60`, sql`SELECT * FROM backups ORDER BY created_at DESC LIMIT 15`, sql`SELECT count(*)::int AS files, coalesce(sum(bytes), 0)::bigint AS bytes FROM files`, embeddingsEnabled() ? verifyEmbeddingModel() : Promise.resolve({ ok: false, model: "—", reason: "AI provider disabled (AI_PROVIDER=none)" })]);
11 + const Btn = ({ op, label }: { op: string; label: string }) => <form action={systemAction}><Csrf token={csrf} /><input type="hidden" name="op" value={op} /><button className="btn btn-sm">{label}</button></form>;
12 + return (<div><PageTitle title="System" /><Notice sp={sp} />
13 + <div className="flex flex-wrap gap-2 mb-6"><Btn op="backup" label="Run backup now" /><Btn op="relations" label="Rebuild relations" /><Btn op="stats" label="Aggregate stats now" /><Btn op="embed_all" label="Re-embed all papers" /><Btn op="retry" label="Retry failed jobs" /></div>
14 + <div className="grid md:grid-cols-3 gap-3 mb-6"><div className="card p-4 text-sm"><p className="label">Storage</p><p>{storage.files} files · {(Number(storage.bytes) / 1e6).toFixed(1)} MB</p><p className="text-xs text-fg-3 break-all">{STORAGE_DIR}</p></div><div className="card p-4 text-sm"><p className="label">AI embeddings</p><p>{ai.ok ? <span className="text-ok">OK · {ai.model}</span> : <span className="text-warn">off · {ai.reason}</span>}</p><p className="text-xs text-fg-3">Optional. Search and relations degrade to lexical signals without it.</p></div><div className="card p-4 text-sm"><p className="label">Worker schedule</p><p>stats 00:10 · relations 03:00 · backup 04:40 (node local time)</p></div></div>
15 + <section className="card p-4 mb-6"><h2 className="text-lg mb-2">Backups</h2><table className="data"><thead><tr><th>When</th><th>OK</th><th>Remote</th><th>Size</th><th>Log</th></tr></thead><tbody>{backups.map((b) => <tr key={b.id}><td className="whitespace-nowrap text-xs">{new Date(b.created_at).toLocaleString("en-CA")}</td><td>{b.ok ? "✓" : <span className="text-bad">✗</span>}</td><td>{b.remote_ok === null ? "—" : b.remote_ok ? "✓" : <span className="text-bad">✗</span>}</td><td>{(Number(b.bytes) / 1e6).toFixed(1)} MB</td><td className="text-xs text-fg-3 whitespace-pre-wrap">{b.log}</td></tr>)}</tbody></table></section>
16 + <section className="card p-4"><h2 className="text-lg mb-2">Jobs (latest 60)</h2><table className="data"><thead><tr><th>ID</th><th>Kind</th><th>Status</th><th>Attempts</th><th>Created</th><th>Finished</th><th>Error</th></tr></thead><tbody>{jobs.map((j) => <tr key={j.id}><td>{j.id}</td><td className="font-mono text-xs">{j.kind}</td><td><span className={`chip ${j.status === "failed" ? "chip-burgundy" : j.status === "done" ? "" : "chip-navy"}`}>{j.status}</span></td><td>{j.attempts}</td><td className="text-xs whitespace-nowrap">{new Date(j.created_at).toLocaleString("en-CA")}</td><td className="text-xs whitespace-nowrap">{j.finished_at ? new Date(j.finished_at).toLocaleString("en-CA") : ""}</td><td className="text-xs text-bad max-w-md truncate">{j.last_error ?? ""}</td></tr>)}</tbody></table></section></div>);
17 +}
added src/app/admin/taxonomy/page.tsx +29 −0
@@ -0,0 +1,29 @@
1 +import { taxonomy } from "@/lib/admin/queries";
2 +import { requireUserPage } from "@/lib/auth/guard";
3 +import { csrfToken } from "@/lib/auth/session";
4 +import { taxonomyAction } from "@/lib/admin/actions";
5 +import { Check, Csrf, Field, Notice, PageTitle, Select } from "@/components/admin-ui";
6 +export default async function Taxonomy({ searchParams }: { searchParams: Promise<Record<string, string | undefined>> }) {
7 + const sp = await searchParams; const u = await requireUserPage(); const csrf = csrfToken(u); const tax = await taxonomy();
8 + const Del = ({ kind, id }: { kind: string; id: number }) => <form action={taxonomyAction} className="inline"><Csrf token={csrf} /><input type="hidden" name="kind" value={kind} /><input type="hidden" name="op" value="delete" /><input type="hidden" name="id" value={id} /><button className="text-bad text-xs">delete</button></form>;
9 + return (<div><PageTitle title="Taxonomy" /><Notice sp={sp} /><p className="meta mb-6">Authors, research domains and topics, methods, datasets and series. Editing a name updates every paper that uses it. Deleting a topic/method/dataset removes its assignments; authors with papers cannot be deleted.</p>
10 + <div className="grid xl:grid-cols-2 gap-6">
11 + <section className="card p-4"><h2 className="text-lg mb-2">Authors ({tax.authors.length})</h2>
12 + {tax.authors.map((a) => <details key={a.id} className="border-b border-line py-2"><summary className="text-sm flex justify-between"><span>{a.display_name}{a.is_primary ? <span className="chip chip-navy ml-2">primary</span> : null}</span><Del kind="author" id={a.id} /></summary>
13 + <form action={taxonomyAction} className="grid sm:grid-cols-2 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="author" /><input type="hidden" name="id" value={a.id} /><Field label="Given name" name="given_name" value={a.given_name} required /><Field label="Family name" name="family_name" value={a.family_name} required /><Field label="Display name" name="display_name" value={a.display_name} /><Field label="Slug" name="slug" value={a.slug} /><Field label="ORCID" name="orcid" value={a.orcid} /><Field label="E-mail" name="email" value={a.email} /><Field label="Website" name="website" value={a.website} /><Field label="Google Scholar" name="google_scholar" value={a.google_scholar} /><Field label="SSRN author URL" name="ssrn_url" value={a.ssrn_url} /><Field label="GitHub" name="github" value={a.github} /><Field label="Affiliation (EN)" name="affiliation_en" value={a.affiliation_en} /><Field label="Affiliation (FR)" name="affiliation_fr" value={a.affiliation_fr} /><Field label="Department (EN)" name="department_en" value={a.department_en} /><Field label="Department (FR)" name="department_fr" value={a.department_fr} /><Field label="Bio (EN)" name="bio_en" value={a.bio_en} rows={3} /><Field label="Bio (FR)" name="bio_fr" value={a.bio_fr} rows={3} /><Check label="Primary author of the observatory (excluded from co-author links)" name="is_primary" checked={a.is_primary} /><Field label="Sort" name="sort" type="number" value={a.sort} /><button className="btn btn-sm justify-self-start">Save</button></form></details>)}
14 + <details className="py-2"><summary className="text-sm text-navy">+ New author</summary><form action={taxonomyAction} className="grid sm:grid-cols-2 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="author" /><Field label="Given name" name="given_name" required /><Field label="Family name" name="family_name" required /><Field label="ORCID" name="orcid" /><Field label="Affiliation (EN)" name="affiliation_en" /><Field label="Affiliation (FR)" name="affiliation_fr" /><Field label="Website" name="website" /><button className="btn btn-sm justify-self-start">Create</button></form></details></section>
15 + <section className="card p-4"><h2 className="text-lg mb-2">Domains and topics ({tax.topics.length})</h2>
16 + {tax.topics.map((t) => <details key={t.id} className="border-b border-line py-2"><summary className="text-sm flex justify-between"><span>{t.kind === "domain" ? "◆ " : ""}{t.name_en} <span className="text-fg-3">· {t.n}</span></span><Del kind="topic" id={t.id} /></summary>
17 + <form action={taxonomyAction} className="grid sm:grid-cols-2 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="topic" /><input type="hidden" name="id" value={t.id} /><Field label="Name (EN)" name="name_en" value={t.name_en} required /><Field label="Name (FR)" name="name_fr" value={t.name_fr} /><Field label="Slug" name="slug" value={t.slug} /><Select label="Kind" name="topic_kind" value={t.kind} options={[{ v: "domain", label: "Research domain" }, { v: "topic", label: "Topic" }]} /><Field label="Description (EN)" name="description_en" value={t.description_en} rows={2} /><Field label="Description (FR)" name="description_fr" value={t.description_fr} rows={2} /><Field label="Accent colour" name="accent" value={t.accent} /><Field label="Sort" name="sort" type="number" value={t.sort} /><button className="btn btn-sm justify-self-start">Save</button></form></details>)}
18 + <details className="py-2"><summary className="text-sm text-navy">+ New domain or topic</summary><form action={taxonomyAction} className="grid sm:grid-cols-2 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="topic" /><Field label="Name (EN)" name="name_en" required /><Field label="Name (FR)" name="name_fr" /><Select label="Kind" name="topic_kind" value="topic" options={[{ v: "domain", label: "Research domain" }, { v: "topic", label: "Topic" }]} /><Field label="Accent colour" name="accent" /><button className="btn btn-sm justify-self-start">Create</button></form></details></section>
19 + <section className="card p-4"><h2 className="text-lg mb-2">Methods ({tax.methods.length})</h2>
20 + {tax.methods.map((m) => <details key={m.id} className="border-b border-line py-2"><summary className="text-sm flex justify-between"><span>{m.name_en} <span className="text-fg-3">· {m.category} · {m.n}</span></span><Del kind="method" id={m.id} /></summary><form action={taxonomyAction} className="grid sm:grid-cols-2 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="method" /><input type="hidden" name="id" value={m.id} /><Field label="Name (EN)" name="name_en" value={m.name_en} required /><Field label="Name (FR)" name="name_fr" value={m.name_fr} /><Field label="Slug" name="slug" value={m.slug} /><Field label="Category" name="category" value={m.category} /><Field label="Description (EN)" name="description_en" value={m.description_en} rows={2} /><Field label="Description (FR)" name="description_fr" value={m.description_fr} rows={2} /><button className="btn btn-sm justify-self-start">Save</button></form></details>)}
21 + <details className="py-2"><summary className="text-sm text-navy">+ New method</summary><form action={taxonomyAction} className="grid sm:grid-cols-2 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="method" /><Field label="Name (EN)" name="name_en" required /><Field label="Name (FR)" name="name_fr" /><Field label="Category" name="category" value="general" /><button className="btn btn-sm justify-self-start">Create</button></form></details></section>
22 + <section className="card p-4"><h2 className="text-lg mb-2">Datasets ({tax.datasets.length})</h2>
23 + {tax.datasets.map((d) => <details key={d.id} className="border-b border-line py-2"><summary className="text-sm flex justify-between"><span>{d.name_en} <span className="text-fg-3">· {d.n}</span></span><Del kind="dataset" id={d.id} /></summary><form action={taxonomyAction} className="grid sm:grid-cols-2 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="dataset" /><input type="hidden" name="id" value={d.id} /><Field label="Name (EN)" name="name_en" value={d.name_en} required /><Field label="Name (FR)" name="name_fr" value={d.name_fr} /><Field label="Slug" name="slug" value={d.slug} /><Field label="Provider" name="provider" value={d.provider} /><Field label="Coverage" name="coverage" value={d.coverage} /><Field label="Frequency" name="frequency" value={d.frequency} /><Field label="Access conditions" name="access_conditions" value={d.access_conditions} /><Field label="License" name="license" value={d.license} /><Field label="URL" name="url" value={d.url} /><Check label="Publicly available" name="is_public" checked={d.is_public} /><Field label="Description (EN)" name="description_en" value={d.description_en} rows={2} /><Field label="Description (FR)" name="description_fr" value={d.description_fr} rows={2} /><button className="btn btn-sm justify-self-start">Save</button></form></details>)}
24 + <details className="py-2"><summary className="text-sm text-navy">+ New dataset</summary><form action={taxonomyAction} className="grid sm:grid-cols-2 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="dataset" /><Field label="Name (EN)" name="name_en" required /><Field label="Name (FR)" name="name_fr" /><Field label="Provider" name="provider" /><Field label="Access conditions" name="access_conditions" /><Check label="Publicly available" name="is_public" /><button className="btn btn-sm justify-self-start">Create</button></form></details></section>
25 + <section className="card p-4 xl:col-span-2"><h2 className="text-lg mb-2">Series ({tax.series.length})</h2>
26 + {tax.series.map((s) => <details key={s.id} className="border-b border-line py-2"><summary className="text-sm">{s.code} — {s.name_en}</summary><form action={taxonomyAction} className="grid sm:grid-cols-3 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="series" /><input type="hidden" name="id" value={s.id} /><Field label="Code" name="code" value={s.code} required /><Field label="Name (EN)" name="name_en" value={s.name_en} required /><Field label="Name (FR)" name="name_fr" value={s.name_fr} /><Field label="Slug" name="slug" value={s.slug} /><Field label="ISSN" name="issn" value={s.issn} /><Field label="Sort" name="sort" type="number" value={s.sort} /><Field label="Description (EN)" name="description_en" value={s.description_en} rows={2} className="sm:col-span-3" /><Field label="Description (FR)" name="description_fr" value={s.description_fr} rows={2} className="sm:col-span-3" /><button className="btn btn-sm justify-self-start">Save</button></form></details>)}
27 + <details className="py-2"><summary className="text-sm text-navy">+ New series</summary><form action={taxonomyAction} className="grid sm:grid-cols-3 gap-2 mt-2 text-sm"><Csrf token={csrf} /><input type="hidden" name="kind" value="series" /><Field label="Code (e.g. TN)" name="code" required /><Field label="Name (EN)" name="name_en" required /><Field label="Name (FR)" name="name_fr" /><button className="btn btn-sm justify-self-start">Create</button></form></details></section>
28 + </div></div>);
29 +}
added src/app/api/admin/preview/[sha]/route.ts +10 −0
@@ -0,0 +1,10 @@
1 +import { createReadStream, promises as fs } from "node:fs";
2 +import { Readable } from "node:stream";
3 +import { join } from "node:path";
4 +import { requireUser, jsonError } from "@/lib/auth/guard";
5 +import { STORAGE_DIR } from "@/lib/env";
6 +export const dynamic = "force-dynamic";
7 +export async function GET(req: Request, { params }: { params: Promise<{ sha: string }> }) {
8 + try { await requireUser(); const { sha } = await params; if (!/^[0-9a-f]{64}$/.test(sha)) return new Response("Not found", { status: 404 }); const p = join(STORAGE_DIR, "previews", `${sha}.png`); await fs.access(p); return new Response(Readable.toWeb(createReadStream(p)) as unknown as ReadableStream, { headers: { "Content-Type": "image/png", "Cache-Control": "private, max-age=3600" } }); }
9 + catch (e) { if ((e as { code?: string }).code === "ENOENT") return new Response("Not found", { status: 404 }); return jsonError(e); }
10 +}
added src/app/api/admin/upload/route.ts +36 −0
@@ -0,0 +1,36 @@
1 +import { promises as fs } from "node:fs";
2 +import { join } from "node:path";
3 +import { sql } from "@/db";
4 +import { requireUser, jsonError } from "@/lib/auth/guard";
5 +import { clientIp } from "@/lib/auth/session";
6 +import { audit } from "@/lib/audit";
7 +import { enqueue } from "@/lib/jobs/queue";
8 +import { parseVersionLabel } from "@/lib/papers/identifiers";
9 +import { ensureStorage, pdfIntegrity, sha256File, storeFile } from "@/lib/pdf/process";
10 +import { STORAGE_DIR } from "@/lib/env";
11 +import { headers } from "next/headers";
12 +export const dynamic = "force-dynamic";
13 +const MAX = 150 * 1024 * 1024;
14 +export async function POST(req: Request) {
15 + try {
16 + const u = await requireUser(req); const fd = await req.formData();
17 + const file = fd.get("file"); if (!(file instanceof File)) throw new Error("file missing"); if (file.size > MAX) throw new Error("file too large (max 150 MB)"); if (file.type && file.type !== "application/pdf") throw new Error("PDF only");
18 + const paperId = String(fd.get("paper_id") ?? ""); const [p] = await sql`SELECT id, slug FROM papers WHERE id = ${paperId}`; if (!p) throw new Error("paper not found");
19 + await ensureStorage(); const tmp = join(STORAGE_DIR, "tmp", `${crypto.randomUUID()}.pdf`); await fs.writeFile(tmp, Buffer.from(await file.arrayBuffer()), { mode: 0o600 });
20 + const head = await fs.readFile(tmp, { encoding: "latin1", flag: "r" }).then((s) => s.slice(0, 5)); if (!head.startsWith("%PDF")) { await fs.unlink(tmp); throw new Error("not a PDF file"); }
21 + const integ = await pdfIntegrity(tmp); if (!integ.ok) { await fs.unlink(tmp); throw new Error(`PDF integrity check failed: ${integ.reason}`); }
22 + const sha = await sha256File(tmp); let [f] = await sql`SELECT id FROM files WHERE sha256 = ${sha}`;
23 + if (!f) { const key = await storeFile(tmp, sha); [f] = await sql`INSERT INTO files (kind, storage_key, original_name, mime, bytes, sha256, pages, integrity_ok) VALUES ('manuscript', ${key}, ${file.name.slice(0, 200)}, 'application/pdf', ${file.size}, ${sha}, ${integ.info?.pages ?? null}, true) RETURNING id`; }
24 + await fs.unlink(tmp).catch(() => {});
25 + const [dupv] = await sql`SELECT label FROM paper_versions WHERE paper_id = ${p.id} AND sha256 = ${sha}`; if (dupv) throw new Error(`This exact file is already version ${dupv.label}`);
26 + const [{ next }] = await sql`SELECT coalesce(max(version_no), 0) + 1 AS next FROM paper_versions WHERE paper_id = ${p.id}`;
27 + const label = parseVersionLabel(String(fd.get("label") ?? "")) ?? { versionNo: next, label: `v${next}` };
28 + const dateRaw = String(fd.get("released_on") ?? "").trim(); const precision = /^\d{4}$/.test(dateRaw) ? "year" : /^\d{4}-\d{2}$/.test(dateRaw) ? "month" : "day"; const date = !dateRaw ? null : precision === "year" ? `${dateRaw}-01-01` : precision === "month" ? `${dateRaw}-01` : dateRaw;
29 + const makeCurrent = fd.get("make_current") === "on";
30 + const [v] = await sql`INSERT INTO paper_versions (paper_id, version_no, label, released_on, released_precision, description, file_id, pages, sha256, bytes, is_current) VALUES (${p.id}, ${next}, ${label.label}, ${date}, ${precision}, ${String(fd.get("description") ?? "") || null}, ${f.id}, ${integ.info?.pages ?? null}, ${sha}, ${file.size}, ${makeCurrent}) RETURNING id`;
31 + if (makeCurrent) { await sql`UPDATE paper_versions SET is_current = false WHERE paper_id = ${p.id} AND id <> ${v.id}`; await sql`UPDATE papers SET current_version_id = ${v.id}, last_revised_on = coalesce(${date}, last_revised_on), last_revised_precision = ${date ? precision : sql`last_revised_precision`} WHERE id = ${p.id}`; }
32 + await enqueue("process_pdf", { fileId: f.id, paperId: makeCurrent ? p.id : undefined, versionId: v.id });
33 + await audit({ id: u.id, label: u.email, ip: clientIp(await headers()) }, "version.upload", "paper_version", v.id, null, { paper: p.slug, label: label.label, sha256: sha, bytes: file.size });
34 + return Response.json({ ok: true, versionId: v.id, label: label.label, sha256: sha });
35 + } catch (e) { return jsonError(e); }
36 +}
added src/app/api/events/route.ts +12 −0
@@ -0,0 +1,12 @@
1 +import { z } from "zod";
2 +import { sqlPublic } from "@/db";
3 +import { recordEvent } from "@/lib/papers/events";
4 +import { ip, limited } from "@/lib/api";
5 +const Body = z.object({ kind: z.literal("view"), slug: z.string().regex(/^[a-z0-9]{2,20}$/) });
6 +export async function POST(req: Request) {
7 + if (limited(req, "events", 60)) return new Response(null, { status: 204 });
8 + let b; try { b = Body.parse(await req.json()); } catch { return new Response(null, { status: 204 }); }
9 + const [p] = await sqlPublic`SELECT id FROM public_papers WHERE slug = ${b.slug} AND status = 'published'`; if (!p) return new Response(null, { status: 204 });
10 + await recordEvent("view", p.id, { ip: ip(req), ua: req.headers.get("user-agent"), referrer: req.headers.get("referer"), locale: req.headers.get("x-locale") ?? undefined }).catch(() => {});
11 + return new Response(null, { status: 204 });
12 +}
added src/app/api/files/[fileId]/[name]/route.ts +26 −0
@@ -0,0 +1,26 @@
1 +import { createReadStream, promises as fs } from "node:fs";
2 +import { Readable } from "node:stream";
3 +import { publicFile } from "@/lib/papers/queries";
4 +import { recordEvent } from "@/lib/papers/events";
5 +import { storagePath } from "@/lib/pdf/process";
6 +import { ip, limited, tooMany } from "@/lib/api";
7 +export const dynamic = "force-dynamic";
8 +/** Serves a published manuscript version (content-addressed on disk). Counts one download per visitor per paper per day. Supports Range and HEAD. */
9 +export async function GET(req: Request, ctx: { params: Promise<{ fileId: string; name: string }> }) { return serve(req, ctx, false); }
10 +export async function HEAD(req: Request, ctx: { params: Promise<{ fileId: string; name: string }> }) { return serve(req, ctx, true); }
11 +async function serve(req: Request, { params }: { params: Promise<{ fileId: string; name: string }> }, head: boolean) {
12 + if (limited(req, "files", 60)) return tooMany();
13 + const { fileId, name } = await params; if (!/^[0-9a-f-]{36}$/.test(fileId)) return new Response("Not found", { status: 404 });
14 + const f = await publicFile(fileId); if (!f) return new Response("Not found", { status: 404 });
15 + const path = storagePath(f.storage_key); let st; try { st = await fs.stat(path); } catch { return new Response("Missing file", { status: 404 }); }
16 + const inline = new URL(req.url).searchParams.get("inline") === "1";
17 + const filename = name.endsWith(".pdf") ? name.replace(/[^A-Za-z0-9._-]/g, "_") : `${f.slug}-${f.label}.pdf`;
18 + const headers: Record<string, string> = { "Content-Type": "application/pdf", "Content-Disposition": `${inline ? "inline" : "attachment"}; filename="${filename}"`, "Accept-Ranges": "bytes", "Cache-Control": "public, max-age=86400, immutable", ETag: `"${f.sha256}"`, "X-Content-Type-Options": "nosniff", "Content-Security-Policy": "sandbox", "X-Checksum-SHA256": f.sha256 };
19 + if (req.headers.get("if-none-match") === `"${f.sha256}"`) return new Response(null, { status: 304, headers });
20 + if (!head && !inline) recordEvent("download", f.paper_id, { ip: ip(req), ua: req.headers.get("user-agent"), referrer: req.headers.get("referer"), versionId: f.version_id }).catch(() => {});
21 + const range = req.headers.get("range"); let start = 0, end = st.size - 1, status = 200;
22 + if (range) { const m = /bytes=(\d*)-(\d*)/.exec(range); if (m) { if (m[1]) start = Number(m[1]); if (m[2]) end = Number(m[2]); if (!m[1] && m[2]) { start = st.size - Number(m[2]); end = st.size - 1; } if (start > end || end >= st.size) return new Response(null, { status: 416, headers: { "Content-Range": `bytes */${st.size}` } }); status = 206; headers["Content-Range"] = `bytes ${start}-${end}/${st.size}`; } }
23 + headers["Content-Length"] = String(end - start + 1);
24 + if (head) return new Response(null, { status, headers });
25 + return new Response(Readable.toWeb(createReadStream(path, { start, end })) as unknown as ReadableStream, { status, headers });
26 +}
added src/app/api/health/route.ts +6 −0
@@ -0,0 +1,6 @@
1 +import { sqlPublic } from "@/db";
2 +export const dynamic = "force-dynamic";
3 +export async function GET() {
4 + try { const [r] = await sqlPublic`SELECT count(*)::int AS n FROM public_papers WHERE status = 'published'`; return Response.json({ ok: true, papers: r.n, time: new Date().toISOString() }, { headers: { "Cache-Control": "no-store" } }); }
5 + catch (e) { return Response.json({ ok: false, error: (e as Error).message }, { status: 503, headers: { "Cache-Control": "no-store" } }); }
6 +}
added src/app/api/oai/route.ts +42 −0
@@ -0,0 +1,42 @@
1 +import { sqlPublic as sql } from "@/db";
2 +import { APP_URL, INSTITUTION, SITE_NAME } from "@/lib/env";
3 +import { limited, tooMany } from "@/lib/api";
4 +export const dynamic = "force-dynamic";
5 +const esc = (s: string) => s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
6 +const REPO = "wp.spboucher.ai"; const PAGE = 100;
7 +function envelope(verb: string, args: Record<string, string>, body: string) {
8 + return `<?xml version="1.0" encoding="UTF-8"?>\n<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>${new Date().toISOString().replace(/\.\d+Z$/, "Z")}</responseDate><request${Object.entries(args).map(([k, v]) => ` ${k}="${esc(v)}"`).join("")}>${APP_URL}/api/oai</request>${body}</OAI-PMH>`;
9 +}
10 +const err = (args: Record<string, string>, code: string, msg: string) => new Response(envelope("", args, `<error code="${code}">${esc(msg)}</error>`), { headers: { "Content-Type": "text/xml; charset=utf-8" } });
11 +function record(p: Record<string, unknown>, headerOnly = false) {
12 + const date = String(p.updated_at).slice(0, 10); const id = `oai:${REPO}:${p.slug}`; const sets = `<setSpec>${String(p.series_code).toLowerCase()}</setSpec>`;
13 + const header = `<header${p.status === "withdrawn" ? ' status="deleted"' : ""}><identifier>${id}</identifier><datestamp>${date}</datestamp>${sets}</header>`;
14 + if (headerOnly) return header;
15 + const authors = (p.authors as string[]) ?? []; const kw = (p.keywords as string[]) ?? []; const abstract = ((p.abstract as string[]) ?? []).join("\n\n");
16 + const dc = `<metadata><oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
17 +<dc:title>${esc(String(p.title))}${p.subtitle ? `: ${esc(String(p.subtitle))}` : ""}</dc:title>${authors.map((a) => `<dc:creator>${esc(a)}</dc:creator>`).join("")}${kw.map((k) => `<dc:subject>${esc(k)}</dc:subject>`).join("")}${abstract ? `<dc:description>${esc(abstract)}</dc:description>` : ""}<dc:publisher>${esc(INSTITUTION)}</dc:publisher>${p.first_released_on ? `<dc:date>${String(p.first_released_on).slice(0, p.first_released_precision === "year" ? 4 : p.first_released_precision === "month" ? 7 : 10)}</dc:date>` : ""}<dc:type>Working paper</dc:type><dc:format>application/pdf</dc:format><dc:identifier>${APP_URL}/papers/${p.slug}</dc:identifier><dc:identifier>${esc(String(p.identifier))}</dc:identifier>${p.ssrn_id ? `<dc:identifier>https://ssrn.com/abstract=${p.ssrn_id}</dc:identifier>` : ""}${p.doi ? `<dc:identifier>doi:${esc(String(p.doi))}</dc:identifier>` : ""}<dc:language>${esc(String(p.language))}</dc:language><dc:relation>${esc(String(p.series_name_en))}</dc:relation><dc:rights>Working paper circulated for discussion; all rights reserved by the authors.</dc:rights>
18 +</oai_dc:dc></metadata>`;
19 + return `<record>${header}${p.status === "withdrawn" ? "" : dc}</record>`;
20 +}
21 +const SELECT = sql`SELECT p.slug, p.identifier, p.title, p.subtitle, p.abstract, p.keywords, p.language, p.status, p.ssrn_id, p.doi, p.first_released_on::text, p.first_released_precision, p.updated_at::text, s.code AS series_code, s.name_en AS series_name_en, (SELECT array_agg(a.display_name ORDER BY pa.position) FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id) AS authors FROM public_papers p JOIN series s ON s.id = p.series_id`;
22 +export async function GET(req: Request) {
23 + if (limited(req, "oai", 60)) return tooMany();
24 + const u = new URL(req.url); const args: Record<string, string> = {}; for (const [k, v] of u.searchParams) args[k] = v;
25 + const verb = args.verb; const xml = (body: string) => new Response(envelope(verb, args, body), { headers: { "Content-Type": "text/xml; charset=utf-8", "Cache-Control": "public, max-age=300" } });
26 + if (!verb) return err(args, "badVerb", "verb missing");
27 + if (verb === "Identify") { const [e] = await sql`SELECT min(updated_at)::text AS earliest FROM public_papers`; return xml(`<Identify><repositoryName>${esc(SITE_NAME)} — Simon-Pierre Boucher Research Observatory</repositoryName><baseURL>${APP_URL}/api/oai</baseURL><protocolVersion>2.0</protocolVersion><adminEmail>simon-pierre.boucher@uqo.ca</adminEmail><earliestDatestamp>${String(e?.earliest ?? new Date().toISOString()).slice(0, 10)}</earliestDatestamp><deletedRecord>persistent</deletedRecord><granularity>YYYY-MM-DD</granularity><description><oai-identifier xmlns="http://www.openarchives.org/OAI/2.0/oai-identifier" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai-identifier http://www.openarchives.org/OAI/2.0/oai-identifier.xsd"><scheme>oai</scheme><repositoryIdentifier>${REPO}</repositoryIdentifier><delimiter>:</delimiter><sampleIdentifier>oai:${REPO}:wp38</sampleIdentifier></oai-identifier></description></Identify>`); }
28 + if (verb === "ListMetadataFormats") return xml(`<ListMetadataFormats><metadataFormat><metadataPrefix>oai_dc</metadataPrefix><schema>http://www.openarchives.org/OAI/2.0/oai_dc.xsd</schema><metadataNamespace>http://www.openarchives.org/OAI/2.0/oai_dc/</metadataNamespace></metadataFormat></ListMetadataFormats>`);
29 + if (verb === "ListSets") { const rows = await sql`SELECT code, name_en FROM series WHERE is_active ORDER BY sort`; return xml(`<ListSets>${rows.map((r) => `<set><setSpec>${String(r.code).toLowerCase()}</setSpec><setName>${esc(r.name_en)}</setName></set>`).join("")}</ListSets>`); }
30 + if (verb === "GetRecord") { if (args.metadataPrefix !== "oai_dc") return err(args, "cannotDisseminateFormat", "only oai_dc"); const slug = (args.identifier ?? "").replace(`oai:${REPO}:`, ""); const [p] = await sql`${SELECT} WHERE p.slug = ${slug}`; if (!p) return err(args, "idDoesNotExist", "unknown identifier"); return xml(`<GetRecord>${record(p)}</GetRecord>`); }
31 + if (verb === "ListRecords" || verb === "ListIdentifiers") {
32 + let from = args.from, until = args.until, set = args.set, offset = 0;
33 + if (args.resumptionToken) { try { const t = JSON.parse(Buffer.from(args.resumptionToken, "base64url").toString()); from = t.from; until = t.until; set = t.set; offset = t.offset; } catch { return err(args, "badResumptionToken", "invalid token"); } }
34 + else if (args.metadataPrefix !== "oai_dc") return err(args, "cannotDisseminateFormat", "only oai_dc");
35 + const rows = await sql`${SELECT} WHERE (${from ?? null}::text IS NULL OR p.updated_at >= ${from ?? null}::timestamptz) AND (${until ?? null}::text IS NULL OR p.updated_at < (${until ?? null}::date + 1)) AND (${set ?? null}::text IS NULL OR lower(s.code) = ${set ?? null}) ORDER BY p.updated_at, p.number LIMIT ${PAGE + 1} OFFSET ${offset}`;
36 + if (!rows.length) return err(args, "noRecordsMatch", "no records match");
37 + const page = rows.slice(0, PAGE); const more = rows.length > PAGE;
38 + const token = more ? `<resumptionToken>${Buffer.from(JSON.stringify({ from, until, set, offset: offset + PAGE })).toString("base64url")}</resumptionToken>` : "";
39 + return xml(`<${verb}>${page.map((p) => (verb === "ListRecords" ? record(p) : record(p, true))).join("")}${token}</${verb}>`);
40 + }
41 + return err(args, "badVerb", "unknown verb");
42 +}
added src/app/api/preview/[name]/route.ts +13 −0
@@ -0,0 +1,13 @@
1 +import { createReadStream, promises as fs } from "node:fs";
2 +import { Readable } from "node:stream";
3 +import { join } from "node:path";
4 +import { previewFor } from "@/lib/papers/queries";
5 +import { STORAGE_DIR } from "@/lib/env";
6 +export const dynamic = "force-dynamic";
7 +/** First-page preview of the current manuscript: /api/preview/<slug>.png */
8 +export async function GET(_req: Request, { params }: { params: Promise<{ name: string }> }) {
9 + const { name } = await params; const slug = name.replace(/\.png$/, ""); if (!/^[a-z0-9]+$/.test(slug)) return new Response("Not found", { status: 404 });
10 + const sha = await previewFor(slug); if (!sha) return new Response("Not found", { status: 404 });
11 + const path = join(STORAGE_DIR, "previews", `${sha}.png`); try { await fs.access(path); } catch { return new Response("Not found", { status: 404 }); }
12 + return new Response(Readable.toWeb(createReadStream(path)) as unknown as ReadableStream, { headers: { "Content-Type": "image/png", "Cache-Control": "public, max-age=604800, immutable", ETag: `"${sha}"` } });
13 +}
added src/app/api/v1/authors/route.ts +3 −0
@@ -0,0 +1,3 @@
1 +import { authorsWithCounts } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) { if (limited(req, "api")) return tooMany(); return json(await authorsWithCounts(), { maxAge: 120 }); }
added src/app/api/v1/datasets/route.ts +3 −0
@@ -0,0 +1,3 @@
1 +import { datasetsWithCounts } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) { if (limited(req, "api")) return tooMany(); return json(await datasetsWithCounts(), { maxAge: 120 }); }
added src/app/api/v1/graph/route.ts +3 −0
@@ -0,0 +1,3 @@
1 +import { graph } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) { if (limited(req, "api")) return tooMany(); const kinds = new URL(req.url).searchParams.get("kinds")?.split(",").filter(Boolean); return json(await graph(kinds?.length ? kinds : undefined), { maxAge: 120 }); }
added src/app/api/v1/methods/route.ts +3 −0
@@ -0,0 +1,3 @@
1 +import { methodsWithCounts } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) { if (limited(req, "api")) return tooMany(); return json(await methodsWithCounts(), { maxAge: 120 }); }
added src/app/api/v1/openapi.json/route.ts +2 −0
@@ -0,0 +1,2 @@
1 +import { openapi } from "@/lib/papers/openapi";
2 +export function GET() { return Response.json(openapi(), { headers: { "Cache-Control": "public, max-age=3600" } }); }
added src/app/api/v1/papers/[id]/citation.[format]/route.ts +13 −0
@@ -0,0 +1,13 @@
1 +import { getPaper } from "@/lib/papers/queries";
2 +import { bibtex, citationApa, citationChicago, ris } from "@/lib/papers/citation";
3 +import { citable } from "@/lib/papers/meta";
4 +import { limited, tooMany } from "@/lib/api";
5 +export async function GET(req: Request, { params }: { params: Promise<{ id: string; format: string }> }) {
6 + if (limited(req, "api")) return tooMany();
7 + const { id, format } = await params; const p = await getPaper(id); if (!p) return Response.json({ error: "Not found" }, { status: 404 });
8 + const c = citable(p); const h = { "Cache-Control": "public, max-age=300" };
9 + if (format === "bib") return new Response(bibtex(c), { headers: { ...h, "Content-Type": "application/x-bibtex; charset=utf-8", "Content-Disposition": `attachment; filename="${p.slug}.bib"` } });
10 + if (format === "ris") return new Response(ris(c), { headers: { ...h, "Content-Type": "application/x-research-info-systems; charset=utf-8", "Content-Disposition": `attachment; filename="${p.slug}.ris"` } });
11 + if (format === "json") return Response.json({ apa: citationApa(c), chicago: citationChicago(c), bibtex: bibtex(c), ris: ris(c) }, { headers: h });
12 + return Response.json({ error: "Unknown format" }, { status: 400 });
13 +}
added src/app/api/v1/papers/[id]/route.ts +9 −0
@@ -0,0 +1,9 @@
1 +import { getPaper } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +import { APP_URL } from "@/lib/env";
4 +export async function GET(req: Request, { params }: { params: Promise<{ id: string }> }) {
5 + if (limited(req, "api")) return tooMany();
6 + const { id } = await params; const p = await getPaper(id); if (!p) return Response.json({ error: "Not found" }, { status: 404 });
7 + const { embedding_model, ...rest } = p; void embedding_model;
8 + return json({ ...rest, url: `${APP_URL}/papers/${p.slug}`, pdf_url: p.has_pdf && p.file_id ? `${APP_URL}/api/files/${p.file_id}/${p.slug}-${p.version_label ?? "current"}.pdf` : null, versions: p.versions.map((v) => ({ ...v, pdf_url: v.file_id && !v.withdrawn ? `${APP_URL}/api/files/${v.file_id}/${p.slug}-${v.label}.pdf` : null })) });
9 +}
added src/app/api/v1/papers/route.ts +10 −0
@@ -0,0 +1,10 @@
1 +import { listPapers } from "@/lib/papers/queries";
2 +import { parseListParams } from "@/lib/papers/params";
3 +import { json, limited, tooMany } from "@/lib/api";
4 +import { APP_URL } from "@/lib/env";
5 +export async function GET(req: Request) {
6 + if (limited(req, "api")) return tooMany();
7 + const sp: Record<string, string | string[]> = {}; for (const [k, v] of new URL(req.url).searchParams) sp[k] = sp[k] ? ([] as string[]).concat(sp[k], v) : v;
8 + const res = await listPapers(parseListParams(sp));
9 + return json({ ...res, items: res.items.map((p) => ({ ...p, url: `${APP_URL}/papers/${p.slug}`, pdf_url: p.has_pdf && p.file_id ? `${APP_URL}/api/files/${p.file_id}/${p.slug}-${p.version_label ?? "current"}.pdf` : null })) });
10 +}
added src/app/api/v1/search/route.ts +9 −0
@@ -0,0 +1,9 @@
1 +import { logSearch, search, type SearchMode } from "@/lib/papers/search";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) {
4 + if (limited(req, "search", 60)) return tooMany();
5 + const u = new URL(req.url); const q = (u.searchParams.get("q") ?? "").trim(); if (!q) return Response.json({ error: "q required" }, { status: 400 });
6 + const mode = (["hybrid", "lexical", "semantic"].includes(u.searchParams.get("mode") ?? "") ? u.searchParams.get("mode") : "hybrid") as SearchMode;
7 + const res = await search(q, mode, Math.min(50, Number(u.searchParams.get("limit") ?? 20) || 20)); await logSearch(q, res.mode, res.hits.length, "api", res.tookMs);
8 + return json(res, { maxAge: 30 });
9 +}
added src/app/api/v1/series/route.ts +3 −0
@@ -0,0 +1,3 @@
1 +import { seriesWithCounts } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) { if (limited(req, "api")) return tooMany(); return json(await seriesWithCounts(), { maxAge: 120 }); }
added src/app/api/v1/stats/route.ts +3 −0
@@ -0,0 +1,3 @@
1 +import { stats } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) { if (limited(req, "api")) return tooMany(); return json(await stats(), { maxAge: 120 }); }
added src/app/api/v1/timeline/route.ts +3 −0
@@ -0,0 +1,3 @@
1 +import { timeline } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) { if (limited(req, "api")) return tooMany(); return json(await timeline(), { maxAge: 120 }); }
added src/app/api/v1/topics/route.ts +3 −0
@@ -0,0 +1,3 @@
1 +import { topicsWithCounts } from "@/lib/papers/queries";
2 +import { json, limited, tooMany } from "@/lib/api";
3 +export async function GET(req: Request) { if (limited(req, "api")) return tooMany(); return json(await topicsWithCounts(), { maxAge: 120 }); }
added src/app/feed.xml/route.ts +13 −0
@@ -0,0 +1,13 @@
1 +import { listPapers } from "@/lib/papers/queries";
2 +import { APP_URL, SITE_NAME, SITE_SECONDARY } from "@/lib/env";
3 +export const revalidate = 600;
4 +const esc = (s: string) => s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;").replace(/"/g, "&quot;");
5 +export async function GET() {
6 + const { items } = await listPapers({ sort: "revised", perPage: 30 });
7 + const xml = `<?xml version="1.0" encoding="UTF-8"?>
8 +<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
9 +<channel><title>${esc(SITE_NAME)} — ${esc(SITE_SECONDARY)}</title><link>${APP_URL}</link><description>Latest research working papers and revisions.</description><language>en-ca</language><atom:link href="${APP_URL}/feed.xml" rel="self" type="application/rss+xml"/>
10 +${items.map((p) => `<item><title>${esc(`${p.identifier} · ${p.title}`)}</title><link>${APP_URL}/papers/${p.slug}</link><guid isPermaLink="true">${APP_URL}/papers/${p.slug}</guid>${p.last_revised_on ? `<pubDate>${new Date(p.last_revised_on + "T12:00:00Z").toUTCString()}</pubDate>` : ""}${p.authors.map((a) => `<dc:creator>${esc(a.display_name)}</dc:creator>`).join("")}<description>${esc((p.abstract[0] ?? "").slice(0, 600))}</description></item>`).join("\n")}
11 +</channel></rss>`;
12 + return new Response(xml, { headers: { "Content-Type": "application/rss+xml; charset=utf-8", "Cache-Control": "public, max-age=600" } });
13 +}
added src/app/globals.css +64 −0
@@ -0,0 +1,64 @@
1 +@import "tailwindcss";
2 +
3 +@custom-variant dark (&:where(.dark, .dark *));
4 +
5 +@theme {
6 + --font-sans: ui-sans-serif, -apple-system, BlinkMacSystemFont, "SF Pro Text", "Segoe UI", Inter, Roboto, "Helvetica Neue", Arial, sans-serif;
7 + --font-serif: "Iowan Old Style", "Palatino Linotype", "Book Antiqua", Palatino, "Source Serif 4", Georgia, "Times New Roman", serif;
8 + --font-mono: ui-monospace, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace;
9 + --color-bg: var(--bg); --color-bg-2: var(--bg-2); --color-bg-3: var(--bg-3);
10 + --color-fg: var(--fg); --color-fg-2: var(--fg-2); --color-fg-3: var(--fg-3);
11 + --color-line: var(--line); --color-line-2: var(--line-2);
12 + --color-navy: var(--navy); --color-navy-2: var(--navy-2); --color-burgundy: var(--burgundy); --color-burgundy-2: var(--burgundy-2);
13 + --color-accent-fg: var(--accent-fg); --color-ok: var(--ok); --color-warn: var(--warn); --color-bad: var(--bad);
14 + --shadow-soft: 0 1px 2px rgb(20 18 14 / 0.04), 0 6px 20px -12px rgb(20 18 14 / 0.14);
15 + --shadow-lift: 0 2px 6px rgb(20 18 14 / 0.06), 0 18px 36px -20px rgb(20 18 14 / 0.25);
16 +}
17 +:root {
18 + color-scheme: light;
19 + --bg: #faf8f4; --bg-2: #ffffff; --bg-3: #f2eee7;
20 + --fg: #1c1b1f; --fg-2: #444349; --fg-3: #6a6873;
21 + --line: #e4dfd6; --line-2: #cfc8bc;
22 + --navy: #1f3a5f; --navy-2: #2d4f7c; --burgundy: #6b2737; --burgundy-2: #87384a; --accent-fg: #ffffff;
23 + --ok: #2f6b3f; --warn: #8a5a13; --bad: #9b2c2c;
24 +}
25 +.dark {
26 + color-scheme: dark;
27 + --bg: #15161a; --bg-2: #1d1f25; --bg-3: #24262e;
28 + --fg: #ecebe6; --fg-2: #c3c1ba; --fg-3: #8f8d87;
29 + --line: #2f323b; --line-2: #3f434e;
30 + --navy: #9fbce0; --navy-2: #b9cfe9; --burgundy: #d8919f; --burgundy-2: #e4aab5; --accent-fg: #15161a;
31 + --ok: #7fc08f; --warn: #d9b25b; --bad: #e08a8a;
32 +}
33 +html { scroll-behavior: smooth; }
34 +body { background: var(--bg); color: var(--fg); font-family: var(--font-sans); -webkit-font-smoothing: antialiased; text-rendering: optimizeLegibility; }
35 +h1, h2, h3, .serif { font-family: var(--font-serif); letter-spacing: -0.01em; }
36 +h1 { font-weight: 500; } h2, h3 { font-weight: 500; }
37 +a { text-decoration-thickness: 1px; text-underline-offset: 3px; }
38 +.skip-link { position: absolute; left: -999px; top: 0; background: var(--navy); color: var(--accent-fg); padding: .5rem 1rem; z-index: 100; }
39 +.skip-link:focus { left: 1rem; top: 1rem; }
40 +.container { width: 100%; max-width: 76rem; margin-inline: auto; padding-inline: 1rem; }
41 +@media (min-width: 640px) { .container { padding-inline: 1.5rem; } }
42 +.prose-academic p { margin-bottom: 1rem; line-height: 1.72; font-size: 1.02rem; }
43 +.prose-academic p:last-child { margin-bottom: 0; }
44 +mark { background: color-mix(in oklab, var(--navy) 18%, transparent); color: inherit; padding: 0 .1em; border-radius: 2px; }
45 +.card { background: var(--bg-2); border: 1px solid var(--line); border-radius: .75rem; box-shadow: var(--shadow-soft); }
46 +.chip { display: inline-flex; align-items: center; gap: .3rem; border: 1px solid var(--line); border-radius: 999px; padding: .15rem .6rem; font-size: .78rem; line-height: 1.3; color: var(--fg-2); background: var(--bg-2); white-space: nowrap; }
47 +.chip-navy { border-color: color-mix(in oklab, var(--navy) 40%, transparent); color: var(--navy); background: color-mix(in oklab, var(--navy) 7%, transparent); }
48 +.chip-burgundy { border-color: color-mix(in oklab, var(--burgundy) 40%, transparent); color: var(--burgundy); background: color-mix(in oklab, var(--burgundy) 7%, transparent); }
49 +.btn { display: inline-flex; align-items: center; justify-content: center; gap: .45rem; border-radius: .5rem; padding: .55rem 1rem; font-size: .9rem; font-weight: 500; border: 1px solid var(--line-2); background: var(--bg-2); color: var(--fg); transition: background .15s, border-color .15s; }
50 +.btn:hover { background: var(--bg-3); } .btn:focus-visible { outline: 2px solid var(--navy); outline-offset: 2px; }
51 +.btn-primary { background: var(--navy); border-color: var(--navy); color: var(--accent-fg); } .btn-primary:hover { background: var(--navy-2); }
52 +.btn-sm { padding: .35rem .7rem; font-size: .82rem; }
53 +.input { width: 100%; border: 1px solid var(--line-2); border-radius: .5rem; background: var(--bg-2); color: var(--fg); padding: .6rem .85rem; font-size: .95rem; }
54 +.input:focus { outline: 2px solid var(--navy); outline-offset: 1px; border-color: var(--navy); }
55 +select.input { appearance: auto; }
56 +.label { font-size: .72rem; text-transform: uppercase; letter-spacing: .08em; color: var(--fg-3); font-weight: 600; }
57 +.meta { font-size: .85rem; color: var(--fg-3); }
58 +.kicker { font-size: .78rem; text-transform: uppercase; letter-spacing: .12em; color: var(--burgundy); font-weight: 600; }
59 +table.data { width: 100%; border-collapse: collapse; font-size: .9rem; }
60 +table.data th { text-align: left; font-weight: 600; color: var(--fg-3); font-size: .75rem; text-transform: uppercase; letter-spacing: .06em; padding: .5rem .6rem; border-bottom: 1px solid var(--line-2); }
61 +table.data td { padding: .55rem .6rem; border-bottom: 1px solid var(--line); vertical-align: top; }
62 +.hairline { border-top: 1px solid var(--line); }
63 +details > summary { cursor: pointer; list-style: none; } details > summary::-webkit-details-marker { display: none; }
64 +@media (prefers-reduced-motion: reduce) { * { animation: none !important; transition: none !important; } html { scroll-behavior: auto; } }
added src/app/layout.tsx +25 −0
@@ -0,0 +1,25 @@
1 +import type { Metadata, Viewport } from "next";
2 +import "./globals.css";
3 +import { APP_URL, SITE_NAME, SITE_SECONDARY } from "@/lib/env";
4 +import { getLocale } from "@/lib/i18n/server";
5 +
6 +export const metadata: Metadata = {
7 + metadataBase: new URL(APP_URL),
8 + title: { default: `${SITE_NAME} — ${SITE_SECONDARY}`, template: `%s · ${SITE_NAME}` },
9 + description: "An open collection of research working papers in finance, economics, artificial intelligence, real estate, and quantitative science by Simon-Pierre Boucher (Université du Québec en Outaouais).",
10 + applicationName: SITE_NAME,
11 + openGraph: { type: "website", siteName: SITE_NAME, locale: "en_CA" },
12 + robots: { index: true, follow: true },
13 +};
14 +export const viewport: Viewport = { themeColor: [{ media: "(prefers-color-scheme: light)", color: "#faf8f4" }, { media: "(prefers-color-scheme: dark)", color: "#15161a" }], width: "device-width", initialScale: 1 };
15 +const themeScript = `(function(){try{var m=localStorage.getItem('theme');var d=m==='dark'||((!m||m==='system')&&matchMedia('(prefers-color-scheme: dark)').matches);if(d)document.documentElement.classList.add('dark');}catch(e){}})();`;
16 +
17 +export default async function RootLayout({ children }: { children: React.ReactNode }) {
18 + const locale = await getLocale();
19 + return (
20 + <html lang={locale} suppressHydrationWarning>
21 + <head><script dangerouslySetInnerHTML={{ __html: themeScript }} /></head>
22 + <body className="min-h-dvh flex flex-col">{children}</body>
23 + </html>
24 + );
25 +}
added src/app/not-found.tsx +7 −0
@@ -0,0 +1,7 @@
1 +import Link from "next/link";
2 +import { getLocale } from "@/lib/i18n/server";
3 +import { href, t } from "@/lib/i18n";
4 +export default async function NotFound() {
5 + const locale = await getLocale(); const d = t(locale);
6 + return (<main className="container py-24 max-w-xl text-center"><p className="kicker">404</p><h1 className="text-3xl mt-2">{d.misc.notFound}</h1><p className="text-fg-2 mt-3">{d.misc.notFoundLead}</p><p className="mt-6 flex justify-center gap-2"><Link href={href(locale, "/")} className="btn no-underline">{d.misc.home}</Link><Link href={href(locale, "/papers")} className="btn btn-primary no-underline">{d.nav.papers}</Link></p></main>);
7 +}
added src/app/robots.ts +5 −0
@@ -0,0 +1,5 @@
1 +import type { MetadataRoute } from "next";
2 +import { APP_URL } from "@/lib/env";
3 +export default function robots(): MetadataRoute.Robots {
4 + return { rules: [{ userAgent: "*", allow: "/", disallow: ["/admin", "/login", "/api/admin", "/api/events"] }], sitemap: `${APP_URL}/sitemap.xml`, host: APP_URL };
5 +}
added src/app/sitemap.ts +14 −0
@@ -0,0 +1,14 @@
1 +import type { MetadataRoute } from "next";
2 +import { sqlPublic as sql } from "@/db";
3 +import { APP_URL } from "@/lib/env";
4 +export const revalidate = 3600;
5 +export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
6 + const papers = await sql`SELECT slug, updated_at FROM public_papers WHERE status = 'published' ORDER BY number`;
7 + const topics = await sql`SELECT slug FROM topics WHERE is_active`; const methods = await sql`SELECT slug FROM methods`; const datasets = await sql`SELECT slug FROM datasets`; const authors = await sql`SELECT slug FROM authors`;
8 + const statics = ["", "/papers", "/series", "/topics", "/methods", "/datasets", "/authors", "/research-map", "/timeline", "/search", "/about", "/citation", "/api/docs"];
9 + const out: MetadataRoute.Sitemap = [];
10 + for (const s of statics) out.push({ url: `${APP_URL}${s}`, changeFrequency: "weekly", priority: s === "" ? 1 : 0.6, alternates: { languages: { en: `${APP_URL}${s}`, fr: `${APP_URL}/fr${s}` } } });
11 + for (const p of papers) out.push({ url: `${APP_URL}/papers/${p.slug}`, lastModified: new Date(p.updated_at), changeFrequency: "monthly", priority: 0.9, alternates: { languages: { en: `${APP_URL}/papers/${p.slug}`, fr: `${APP_URL}/fr/papers/${p.slug}` } } });
12 + for (const [prefix, rows] of [["/topics", topics], ["/methods", methods], ["/datasets", datasets], ["/authors", authors]] as const) for (const r of rows) out.push({ url: `${APP_URL}${prefix}/${r.slug}`, changeFrequency: "monthly", priority: 0.5 });
13 + return out;
14 +}
added src/components/admin-ui.tsx +20 −0
@@ -0,0 +1,20 @@
1 +import Link from "next/link";
2 +export function Field({ label, name, value, type = "text", hint, required, rows, className = "" }: { label: string; name: string; value?: string | number | null; type?: string; hint?: string; required?: boolean; rows?: number; className?: string }) {
3 + return (<label className={`block text-sm ${className}`}><span className="label">{label}{required ? " *" : ""}</span>
4 + {rows ? <textarea name={name} defaultValue={value ?? ""} rows={rows} className="input mt-1 font-mono text-xs" /> : <input type={type} name={name} defaultValue={value ?? ""} required={required} className="input mt-1" />}
5 + {hint && <span className="block text-xs text-fg-3 mt-1">{hint}</span>}</label>);
6 +}
7 +export function Select({ label, name, value, options, hint }: { label: string; name: string; value?: string | number | null; options: { v: string | number; label: string }[]; hint?: string }) {
8 + return (<label className="block text-sm"><span className="label">{label}</span><select name={name} defaultValue={value ?? ""} className="input mt-1">{options.map((o) => <option key={o.v} value={o.v}>{o.label}</option>)}</select>{hint && <span className="block text-xs text-fg-3 mt-1">{hint}</span>}</label>);
9 +}
10 +export function Check({ label, name, checked }: { label: string; name: string; checked?: boolean }) { return <label className="flex items-center gap-2 text-sm"><input type="checkbox" name={name} defaultChecked={checked} className="accent-[var(--navy)]" />{label}</label>; }
11 +export function Csrf({ token }: { token: string }) { return <input type="hidden" name="_csrf" value={token} />; }
12 +export function Notice({ sp }: { sp: Record<string, string | string[] | undefined> }) {
13 + if (sp.saved) return <p className="card p-3 mb-4 text-sm text-ok border-ok">Saved.</p>;
14 + if (sp.queued) return <p className="card p-3 mb-4 text-sm text-ok border-ok">Queued — the worker will process it shortly.</p>;
15 + if (sp.deleted) return <p className="card p-3 mb-4 text-sm text-ok border-ok">Deleted.</p>;
16 + if (sp.error) return <p className="card p-3 mb-4 text-sm text-bad border-bad">{String(sp.error)}</p>;
17 + return null;
18 +}
19 +export function PageTitle({ title, children }: { title: string; children?: React.ReactNode }) { return <div className="flex flex-wrap items-center gap-3 mb-6"><h1 className="text-2xl sm:text-3xl">{title}</h1><div className="ml-auto flex gap-2">{children}</div></div>; }
20 +export function Tabs({ items, current }: { items: [string, string][]; current: string }) { return <nav className="flex flex-wrap gap-1 border-b border-line mb-6 text-sm">{items.map(([h, l]) => <Link key={h} href={h} className={`px-3 py-2 -mb-px border-b-2 no-underline ${current === h ? "border-navy text-navy" : "border-transparent text-fg-2 hover:text-fg"}`}>{l}</Link>)}</nav>; }
added src/components/cite-menu.tsx +25 −0
@@ -0,0 +1,25 @@
1 +"use client";
2 +import { useState } from "react";
3 +import { Check, Copy, Download, Link2, Quote } from "lucide-react";
4 +import { t, type Locale } from "@/lib/i18n";
5 +export function CiteMenu({ locale, slug, apa, chicago, shareUrl }: { locale: Locale; slug: string; apa: string; chicago: string; shareUrl: string }) {
6 + const d = t(locale); const [copied, setCopied] = useState<string | null>(null); const [open, setOpen] = useState(false);
7 + const copy = async (key: string, text: string) => { try { await navigator.clipboard.writeText(text); setCopied(key); setTimeout(() => setCopied(null), 1800); } catch { /* clipboard unavailable */ } };
8 + return (
9 + <div className="relative">
10 + <button type="button" className="btn" aria-expanded={open} aria-haspopup="dialog" onClick={() => setOpen((o) => !o)}><Quote size={16} aria-hidden /> {d.paper.cite}</button>
11 + {open && (
12 + <div role="dialog" aria-label={d.paper.cite} className="absolute z-30 mt-2 w-[min(92vw,34rem)] card p-4 text-sm left-0 sm:left-auto">
13 + <p className="label mb-1">APA</p><p className="text-fg-2 leading-relaxed">{apa}</p>
14 + <p className="label mt-3 mb-1">Chicago</p><p className="text-fg-2 leading-relaxed">{chicago}</p>
15 + <div className="mt-3 flex flex-wrap gap-2">
16 + <button type="button" className="btn btn-sm" onClick={() => copy("apa", apa)}>{copied === "apa" ? <Check size={14} aria-hidden /> : <Copy size={14} aria-hidden />} {d.paper.copyCitation}</button>
17 + <a className="btn btn-sm no-underline" href={`/api/v1/papers/${slug}/citation.bib`} download><Download size={14} aria-hidden /> {d.paper.bibtex}</a>
18 + <a className="btn btn-sm no-underline" href={`/api/v1/papers/${slug}/citation.ris`} download><Download size={14} aria-hidden /> {d.paper.ris}</a>
19 + <button type="button" className="btn btn-sm" onClick={() => copy("url", shareUrl)}>{copied === "url" ? <Check size={14} aria-hidden /> : <Link2 size={14} aria-hidden />} {copied === "url" ? d.paper.copied : d.paper.share}</button>
20 + </div>
21 + </div>
22 + )}
23 + </div>
24 + );
25 +}
added src/components/constellation.tsx +60 −0
@@ -0,0 +1,60 @@
1 +"use client";
2 +import { useEffect, useMemo, useRef, useState } from "react";
3 +import { useRouter } from "next/navigation";
4 +import { forceCenter, forceCollide, forceLink, forceManyBody, forceSimulation, type SimulationLinkDatum, type SimulationNodeDatum } from "d3-force";
5 +import { href, t, type Locale } from "@/lib/i18n";
6 +
7 +export type GNode = { slug: string; identifier: string; title: string; year: number | null; domain: string | null; accent: string | null; views: number };
8 +export type GEdge = { source: string; target: string; kind: string; weight: number; inferred: boolean; explanation: string | null; explanation_fr: string | null };
9 +type N = GNode & SimulationNodeDatum; type L = SimulationLinkDatum<N> & { kind: string; weight: number; inferred: boolean; explanation: string | null; explanation_fr: string | null };
10 +const KIND_COLOR: Record<string, string> = { topic: "#1f3a5f", method: "#2d4a3e", dataset: "#6b4e16", author: "#5c4b6b", semantic: "#6b2737", lexical: "#8a6d7a", curated: "#9b2c2c" };
11 +/** Deterministic pseudo-random for stable initial positions (same layout on every load). */
12 +function seeded(s: string) { let h = 2166136261; for (let i = 0; i < s.length; i++) { h ^= s.charCodeAt(i); h = Math.imul(h, 16777619) >>> 0; } return (h % 10000) / 10000; }
13 +
14 +export function Constellation({ locale, nodes, edges, height = 520, compact = false, kinds: initialKinds }: { locale: Locale; nodes: GNode[]; edges: GEdge[]; height?: number; compact?: boolean; kinds?: string[] }) {
15 + const d = t(locale); const router = useRouter(); const ref = useRef<SVGSVGElement>(null);
16 + const allKinds = useMemo(() => [...new Set(edges.map((e) => e.kind))], [edges]);
17 + const [kinds, setKinds] = useState<Set<string>>(new Set(initialKinds ?? allKinds));
18 + const [pos, setPos] = useState<{ nodes: N[]; links: L[] } | null>(null);
19 + const [hover, setHover] = useState<string | null>(null); const [sel, setSel] = useState<L | null>(null);
20 + const [view, setView] = useState({ x: 0, y: 0, k: 1 }); const drag = useRef<{ x: number; y: number; vx: number; vy: number } | null>(null);
21 + const W = 960, H = height;
22 + useEffect(() => {
23 + const ns: N[] = nodes.map((n) => ({ ...n, x: W / 2 + (seeded(n.slug) - 0.5) * W * 0.8, y: H / 2 + (seeded(n.slug + "y") - 0.5) * H * 0.8 }));
24 + const idx = new Map(ns.map((n) => [n.slug, n]));
25 + const ls: L[] = edges.filter((e) => kinds.has(e.kind) && idx.has(e.source) && idx.has(e.target)).map((e) => ({ source: idx.get(e.source)!, target: idx.get(e.target)!, kind: e.kind, weight: e.weight, inferred: e.inferred, explanation: e.explanation, explanation_fr: e.explanation_fr }));
26 + const sim = forceSimulation(ns).force("link", forceLink<N, L>(ls).id((n) => n.slug).distance((l) => 70 + (1 - Math.min(1, l.weight)) * 90).strength((l) => 0.2 + Math.min(1, l.weight) * 0.5))
27 + .force("charge", forceManyBody().strength(-220)).force("center", forceCenter(W / 2, H / 2)).force("collide", forceCollide(22)).stop();
28 + for (let i = 0; i < 300; i++) sim.tick();
29 + setPos({ nodes: ns, links: ls });
30 + }, [nodes, edges, kinds, H]);
31 + const neighbours = useMemo(() => { const m = new Map<string, Set<string>>(); pos?.links.forEach((l) => { const s = (l.source as N).slug, tt = (l.target as N).slug; (m.get(s) ?? m.set(s, new Set()).get(s)!).add(tt); (m.get(tt) ?? m.set(tt, new Set()).get(tt)!).add(s); }); return m; }, [pos]);
32 + const onWheel = (e: React.WheelEvent) => { e.preventDefault(); setView((v) => ({ ...v, k: Math.min(4, Math.max(0.4, v.k * (e.deltaY < 0 ? 1.1 : 0.9))) })); };
33 + const onDown = (e: React.PointerEvent) => { drag.current = { x: e.clientX, y: e.clientY, vx: view.x, vy: view.y }; };
34 + const onMove = (e: React.PointerEvent) => { if (!drag.current) return; setView((v) => ({ ...v, x: drag.current!.vx + (e.clientX - drag.current!.x), y: drag.current!.vy + (e.clientY - drag.current!.y) })); };
35 + const onUp = () => { drag.current = null; };
36 + return (
37 + <div className="card overflow-hidden">
38 + {!compact && (
39 + <div className="flex flex-wrap items-center gap-2 p-3 border-b border-line text-xs">
40 + <span className="label mr-1">{d.map.filter}</span>
41 + {allKinds.map((k) => <label key={k} className="chip cursor-pointer select-none"><input type="checkbox" className="mr-1 accent-[var(--navy)]" checked={kinds.has(k)} onChange={(e) => setKinds((s) => { const n = new Set(s); if (e.target.checked) n.add(k); else n.delete(k); return n; })} /><span className="inline-block w-2.5 h-2.5 rounded-full mr-1" style={{ background: KIND_COLOR[k] ?? "#888" }} />{(d.map as Record<string, string>)[k] ?? k}</label>)}
42 + <span className="ml-auto text-fg-3">{nodes.length} {d.map.nodes} · {pos?.links.length ?? 0} {d.map.edges}</span>
43 + <button type="button" className="btn btn-sm" onClick={() => setView({ x: 0, y: 0, k: 1 })}>{d.map.reset}</button>
44 + </div>
45 + )}
46 + <svg ref={ref} viewBox={`0 0 ${W} ${H}`} role="img" aria-label={d.map.title} className="w-full touch-none select-none bg-bg-2" style={{ height: compact ? 420 : "min(70vh, 640px)" }} onWheel={onWheel} onPointerDown={onDown} onPointerMove={onMove} onPointerUp={onUp} onPointerLeave={onUp}>
47 + <g transform={`translate(${view.x},${view.y}) scale(${view.k})`}>
48 + {pos?.links.map((l, i) => { const s = l.source as N, tt = l.target as N; const dim = hover && hover !== s.slug && hover !== tt.slug; return <line key={i} x1={s.x} y1={s.y} x2={tt.x} y2={tt.y} stroke={KIND_COLOR[l.kind] ?? "#888"} strokeOpacity={dim ? 0.08 : 0.35 + Math.min(1, l.weight) * 0.4} strokeWidth={0.8 + Math.min(1, l.weight) * 1.6} strokeDasharray={l.inferred ? "4 3" : undefined} onClick={() => setSel(l)} className="cursor-pointer" />; })}
49 + {pos?.nodes.map((n) => { const dim = hover && hover !== n.slug && !neighbours.get(hover)?.has(n.slug); const r = 7 + Math.min(6, Math.log1p(n.views)); return (
50 + <g key={n.slug} transform={`translate(${n.x},${n.y})`} opacity={dim ? 0.25 : 1} className="cursor-pointer" onMouseEnter={() => setHover(n.slug)} onMouseLeave={() => setHover(null)} onClick={() => router.push(href(locale, `/papers/${n.slug}`))} tabIndex={0} role="link" aria-label={`${n.identifier}: ${n.title}`} onKeyDown={(e) => { if (e.key === "Enter") router.push(href(locale, `/papers/${n.slug}`)); }}>
51 + <circle r={r} fill={n.accent ?? "#1f3a5f"} stroke="var(--bg-2)" strokeWidth={1.5} />
52 + <text y={r + 11} textAnchor="middle" fontSize={10} fill="var(--fg-2)" fontFamily="var(--font-mono)">{n.identifier}</text>
53 + {hover === n.slug && <text y={-r - 6} textAnchor="middle" fontSize={11} fill="var(--fg)" style={{ paintOrder: "stroke", stroke: "var(--bg-2)", strokeWidth: 3 }}>{n.title.length > 70 ? n.title.slice(0, 68) + "…" : n.title}</text>}
54 + </g>); })}
55 + </g>
56 + </svg>
57 + {sel && <div className="p-3 border-t border-line text-sm flex gap-3 items-start"><span className="inline-block mt-1 w-2.5 h-2.5 rounded-full shrink-0" style={{ background: KIND_COLOR[sel.kind] }} /><span className="text-fg-2">{(sel.source as N).identifier} ↔ {(sel.target as N).identifier} — {locale === "fr" ? sel.explanation_fr ?? sel.explanation : sel.explanation}{sel.inferred ? ` (${d.paper.inferred})` : ""}</span><button type="button" className="ml-auto btn btn-sm" onClick={() => setSel(null)}>×</button></div>}
58 + </div>
59 + );
60 +}
added src/components/display-toggle.tsx +19 −0
@@ -0,0 +1,19 @@
1 +"use client";
2 +import { useRouter, useSearchParams } from "next/navigation";
3 +import { useEffect } from "react";
4 +import { LayoutGrid, List, Rows3, Table2 } from "lucide-react";
5 +import { t, type Locale } from "@/lib/i18n";
6 +const MODES = ["list", "compact", "cards", "table"] as const;
7 +export type DisplayMode = (typeof MODES)[number];
8 +/** Display preference is remembered in localStorage and applied through the `view` query parameter. */
9 +export function DisplayToggle({ locale, current }: { locale: Locale; current: DisplayMode }) {
10 + const d = t(locale); const router = useRouter(); const sp = useSearchParams();
11 + useEffect(() => { try { const saved = localStorage.getItem("papers.view") as DisplayMode | null; if (saved && MODES.includes(saved) && !sp.get("view") && saved !== "list") { const q = new URLSearchParams(sp.toString()); q.set("view", saved); router.replace(`?${q.toString()}`); } } catch {} }, [router, sp]);
12 + const set = (m: DisplayMode) => { try { localStorage.setItem("papers.view", m); } catch {} const q = new URLSearchParams(sp.toString()); q.set("view", m); router.push(`?${q.toString()}`); };
13 + const icons = { list: List, compact: Rows3, cards: LayoutGrid, table: Table2 }; const labels = { list: d.papers.list, compact: d.papers.compact, cards: d.papers.cards, table: d.papers.table };
14 + return (
15 + <div role="group" aria-label={d.papers.display} className="inline-flex rounded-lg border border-line-2 overflow-hidden">
16 + {MODES.map((m) => { const I = icons[m]; return <button key={m} type="button" aria-pressed={current === m} title={labels[m]} aria-label={labels[m]} onClick={() => set(m)} className={`px-2.5 py-1.5 text-sm ${current === m ? "bg-navy text-accent-fg" : "bg-bg-2 text-fg-2 hover:bg-bg-3"}`}><I size={16} aria-hidden /></button>; })}
17 + </div>
18 + );
19 +}
added src/components/pagination.tsx +12 −0
@@ -0,0 +1,12 @@
1 +import Link from "next/link";
2 +import { t, type Locale } from "@/lib/i18n";
3 +export function Pagination({ locale, page, total, perPage, base, params }: { locale: Locale; page: number; total: number; perPage: number; base: string; params: URLSearchParams }) {
4 + const d = t(locale); const pages = Math.max(1, Math.ceil(total / perPage)); if (pages <= 1) return null;
5 + const link = (p: number) => { const q = new URLSearchParams(params.toString()); q.set("page", String(p)); return `${base}?${q.toString()}`; };
6 + return (
7 + <nav aria-label="Pagination" className="flex items-center justify-between gap-3 mt-6 text-sm">
8 + <span className="text-fg-3">{d.misc.page} {page} {d.misc.of} {pages}</span>
9 + <span className="flex gap-2">{page > 1 && <Link href={link(page - 1)} className="btn btn-sm no-underline">← {d.misc.previous}</Link>}{page < pages && <Link href={link(page + 1)} className="btn btn-sm no-underline">{d.misc.next} →</Link>}</span>
10 + </nav>
11 + );
12 +}
added src/components/paper-items.tsx +73 −0
@@ -0,0 +1,73 @@
1 +import Link from "next/link";
2 +import { FileText } from "lucide-react";
3 +import { fmtDate, fmtInt, href, t, type Locale } from "@/lib/i18n";
4 +import type { PaperSummary } from "@/lib/papers/types";
5 +
6 +export function AuthorsLine({ authors, locale, link = true }: { authors: PaperSummary["authors"]; locale: Locale; link?: boolean }) {
7 + return <span>{authors.map((a, i) => <span key={a.slug}>{i > 0 && (i === authors.length - 1 ? (locale === "fr" ? " et " : " and ") : ", ")}{link ? <Link href={href(locale, `/authors/${a.slug}`)} className="text-fg-2 no-underline hover:underline">{a.display_name}</Link> : a.display_name}</span>)}</span>;
8 +}
9 +export function StatusChips({ p, locale }: { p: PaperSummary; locale: Locale }) {
10 + const d = t(locale);
11 + return (
12 + <span className="inline-flex flex-wrap gap-1.5">
13 + {p.status !== "published" && <span className="chip chip-burgundy">{d.misc.status[p.status as keyof typeof d.misc.status]}</span>}
14 + {p.publication_status !== "working_paper" && <span className="chip chip-navy">{d.misc.pubStatus[p.publication_status as keyof typeof d.misc.pubStatus]}</span>}
15 + {p.notice && <span className="chip">{p.notice}</span>}
16 + {p.has_pdf ? <span className="chip"><FileText size={12} aria-hidden /> {d.papers.pdf}{p.pages ? ` · ${p.pages} ${d.papers.pages}` : ""}</span> : <span className="chip">{d.paper.notAvailable}</span>}
17 + </span>
18 + );
19 +}
20 +function excerpt(p: PaperSummary, n = 320) { const s = (p.abstract?.[0] ?? "").replace(/\s+/g, " "); return s.length > n ? s.slice(0, n).replace(/\s\S*$/, "") + "…" : s; }
21 +const seriesName = (p: PaperSummary, locale: Locale) => locale === "fr" ? p.series_name_fr : p.series_name_en;
22 +
23 +export function PaperCard({ p, locale, badge }: { p: PaperSummary; locale: Locale; badge?: string }) {
24 + const d = t(locale);
25 + return (
26 + <article className="card p-5 flex flex-col gap-3 h-full">
27 + <div className="flex items-center justify-between gap-2 text-xs">
28 + <span className="font-mono text-navy">{p.identifier}</span>
29 + <span className="flex items-center gap-2">{badge && <span className="chip chip-burgundy">{badge}</span>}<span className="text-fg-3">{p.year ?? ""}</span></span>
30 + </div>
31 + <h3 className="text-lg leading-snug"><Link href={href(locale, `/papers/${p.slug}`)} className="no-underline hover:underline text-fg">{locale === "fr" && p.title_fr ? p.title_fr : p.title}</Link></h3>
32 + <p className="meta"><AuthorsLine authors={p.authors} locale={locale} /></p>
33 + <p className="text-sm text-fg-2 leading-relaxed">{excerpt(p, 260)}</p>
34 + <div className="mt-auto flex flex-wrap items-center gap-1.5 pt-1">
35 + {p.domain && <Link href={href(locale, `/topics/${p.domain.slug}`)} className="chip chip-navy no-underline">{locale === "fr" ? p.domain.name_fr : p.domain.name_en}</Link>}
36 + <StatusChips p={p} locale={locale} />
37 + <Link href={href(locale, `/papers/${p.slug}`)} className="ml-auto text-sm text-navy no-underline hover:underline">{d.papers.view} →</Link>
38 + </div>
39 + </article>
40 + );
41 +}
42 +export function PaperRow({ p, locale, compact = false }: { p: PaperSummary; locale: Locale; compact?: boolean }) {
43 + const d = t(locale);
44 + return (
45 + <article className={`py-4 ${compact ? "py-2.5" : ""} border-b border-line last:border-0`}>
46 + <div className="flex gap-4">
47 + <div className="w-16 shrink-0 pt-0.5 text-xs font-mono text-navy">{p.identifier}</div>
48 + <div className="min-w-0 flex-1">
49 + <h3 className={`${compact ? "text-base" : "text-lg"} leading-snug`}><Link href={href(locale, `/papers/${p.slug}`)} className="no-underline hover:underline text-fg">{locale === "fr" && p.title_fr ? p.title_fr : p.title}</Link></h3>
50 + {!compact && p.subtitle && <p className="text-sm text-fg-2 mt-0.5">{p.subtitle}</p>}
51 + <p className="meta mt-1"><AuthorsLine authors={p.authors} locale={locale} /> · {seriesName(p, locale)} · {fmtDate(locale, p.first_released_on, p.first_released_precision)}{p.last_revised_on && p.last_revised_on !== p.first_released_on ? ` · ${d.paper.revised.toLowerCase()} ${fmtDate(locale, p.last_revised_on, p.last_revised_precision)}` : ""}{p.version_label ? ` · ${p.version_label}` : ""}</p>
52 + {!compact && <p className="text-sm text-fg-2 leading-relaxed mt-2">{excerpt(p)}</p>}
53 + <div className="mt-2 flex flex-wrap items-center gap-1.5">
54 + {p.domain && <Link href={href(locale, `/topics/${p.domain.slug}`)} className="chip chip-navy no-underline">{locale === "fr" ? p.domain.name_fr : p.domain.name_en}</Link>}
55 + {!compact && p.keywords.slice(0, 4).map((k) => <Link key={k} href={href(locale, `/papers?keyword=${encodeURIComponent(k)}`)} className="chip no-underline">{k}</Link>)}
56 + <StatusChips p={p} locale={locale} />
57 + </div>
58 + </div>
59 + </div>
60 + </article>
61 + );
62 +}
63 +export function PaperTable({ items, locale }: { items: PaperSummary[]; locale: Locale }) {
64 + const d = t(locale);
65 + return (
66 + <div className="overflow-x-auto card">
67 + <table className="data">
68 + <thead><tr><th>{d.papers.number}</th><th>{d.paper.identifier}</th><th>{d.paper.authors}</th><th>{d.paper.year}</th><th>{d.paper.version}</th><th>{d.paper.domain}</th><th>{d.paper.pagesLabel}</th><th>{d.paper.viewsLabel}</th><th>{d.paper.downloadsLabel}</th></tr></thead>
69 + <tbody>{items.map((p) => <tr key={p.id}><td className="font-mono text-navy whitespace-nowrap">{p.identifier}</td><td><Link href={href(locale, `/papers/${p.slug}`)} className="text-fg no-underline hover:underline">{p.title}</Link></td><td className="text-fg-2 whitespace-nowrap"><AuthorsLine authors={p.authors} locale={locale} link={false} /></td><td>{p.year ?? ""}</td><td className="whitespace-nowrap">{p.version_label ?? ""}</td><td className="text-fg-2">{p.domain ? (locale === "fr" ? p.domain.name_fr : p.domain.name_en) : ""}</td><td>{p.pages ?? ""}</td><td>{fmtInt(locale, p.view_count)}</td><td>{fmtInt(locale, p.download_count)}</td></tr>)}</tbody>
70 + </table>
71 + </div>
72 + );
73 +}
added src/components/site-footer.tsx +31 −0
@@ -0,0 +1,31 @@
1 +import Link from "next/link";
2 +import { href, t, type Locale } from "@/lib/i18n";
3 +export function SiteFooter({ locale, attribution, institution }: { locale: Locale; attribution: { en: string; fr: string } | null; institution: { name_en: string; name_fr: string; department_en?: string; department_fr?: string; url?: string } | null }) {
4 + const d = t(locale); const year = new Date().getFullYear();
5 + return (
6 + <footer className="mt-16 border-t border-line bg-bg-3/60">
7 + <div className="container py-10 grid gap-8 md:grid-cols-3 text-sm">
8 + <div>
9 + <p className="serif text-lg text-fg">{d.site.title}</p>
10 + <p className="text-fg-3 mt-1">{d.site.secondary}</p>
11 + {institution && <p className="text-fg-2 mt-3">{locale === "fr" ? institution.name_fr : institution.name_en}{(locale === "fr" ? institution.department_fr : institution.department_en) ? ` · ${locale === "fr" ? institution.department_fr : institution.department_en}` : ""}</p>}
12 + </div>
13 + <div>
14 + <p className="label mb-2">{d.footer.attribution}</p>
15 + <p className="text-fg-2 leading-relaxed">{attribution ? (locale === "fr" ? attribution.fr : attribution.en) : ""}</p>
16 + </div>
17 + <div className="grid grid-cols-2 gap-x-4 gap-y-1.5">
18 + {[["/papers", d.nav.papers], ["/series", d.nav.series], ["/topics", d.nav.topics], ["/methods", d.nav.methods], ["/datasets", d.nav.datasets], ["/authors", d.nav.authors], ["/research-map", d.nav.map], ["/timeline", d.nav.timeline], ["/search", d.nav.search], ["/citation", d.nav.citation], ["/about", d.nav.about]].map(([p, l]) => <Link key={p} href={href(locale, p)} className="text-fg-2 no-underline hover:underline">{l}</Link>)}
19 + <Link href="/api/docs" className="text-fg-2 no-underline hover:underline">{d.nav.api}</Link>
20 + <a href="/feed.xml" className="text-fg-2 no-underline hover:underline">{d.footer.feed}</a>
21 + <a href="/api/oai?verb=Identify" className="text-fg-2 no-underline hover:underline">{d.footer.oai}</a>
22 + <a href="/sitemap.xml" className="text-fg-2 no-underline hover:underline">{d.footer.sitemap}</a>
23 + <Link href="/admin" className="text-fg-3 no-underline hover:underline">{d.nav.admin}</Link>
24 + </div>
25 + </div>
26 + <div className="container pb-8 text-xs text-fg-3 flex flex-wrap gap-x-4 gap-y-1">
27 + <span>© {year} Simon-Pierre Boucher.</span><span>{d.footer.rights}</span><span>{d.footer.built}</span>
28 + </div>
29 + </footer>
30 + );
31 +}
added src/components/site-header.tsx +43 −0
@@ -0,0 +1,43 @@
1 +"use client";
2 +import Link from "next/link";
3 +import { usePathname } from "next/navigation";
4 +import { useState } from "react";
5 +import { Menu, X, Search } from "lucide-react";
6 +import { href, t, type Locale } from "@/lib/i18n";
7 +import { ThemeToggle } from "./theme-toggle";
8 +
9 +export function SiteHeader({ locale }: { locale: Locale }) {
10 + const d = t(locale); const path = usePathname(); const [open, setOpen] = useState(false);
11 + const canonical = path.replace(/^\/fr(?=\/|$)/, "") || "/";
12 + const items = [["/papers", d.nav.papers], ["/topics", d.nav.topics], ["/methods", d.nav.methods], ["/datasets", d.nav.datasets], ["/authors", d.nav.authors], ["/research-map", d.nav.map], ["/timeline", d.nav.timeline], ["/about", d.nav.about]] as const;
13 + const other: Locale = locale === "en" ? "fr" : "en";
14 + const active = (p: string) => canonical === p || (p !== "/" && canonical.startsWith(p + "/"));
15 + return (
16 + <header className="sticky top-0 z-40 border-b border-line bg-bg/90 backdrop-blur supports-[backdrop-filter]:bg-bg/75">
17 + <div className="container flex items-center gap-3 h-16">
18 + <Link href={href(locale, "/")} className="flex items-baseline gap-2 no-underline min-w-0">
19 + <span className="serif text-lg sm:text-xl text-fg whitespace-nowrap">{d.site.title}</span>
20 + <span className="hidden lg:inline text-xs uppercase tracking-[.14em] text-fg-3">{d.site.affiliation}</span>
21 + </Link>
22 + <nav aria-label="Primary" className="hidden md:flex items-center gap-1 ml-auto">
23 + {items.map(([p, label]) => <Link key={p} href={href(locale, p)} aria-current={active(p) ? "page" : undefined} className={`px-2.5 py-1.5 rounded-md text-sm no-underline hover:bg-bg-3 ${active(p) ? "text-navy font-medium" : "text-fg-2"}`}>{label}</Link>)}
24 + </nav>
25 + <div className="flex items-center gap-1.5 ml-auto md:ml-2">
26 + <Link href={href(locale, "/search")} aria-label={d.nav.search} className="btn btn-sm !px-2"><Search size={16} aria-hidden /></Link>
27 + <Link href={href(other, canonical)} hrefLang={other} lang={other} aria-label={d.nav.language} className="btn btn-sm font-mono uppercase">{other}</Link>
28 + <ThemeToggle label={d.nav.lightDark} />
29 + <button type="button" className="btn btn-sm !px-2 md:hidden" aria-expanded={open} aria-controls="mobile-nav" aria-label={open ? d.nav.close : d.nav.menu} onClick={() => setOpen((o) => !o)}>{open ? <X size={16} aria-hidden /> : <Menu size={16} aria-hidden />}</button>
30 + </div>
31 + </div>
32 + {open && (
33 + <nav id="mobile-nav" aria-label="Mobile" className="md:hidden border-t border-line bg-bg">
34 + <div className="container py-2 grid grid-cols-2 gap-1">
35 + {items.map(([p, label]) => <Link key={p} href={href(locale, p)} onClick={() => setOpen(false)} className={`px-3 py-2 rounded-md text-sm no-underline hover:bg-bg-3 ${active(p) ? "text-navy font-medium" : "text-fg-2"}`}>{label}</Link>)}
36 + <Link href={href(locale, "/citation")} onClick={() => setOpen(false)} className="px-3 py-2 rounded-md text-sm no-underline text-fg-2 hover:bg-bg-3">{d.nav.citation}</Link>
37 + <Link href="/api/docs" onClick={() => setOpen(false)} className="px-3 py-2 rounded-md text-sm no-underline text-fg-2 hover:bg-bg-3">{d.nav.api}</Link>
38 + </div>
39 + </nav>
40 + )}
41 + </header>
42 + );
43 +}
added src/components/stat-tiles.tsx +14 −0
@@ -0,0 +1,14 @@
1 +import { fmtDate, fmtInt, t, type Locale } from "@/lib/i18n";
2 +import type { stats } from "@/lib/papers/queries";
3 +export function StatTiles({ s, locale }: { s: Awaited<ReturnType<typeof stats>>; locale: Locale }) {
4 + const d = t(locale);
5 + const tiles: [string, string][] = [
6 + [d.stats.papers, fmtInt(locale, s.papers)], [d.stats.manuscripts, fmtInt(locale, s.manuscripts)], [d.stats.topics, fmtInt(locale, s.topics)], [d.stats.authors, fmtInt(locale, s.authors)],
7 + [d.stats.datasets, fmtInt(locale, s.datasets)], [d.stats.methods, fmtInt(locale, s.methods)], [d.stats.pages, fmtInt(locale, s.pages)], [d.stats.latest, s.latest ? fmtDate(locale, s.latest, "month") : "—"],
8 + ];
9 + return (
10 + <dl className="grid grid-cols-2 sm:grid-cols-4 gap-3">
11 + {tiles.map(([label, value]) => <div key={label} className="card px-4 py-3"><dt className="label">{label}</dt><dd className="serif text-2xl mt-1 text-fg">{value}</dd></div>)}
12 + </dl>
13 + );
14 +}
added src/components/theme-toggle.tsx +9 −0
@@ -0,0 +1,9 @@
1 +"use client";
2 +import { useEffect, useState } from "react";
3 +import { Moon, Sun } from "lucide-react";
4 +export function ThemeToggle({ label }: { label: string }) {
5 + const [dark, setDark] = useState(false);
6 + useEffect(() => { setDark(document.documentElement.classList.contains("dark")); }, []);
7 + const toggle = () => { const d = !dark; setDark(d); document.documentElement.classList.toggle("dark", d); try { localStorage.setItem("theme", d ? "dark" : "light"); } catch {} };
8 + return <button type="button" onClick={toggle} aria-label={label} title={label} className="btn btn-sm !px-2" >{dark ? <Sun size={16} aria-hidden /> : <Moon size={16} aria-hidden />}</button>;
9 +}
added src/components/view-beacon.tsx +7 −0
@@ -0,0 +1,7 @@
1 +"use client";
2 +import { useEffect } from "react";
3 +/** Records one page view per visitor per day (server-side de-duplicated, bots excluded). Sends nothing else. */
4 +export function ViewBeacon({ slug }: { slug: string }) {
5 + useEffect(() => { try { navigator.sendBeacon?.("/api/events", new Blob([JSON.stringify({ kind: "view", slug })], { type: "application/json" })) ?? fetch("/api/events", { method: "POST", body: JSON.stringify({ kind: "view", slug }), headers: { "Content-Type": "application/json" }, keepalive: true }); } catch {} }, [slug]);
6 + return null;
7 +}
added src/db/index.ts +14 −0
@@ -0,0 +1,14 @@
1 +import postgres from "postgres";
2 +
3 +export type Sql = ReturnType<typeof postgres>;
4 +const g = globalThis as unknown as { __wpSql?: Sql; __wpSqlPublic?: Sql };
5 +const opts = { max: 6, idle_timeout: 30, prepare: false, transform: { undefined: null }, onnotice: () => {} } as const;
6 +function need(name: string) { const v = process.env[name]; if (!v) throw new Error(`${name} is not set`); return v; }
7 +
8 +/** Full-privilege client (admin, worker, scripts). Never import from public route modules. */
9 +export const sql: Sql = g.__wpSql ?? postgres(need("DATABASE_URL"), opts);
10 +/** Read-only client for everything public: `wp_reader` role (SELECT on public views only, INSERT on events/search_log). */
11 +export const sqlPublic: Sql = g.__wpSqlPublic ?? postgres(process.env.DATABASE_URL_PUBLIC ?? need("DATABASE_URL"), { ...opts, max: 12 });
12 +if (process.env.NODE_ENV !== "production") { g.__wpSql = sql; g.__wpSqlPublic = sqlPublic; }
13 +
14 +export async function closeDb() { await Promise.all([sql.end(), sqlPublic.end()]); }
added src/lib/admin/actions.ts +174 −0
@@ -0,0 +1,174 @@
1 +"use server";
2 +import { redirect } from "next/navigation";
3 +import { revalidatePath } from "next/cache";
4 +import { headers } from "next/headers";
5 +import { z } from "zod";
6 +import { sql } from "@/db";
7 +import { requireAction } from "@/lib/auth/guard";
8 +import { clientIp, revokeOtherSessions, revokeSession, clearSessionCookie } from "@/lib/auth/session";
9 +import { hashPassword, passwordPolicy, verifyPassword } from "@/lib/auth/password";
10 +import { audit } from "@/lib/audit";
11 +import { enqueue } from "@/lib/jobs/queue";
12 +import { makeIdentifier, parseVersionLabel } from "@/lib/papers/identifiers";
13 +import { lexvecFromMetadata } from "@/lib/jobs/handlers";
14 +import { verifySync } from "otplib";
15 +
16 +const s = (f: FormData, k: string) => { const v = f.get(k); return typeof v === "string" ? v.trim() : ""; };
17 +const opt = (f: FormData, k: string) => s(f, k) || null;
18 +const list = (f: FormData, k: string) => f.getAll(k).map(String).filter(Boolean);
19 +const lines = (f: FormData, k: string) => s(f, k).split(/\r?\n/).map((x) => x.trim()).filter(Boolean);
20 +const jsonField = <T,>(f: FormData, k: string, fallback: T): T => { const v = s(f, k); if (!v) return fallback; try { return JSON.parse(v) as T; } catch { throw new Error(`Field ${k} is not valid JSON`); } };
21 +const dateField = (f: FormData, k: string): { date: string | null; precision: "day" | "month" | "year" } => { const v = s(f, k); if (!v) return { date: null, precision: "month" }; if (/^\d{4}$/.test(v)) return { date: `${v}-01-01`, precision: "year" }; if (/^\d{4}-\d{2}$/.test(v)) return { date: `${v}-01`, precision: "month" }; if (/^\d{4}-\d{2}-\d{2}$/.test(v)) return { date: v, precision: "day" }; throw new Error(`Date ${k} must be YYYY, YYYY-MM or YYYY-MM-DD`); };
22 +const actor = async (u: { id: string; email: string }) => ({ id: u.id, label: u.email, ip: clientIp(await headers()) });
23 +function revalidateAll(slug?: string) { for (const p of ["/", "/papers", "/topics", "/methods", "/datasets", "/authors", "/series", "/timeline", "/research-map", "/sitemap.xml", "/feed.xml"]) revalidatePath(p); if (slug) { revalidatePath(`/papers/${slug}`); revalidatePath(`/papers/${slug}/read`); } revalidatePath("/", "layout"); }
24 +
25 +const PaperSchema = z.object({ title: z.string().min(3), number: z.number().int().positive(), series_code: z.string().min(1) });
26 +async function savePaperFields(f: FormData, id: string | null, u: { id: string; email: string }) {
27 + const [series] = await sql`SELECT id, code FROM series WHERE code = ${s(f, "series_code").toUpperCase()}`; if (!series) throw new Error("Unknown series");
28 + const number = Number(s(f, "number")); PaperSchema.parse({ title: s(f, "title"), number, series_code: series.code });
29 + const idp = makeIdentifier(series.code, number); const first = dateField(f, "first_released_on"); const rev = dateField(f, "last_revised_on");
30 + const row = {
31 + series_id: series.id, number, identifier: idp.identifier, slug: idp.slug, internal_code: opt(f, "internal_code"),
32 + title: s(f, "title"), subtitle: opt(f, "subtitle"), title_fr: opt(f, "title_fr"), subtitle_fr: opt(f, "subtitle_fr"),
33 + abstract: sql.json(s(f, "abstract").split(/\n\s*\n/).map((x) => x.replace(/\s+/g, " ").trim()).filter(Boolean)), abstract_fr: s(f, "abstract_fr") ? sql.json(s(f, "abstract_fr").split(/\n\s*\n/).map((x) => x.replace(/\s+/g, " ").trim()).filter(Boolean)) : null,
34 + language: s(f, "language") || "en", research_question: opt(f, "research_question"), research_question_fr: opt(f, "research_question_fr"), hero_headline: opt(f, "hero_headline"), hero_subheadline: opt(f, "hero_subheadline"), notice: opt(f, "notice"), notice_fr: opt(f, "notice_fr"),
35 + contributions: sql.json(jsonField(f, "contributions", [])), methodology: sql.json(jsonField(f, "methodology", [])), data_sections: sql.json(jsonField(f, "data_sections", [])), findings: sql.json(jsonField(f, "findings", [])), reproducibility: sql.json(lines(f, "reproducibility")), artifacts: sql.json(jsonField(f, "artifacts", [])), external_links: sql.json(jsonField(f, "external_links", [])),
36 + keywords: lines(f, "keywords"), jel: s(f, "jel").split(/[,\s]+/).filter(Boolean), publication_status: s(f, "publication_status") || "working_paper", journal_ref: opt(f, "journal_ref"), doi: opt(f, "doi"), ssrn_id: opt(f, "ssrn_id"),
37 + year: s(f, "year") ? Number(s(f, "year")) : null, first_released_on: first.date, first_released_precision: first.precision, last_revised_on: rev.date ?? first.date, last_revised_precision: rev.date ? rev.precision : first.precision,
38 + featured: s(f, "featured") === "on", featured_rank: s(f, "featured_rank") ? Number(s(f, "featured_rank")) : null, primary_domain_id: s(f, "primary_domain_id") ? Number(s(f, "primary_domain_id")) : null,
39 + };
40 + let paperId = id;
41 + if (id) { const [before] = await sql`SELECT * FROM papers WHERE id = ${id}`; await sql`UPDATE papers SET ${sql(row as never)} WHERE id = ${id}`; await audit(await actor(u), "paper.update", "paper", id, { title: before.title, identifier: before.identifier, status: before.status }, { title: row.title, identifier: row.identifier }); }
42 + else { const [r] = await sql`INSERT INTO papers ${sql({ ...row, status: "draft" } as never)} RETURNING id`; paperId = r.id; await audit(await actor(u), "paper.create", "paper", paperId, null, { title: row.title, identifier: row.identifier }); }
43 + const pid = paperId!;
44 + await sql`DELETE FROM paper_authors WHERE paper_id = ${pid}`;
45 + let pos = 1; for (const aid of list(f, "author_ids")) await sql`INSERT INTO paper_authors (paper_id, author_id, position) VALUES (${pid}, ${Number(aid)}, ${pos++})`;
46 + await sql`DELETE FROM paper_topics WHERE paper_id = ${pid} AND assigned_by <> 'inferred'`; for (const t of list(f, "topic_ids")) await sql`INSERT INTO paper_topics (paper_id, topic_id, assigned_by) VALUES (${pid}, ${Number(t)}, 'editor') ON CONFLICT DO NOTHING`;
47 + if (row.primary_domain_id) await sql`INSERT INTO paper_topics (paper_id, topic_id, assigned_by) VALUES (${pid}, ${row.primary_domain_id}, 'editor') ON CONFLICT DO NOTHING`;
48 + await sql`DELETE FROM paper_methods WHERE paper_id = ${pid} AND assigned_by <> 'inferred'`; for (const m of list(f, "method_ids")) await sql`INSERT INTO paper_methods (paper_id, method_id, assigned_by) VALUES (${pid}, ${Number(m)}, 'editor') ON CONFLICT DO NOTHING`;
49 + await sql`DELETE FROM paper_datasets WHERE paper_id = ${pid} AND assigned_by <> 'inferred'`; for (const d of list(f, "dataset_ids")) await sql`INSERT INTO paper_datasets (paper_id, dataset_id, assigned_by) VALUES (${pid}, ${Number(d)}, 'editor') ON CONFLICT DO NOTHING`;
50 + await lexvecFromMetadata(pid);
51 + await enqueue("embed_paper", { paperId: pid }, { dedupe: true }); await enqueue("relations", {}, { dedupe: true, runAfter: new Date(Date.now() + 20_000) });
52 + return { id: pid, slug: idp.slug };
53 +}
54 +export async function createPaperAction(f: FormData) { const u = await requireAction(f); const r = await savePaperFields(f, null, u); revalidateAll(r.slug); redirect(`/admin/papers/${r.id}?saved=1`); }
55 +export async function updatePaperAction(f: FormData) { const u = await requireAction(f); const id = s(f, "id"); const r = await savePaperFields(f, id, u); revalidateAll(r.slug); redirect(`/admin/papers/${id}?saved=1`); }
56 +
57 +export async function setStatusAction(f: FormData) {
58 + const u = await requireAction(f); const id = s(f, "id"); const status = s(f, "status"); const reason = opt(f, "reason");
59 + if (!["draft", "reserved", "published", "withdrawn", "superseded"].includes(status)) throw new Error("bad status");
60 + const [p] = await sql`SELECT id, slug, status, title, identifier, current_version_id FROM papers WHERE id = ${id}`; if (!p) throw new Error("not found");
61 + if (status === "published" && !p.current_version_id) { const [v] = await sql`SELECT id FROM paper_versions WHERE paper_id = ${id} AND NOT withdrawn ORDER BY version_no DESC LIMIT 1`; if (v) await sql`UPDATE papers SET current_version_id = ${v.id} WHERE id = ${id}`; }
62 + await sql`UPDATE papers SET status = ${status}, published_at = CASE WHEN ${status} = 'published' AND published_at IS NULL THEN now() ELSE published_at END, notice = CASE WHEN ${status} IN ('withdrawn', 'superseded') THEN coalesce(${reason}, notice) ELSE notice END WHERE id = ${id}`;
63 + await audit(await actor(u), `paper.status.${status}`, "paper", id, { status: p.status }, { status, reason, identifier: p.identifier });
64 + await enqueue("relations", {}, { dedupe: true }); revalidateAll(p.slug); redirect(`/admin/papers/${id}?saved=1`);
65 +}
66 +export async function deletePaperAction(f: FormData) {
67 + const u = await requireAction(f); const id = s(f, "id");
68 + const [p] = await sql`SELECT slug, identifier, status, title FROM papers WHERE id = ${id}`; if (!p) throw new Error("not found");
69 + if (p.status === "published") throw new Error("Published papers cannot be deleted; withdraw them instead.");
70 + await sql`DELETE FROM papers WHERE id = ${id}`; await audit(await actor(u), "paper.delete", "paper", id, { identifier: p.identifier, title: p.title, status: p.status }, null);
71 + revalidateAll(); redirect("/admin/papers?deleted=1");
72 +}
73 +export async function versionAction(f: FormData) {
74 + const u = await requireAction(f); const op = s(f, "op"); const vid = s(f, "version_id");
75 + const [v] = await sql`SELECT v.*, p.slug FROM paper_versions v JOIN papers p ON p.id = v.paper_id WHERE v.id = ${vid}`; if (!v) throw new Error("version not found");
76 + if (op === "withdraw") { await sql`UPDATE paper_versions SET withdrawn = true, withdrawn_reason = ${opt(f, "reason")} WHERE id = ${vid}`; await audit(await actor(u), "version.withdraw", "paper_version", vid, { label: v.label }, { reason: opt(f, "reason") }); }
77 + else if (op === "restore") { await sql`UPDATE paper_versions SET withdrawn = false, withdrawn_reason = NULL WHERE id = ${vid}`; await audit(await actor(u), "version.restore", "paper_version", vid, null, { label: v.label }); }
78 + else if (op === "make_current") { await sql.begin(async (tx) => { await tx`UPDATE paper_versions SET is_current = false WHERE paper_id = ${v.paper_id}`; await tx`UPDATE paper_versions SET is_current = true WHERE id = ${vid}`; await tx`UPDATE papers SET current_version_id = ${vid}, last_revised_on = coalesce(${v.released_on}, last_revised_on), last_revised_precision = ${v.released_precision} WHERE id = ${v.paper_id}`; }); if (v.file_id) await enqueue("process_pdf", { fileId: v.file_id, paperId: v.paper_id, versionId: vid }, { dedupe: true }); await audit(await actor(u), "version.make_current", "paper_version", vid, null, { label: v.label }); }
79 + else if (op === "update") { const d = dateField(f, "released_on"); const pv = parseVersionLabel(s(f, "label")) ?? { label: v.label }; await sql`UPDATE paper_versions SET label = ${pv.label}, released_on = ${d.date}, released_precision = ${d.precision}, description = ${opt(f, "description")}, description_fr = ${opt(f, "description_fr")} WHERE id = ${vid}`; if (v.is_current) await sql`UPDATE papers SET last_revised_on = ${d.date}, last_revised_precision = ${d.precision} WHERE id = ${v.paper_id}`; await audit(await actor(u), "version.update", "paper_version", vid, { label: v.label }, { label: pv.label }); }
80 + revalidateAll(v.slug); redirect(`/admin/papers/${v.paper_id}?saved=1#versions`);
81 +}
82 +export async function relationAction(f: FormData) {
83 + const u = await requireAction(f); const op = s(f, "op"); const from = s(f, "paper_id");
84 + if (op === "add") { const [to] = await sql`SELECT id, identifier FROM papers WHERE slug = ${s(f, "to_slug").toLowerCase()} OR identifier = ${s(f, "to_slug").toUpperCase()}`; if (!to) throw new Error("target paper not found"); const expl = s(f, "explanation") || "Author-curated relationship";
85 + for (const [a, b] of [[from, to.id], [to.id, from]]) await sql`INSERT INTO paper_relations (from_paper_id, to_paper_id, kind, weight, explanation, explanation_fr, inferred) VALUES (${a}, ${b}, 'curated', 1, ${expl}, ${opt(f, "explanation_fr") ?? expl}, false) ON CONFLICT (from_paper_id, to_paper_id, kind) DO UPDATE SET explanation = EXCLUDED.explanation, explanation_fr = EXCLUDED.explanation_fr`;
86 + await audit(await actor(u), "relation.add", "paper", from, null, { to: to.identifier, explanation: expl }); }
87 + else if (op === "remove") { const [r] = await sql`DELETE FROM paper_relations WHERE id = ${Number(s(f, "relation_id"))} AND kind = 'curated' RETURNING to_paper_id`; if (r) await sql`DELETE FROM paper_relations WHERE from_paper_id = ${r.to_paper_id} AND to_paper_id = ${from} AND kind = 'curated'`; await audit(await actor(u), "relation.remove", "paper", from, null, { relation_id: s(f, "relation_id") }); }
88 + const [p] = await sql`SELECT slug FROM papers WHERE id = ${from}`; revalidateAll(p?.slug); redirect(`/admin/papers/${from}?saved=1#relations`);
89 +}
90 +export async function redirectAction(f: FormData) {
91 + const u = await requireAction(f); const op = s(f, "op"); const back = s(f, "back") || "/admin/papers";
92 + if (op === "add") { const from = s(f, "from_path"); const to = s(f, "to_path"); if (!from.startsWith("/") || !to.startsWith("/")) throw new Error("paths must start with /"); await sql`INSERT INTO redirects (from_path, to_path, status, note) VALUES (${from}, ${to}, ${Number(s(f, "status") || 301)}, ${opt(f, "note")}) ON CONFLICT (from_path) DO UPDATE SET to_path = EXCLUDED.to_path, status = EXCLUDED.status, note = EXCLUDED.note`; await audit(await actor(u), "redirect.add", "redirect", from, null, { to }); }
93 + else { await sql`DELETE FROM redirects WHERE from_path = ${s(f, "from_path")}`; await audit(await actor(u), "redirect.remove", "redirect", s(f, "from_path"), null, null); }
94 + redirect(back + (back.includes("?") ? "&" : "?") + "saved=1");
95 +}
96 +export async function taxonomyAction(f: FormData) {
97 + const u = await requireAction(f); const kind = s(f, "kind"); const op = s(f, "op"); const id = s(f, "id") ? Number(s(f, "id")) : null;
98 + const slugify = (x: string) => x.toLowerCase().normalize("NFD").replace(/[̀-ͯ]/g, "").replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, "");
99 + const slug = s(f, "slug") ? slugify(s(f, "slug")) : slugify(s(f, "name_en") || s(f, "display_name"));
100 + if (op === "delete") {
101 + if (!id) throw new Error("id required");
102 + if (kind === "author") await sql`DELETE FROM authors WHERE id = ${id} AND NOT EXISTS (SELECT 1 FROM paper_authors WHERE author_id = ${id})`;
103 + if (kind === "topic") await sql`DELETE FROM topics WHERE id = ${id}`; if (kind === "method") await sql`DELETE FROM methods WHERE id = ${id}`; if (kind === "dataset") await sql`DELETE FROM datasets WHERE id = ${id}`;
104 + await audit(await actor(u), `${kind}.delete`, kind, String(id), null, null); redirect("/admin/taxonomy?saved=1");
105 + }
106 + if (kind === "author") { const row = { slug, given_name: s(f, "given_name"), family_name: s(f, "family_name"), display_name: s(f, "display_name") || `${s(f, "given_name")} ${s(f, "family_name")}`, orcid: opt(f, "orcid"), email: opt(f, "email"), website: opt(f, "website"), github: opt(f, "github"), google_scholar: opt(f, "google_scholar"), ssrn_url: opt(f, "ssrn_url"), affiliation_en: opt(f, "affiliation_en"), affiliation_fr: opt(f, "affiliation_fr"), department_en: opt(f, "department_en"), department_fr: opt(f, "department_fr"), bio_en: opt(f, "bio_en"), bio_fr: opt(f, "bio_fr"), is_primary: s(f, "is_primary") === "on", sort: Number(s(f, "sort") || 0) };
107 + if (id) await sql`UPDATE authors SET ${sql(row as never)} WHERE id = ${id}`; else await sql`INSERT INTO authors ${sql(row as never)}`; }
108 + if (kind === "topic") { const row = { slug, name_en: s(f, "name_en"), name_fr: s(f, "name_fr") || s(f, "name_en"), description_en: opt(f, "description_en"), description_fr: opt(f, "description_fr"), kind: s(f, "topic_kind") === "domain" ? "domain" : "topic", accent: opt(f, "accent"), sort: Number(s(f, "sort") || 0), is_active: s(f, "is_active") !== "off" }; if (id) await sql`UPDATE topics SET ${sql(row as never)} WHERE id = ${id}`; else await sql`INSERT INTO topics ${sql(row as never)}`; }
109 + if (kind === "method") { const row = { slug, name_en: s(f, "name_en"), name_fr: s(f, "name_fr") || s(f, "name_en"), description_en: opt(f, "description_en"), description_fr: opt(f, "description_fr"), category: s(f, "category") || "general", sort: Number(s(f, "sort") || 0) }; if (id) await sql`UPDATE methods SET ${sql(row as never)} WHERE id = ${id}`; else await sql`INSERT INTO methods ${sql(row as never)}`; }
110 + if (kind === "dataset") { const row = { slug, name_en: s(f, "name_en"), name_fr: s(f, "name_fr") || s(f, "name_en"), provider: opt(f, "provider"), coverage: opt(f, "coverage"), frequency: opt(f, "frequency"), access_conditions: opt(f, "access_conditions"), license: opt(f, "license"), url: opt(f, "url"), description_en: opt(f, "description_en"), description_fr: opt(f, "description_fr"), is_public: s(f, "is_public") === "on", sort: Number(s(f, "sort") || 0) }; if (id) await sql`UPDATE datasets SET ${sql(row as never)} WHERE id = ${id}`; else await sql`INSERT INTO datasets ${sql(row as never)}`; }
111 + if (kind === "series") { const row = { slug, code: s(f, "code").toUpperCase(), name_en: s(f, "name_en"), name_fr: s(f, "name_fr") || s(f, "name_en"), description_en: opt(f, "description_en"), description_fr: opt(f, "description_fr"), issn: opt(f, "issn"), sort: Number(s(f, "sort") || 0), is_active: s(f, "is_active") !== "off" }; if (id) await sql`UPDATE series SET ${sql(row as never)} WHERE id = ${id}`; else await sql`INSERT INTO series ${sql(row as never)}`; }
112 + await audit(await actor(u), `${kind}.${id ? "update" : "create"}`, kind, id ? String(id) : slug, null, { slug }); await enqueue("relations", {}, { dedupe: true }); revalidateAll(); redirect("/admin/taxonomy?saved=1");
113 +}
114 +export async function settingsAction(f: FormData) {
115 + const u = await requireAction(f);
116 + const values: Record<string, unknown> = {
117 + "site.attribution": { en: s(f, "attribution_en"), fr: s(f, "attribution_fr") },
118 + "site.institution": { name_en: s(f, "inst_name_en"), name_fr: s(f, "inst_name_fr"), department_en: opt(f, "inst_dept_en"), department_fr: opt(f, "inst_dept_fr"), url: opt(f, "inst_url"), logo_authorized: s(f, "logo_authorized") === "on" },
119 + "site.contact": { email: opt(f, "contact_email"), website: opt(f, "contact_website") },
120 + "home.featured_limit": Number(s(f, "featured_limit") || 4),
121 + "ingest.extra_dirs": lines(f, "ingest_dirs"),
122 + };
123 + for (const [k, v] of Object.entries(values)) await sql`INSERT INTO settings (key, value) VALUES (${k}, ${sql.json(v as never)}) ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, updated_at = now()`;
124 + await audit(await actor(u), "settings.update", "settings", null, null, { keys: Object.keys(values) }); revalidateAll(); redirect("/admin/settings?saved=1");
125 +}
126 +export async function ingestAction(f: FormData) {
127 + const u = await requireAction(f); const op = s(f, "op");
128 + if (op === "scan") { await enqueue("ingest_scan", {}, { dedupe: true }); await audit(await actor(u), "ingest.scan", "ingest", null, null, null); redirect("/admin/ingest?queued=1"); }
129 + const cid = s(f, "candidate_id"); const [c] = await sql`SELECT * FROM ingest_candidates WHERE id = ${cid}`; if (!c) throw new Error("candidate not found");
130 + if (op === "reject") { await sql`UPDATE ingest_candidates SET status = 'rejected', reviewed_by = ${u.id}, reviewed_at = now(), review_note = ${opt(f, "note")} WHERE id = ${cid}`; await audit(await actor(u), "ingest.reject", "ingest_candidate", cid, null, { note: opt(f, "note") }); redirect("/admin/ingest?saved=1"); }
131 + if (op === "reprocess") { await sql`UPDATE ingest_candidates SET status = 'queued' WHERE id = ${cid}`; await enqueue("ingest_candidate", { candidateId: cid }); redirect(`/admin/ingest/${cid}?queued=1`); }
132 + const d = dateField(f, "released_on"); const ver = parseVersionLabel(s(f, "version_label") || "v1") ?? { versionNo: 1, label: "v1" };
133 + if (op === "attach") {
134 + const [p] = await sql`SELECT id, slug, identifier FROM papers WHERE id = ${s(f, "target_paper_id")}`; if (!p) throw new Error("target paper not found");
135 + const [{ next }] = await sql`SELECT coalesce(max(version_no), 0) + 1 AS next FROM paper_versions WHERE paper_id = ${p.id}`;
136 + const [v] = await sql`INSERT INTO paper_versions (paper_id, version_no, label, released_on, released_precision, description, file_id, sha256, is_current) VALUES (${p.id}, ${next}, ${ver.label}, ${d.date}, ${d.precision}, ${opt(f, "description")}, ${c.file_id}, ${c.sha256}, ${s(f, "make_current") === "on"}) RETURNING id`;
137 + if (s(f, "make_current") === "on") { await sql`UPDATE paper_versions SET is_current = false WHERE paper_id = ${p.id} AND id <> ${v.id}`; await sql`UPDATE papers SET current_version_id = ${v.id}, last_revised_on = ${d.date}, last_revised_precision = ${d.precision} WHERE id = ${p.id}`; await enqueue("process_pdf", { fileId: c.file_id, paperId: p.id, versionId: v.id }); } else await enqueue("process_pdf", { fileId: c.file_id, versionId: v.id });
138 + await sql`UPDATE ingest_candidates SET status = 'merged', paper_id = ${p.id}, reviewed_by = ${u.id}, reviewed_at = now() WHERE id = ${cid}`;
139 + await audit(await actor(u), "ingest.attach", "ingest_candidate", cid, null, { paper: p.identifier, version: ver.label }); revalidateAll(p.slug); redirect(`/admin/papers/${p.id}?saved=1#versions`);
140 + }
141 + if (op === "approve") {
142 + const [series] = await sql`SELECT id, code FROM series WHERE code = ${s(f, "series_code").toUpperCase()}`; if (!series) throw new Error("Unknown series");
143 + const number = Number(s(f, "number")); if (!number) throw new Error("number required"); const idp = makeIdentifier(series.code, number);
144 + const [dup] = await sql`SELECT id FROM papers WHERE slug = ${idp.slug}`; if (dup) throw new Error(`${idp.identifier} already exists — use "attach as version" instead`);
145 + const prop = c.proposal as Record<string, unknown>;
146 + const [p] = await sql`INSERT INTO papers (series_id, number, identifier, slug, title, abstract, keywords, jel, year, first_released_on, first_released_precision, last_revised_on, last_revised_precision, status, language)
147 + VALUES (${series.id}, ${number}, ${idp.identifier}, ${idp.slug}, ${s(f, "title") || String(prop.title ?? "Untitled")}, ${sql.json(s(f, "abstract") ? [s(f, "abstract")] : prop.abstract ? [String(prop.abstract)] : [])}, ${lines(f, "keywords")}, ${s(f, "jel").split(/[,\s]+/).filter(Boolean)}, ${s(f, "year") ? Number(s(f, "year")) : null}, ${d.date}, ${d.precision}, ${d.date}, ${d.precision}, 'draft', ${s(f, "language") || "en"}) RETURNING id`;
148 + let pos = 1; for (const aid of list(f, "author_ids")) await sql`INSERT INTO paper_authors (paper_id, author_id, position) VALUES (${p.id}, ${Number(aid)}, ${pos++})`;
149 + const [v] = await sql`INSERT INTO paper_versions (paper_id, version_no, label, released_on, released_precision, description, file_id, sha256, is_current) VALUES (${p.id}, 1, ${ver.label}, ${d.date}, ${d.precision}, ${opt(f, "description") ?? "Initial release"}, ${c.file_id}, ${c.sha256}, true) RETURNING id`;
150 + await sql`UPDATE papers SET current_version_id = ${v.id} WHERE id = ${p.id}`;
151 + await sql`UPDATE ingest_candidates SET status = 'approved', paper_id = ${p.id}, reviewed_by = ${u.id}, reviewed_at = now() WHERE id = ${cid}`;
152 + await enqueue("process_pdf", { fileId: c.file_id, paperId: p.id, versionId: v.id }); await lexvecFromMetadata(p.id);
153 + await audit(await actor(u), "ingest.approve", "ingest_candidate", cid, null, { paper: idp.identifier, note: "created as draft; publish explicitly" }); redirect(`/admin/papers/${p.id}?saved=1`);
154 + }
155 + throw new Error("unknown op");
156 +}
157 +export async function systemAction(f: FormData) {
158 + const u = await requireAction(f); const op = s(f, "op");
159 + if (op === "backup") await enqueue("backup", {}, { dedupe: true }); if (op === "relations") await enqueue("relations", {}, { dedupe: true }); if (op === "stats") await enqueue("daily_stats", {}, { dedupe: true });
160 + if (op === "embed_all") { const rows = await sql`SELECT id FROM papers WHERE status = 'published'`; for (const r of rows) await enqueue("embed_paper", { paperId: r.id }, { dedupe: true }); }
161 + if (op === "retry") await sql`UPDATE jobs SET status = 'queued', run_after = now(), attempts = 0 WHERE status = 'failed'`;
162 + await audit(await actor(u), `system.${op}`, "system", null, null, null); redirect("/admin/system?queued=1");
163 +}
164 +export async function accountAction(f: FormData) {
165 + const u = await requireAction(f); const op = s(f, "op");
166 + const [me] = await sql`SELECT * FROM users WHERE id = ${u.id}`;
167 + if (op === "password") { const cur = s(f, "current"), nw = s(f, "new"), conf = s(f, "confirm"); if (!(await verifyPassword(me.password_hash, cur))) throw new Error("Current password is wrong"); if (nw !== conf) throw new Error("Passwords do not match"); const pol = passwordPolicy(nw); if (pol) throw new Error(pol);
168 + await sql`UPDATE users SET password_hash = ${await hashPassword(nw)}, must_change_password = false WHERE id = ${u.id}`; await revokeOtherSessions(u.id, u.sessionId); await audit(await actor(u), "user.password_change", "user", u.id, null, null); redirect("/admin/account?saved=1"); }
169 + if (op === "totp_enable") { const secret = s(f, "secret"); const code = s(f, "code"); if (!verifySync({ token: code, secret }).valid) throw new Error("Invalid code"); await sql`UPDATE users SET totp_secret = ${secret}, totp_enabled = true WHERE id = ${u.id}`; await sql`UPDATE sessions SET mfa_passed = true WHERE id = ${u.sessionId}`; await audit(await actor(u), "user.totp_enable", "user", u.id, null, null); redirect("/admin/account?saved=1"); }
170 + if (op === "totp_disable") { if (!(await verifyPassword(me.password_hash, s(f, "current")))) throw new Error("Current password is wrong"); await sql`UPDATE users SET totp_secret = NULL, totp_enabled = false WHERE id = ${u.id}`; await audit(await actor(u), "user.totp_disable", "user", u.id, null, null); redirect("/admin/account?saved=1"); }
171 + if (op === "revoke_others") { await revokeOtherSessions(u.id, u.sessionId); redirect("/admin/account?saved=1"); }
172 + if (op === "logout") { await revokeSession(u.sessionId); await clearSessionCookie(); await audit(await actor(u), "user.logout", "user", u.id, null, null); redirect("/login"); }
173 + throw new Error("unknown op");
174 +}
added src/lib/admin/queries.ts +38 −0
@@ -0,0 +1,38 @@
1 +import "server-only";
2 +import { sql } from "@/db";
3 +export async function adminPaperList(q?: string, status?: string) {
4 + return sql`SELECT p.id, p.identifier, p.slug, p.title, p.status, p.publication_status, p.year, p.featured, p.updated_at::text, p.view_count::int, p.download_count::int, s.code AS series_code, v.label AS version_label, (v.file_id IS NOT NULL) AS has_pdf,
5 + (SELECT string_agg(a.display_name, ', ' ORDER BY pa.position) FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id) AS authors
6 + FROM papers p JOIN series s ON s.id = p.series_id LEFT JOIN paper_versions v ON v.id = p.current_version_id
7 + WHERE (${q ?? null}::text IS NULL OR p.title ILIKE '%' || ${q ?? ""} || '%' OR p.identifier ILIKE '%' || ${q ?? ""} || '%') AND (${status ?? null}::text IS NULL OR p.status = ${status ?? ""}) ORDER BY s.sort, p.number DESC`;
8 +}
9 +export async function adminPaper(id: string) {
10 + const [p] = await sql`SELECT p.*, s.code AS series_code FROM papers p JOIN series s ON s.id = p.series_id WHERE p.id = ${id} OR p.slug = ${id}`; if (!p) return null;
11 + const [authors, topics, methods, datasets, versions, relations, redirects] = await Promise.all([
12 + sql`SELECT a.id, a.display_name, pa.position FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = ${p.id} ORDER BY pa.position`,
13 + sql`SELECT topic_id AS id, assigned_by FROM paper_topics WHERE paper_id = ${p.id}`,
14 + sql`SELECT method_id AS id, assigned_by FROM paper_methods WHERE paper_id = ${p.id}`,
15 + sql`SELECT dataset_id AS id, assigned_by FROM paper_datasets WHERE paper_id = ${p.id}`,
16 + sql`SELECT v.*, f.integrity_ok, f.text_status, f.bytes AS file_bytes FROM paper_versions v LEFT JOIN files f ON f.id = v.file_id WHERE v.paper_id = ${p.id} ORDER BY v.version_no DESC`,
17 + sql`SELECT r.id, r.kind, r.weight, r.explanation, r.inferred, q.identifier, q.title, q.slug FROM paper_relations r JOIN papers q ON q.id = r.to_paper_id WHERE r.from_paper_id = ${p.id} ORDER BY r.kind, r.weight DESC`,
18 + sql`SELECT * FROM redirects WHERE to_path = ${"/papers/" + p.slug} ORDER BY from_path`,
19 + ]);
20 + return { ...(p as Record<string, any>), authors, topics, methods, datasets, versions, relations, redirects } as Record<string, any> & { authors: Record<string, any>[]; topics: Record<string, any>[]; methods: Record<string, any>[]; datasets: Record<string, any>[]; versions: Record<string, any>[]; relations: Record<string, any>[]; redirects: Record<string, any>[] };
21 +}
22 +export async function taxonomy() {
23 + const [series, authors, topics, methods, datasets] = await Promise.all([sql`SELECT * FROM series ORDER BY sort`, sql`SELECT * FROM authors ORDER BY sort, family_name`, sql`SELECT t.*, (SELECT count(*)::int FROM paper_topics pt WHERE pt.topic_id = t.id) AS n FROM topics t ORDER BY kind, sort`, sql`SELECT m.*, (SELECT count(*)::int FROM paper_methods pm WHERE pm.method_id = m.id) AS n FROM methods m ORDER BY category, sort`, sql`SELECT d.*, (SELECT count(*)::int FROM paper_datasets pd WHERE pd.dataset_id = d.id) AS n FROM datasets d ORDER BY sort`]);
24 + return { series, authors, topics, methods, datasets };
25 +}
26 +export async function dashboard() {
27 + const [[counts], queue, jobs, [backup], audit, searches, views, ai] = await Promise.all([
28 + sql`SELECT count(*) FILTER (WHERE status = 'published')::int AS published, count(*) FILTER (WHERE status = 'draft')::int AS draft, count(*) FILTER (WHERE status = 'reserved')::int AS reserved, count(*) FILTER (WHERE status = 'withdrawn')::int AS withdrawn, count(*)::int AS total FROM papers`,
29 + sql`SELECT status, count(*)::int AS n FROM ingest_candidates GROUP BY status`,
30 + sql`SELECT status, count(*)::int AS n FROM jobs WHERE created_at > now() - interval '7 days' GROUP BY status`,
31 + sql`SELECT * FROM backups ORDER BY created_at DESC LIMIT 1`,
32 + sql`SELECT * FROM audit_log ORDER BY at DESC LIMIT 12`,
33 + sql`SELECT query, count(*)::int AS n, avg(results)::int AS avg_results FROM search_log WHERE at > now() - interval '30 days' GROUP BY query ORDER BY n DESC LIMIT 10`,
34 + sql`SELECT day::text, sum(views)::int AS views, sum(downloads)::int AS downloads FROM paper_stats_daily WHERE day > current_date - 30 GROUP BY day ORDER BY day`,
35 + sql`SELECT count(*) FILTER (WHERE embedding IS NOT NULL)::int AS embedded, count(*)::int AS total, (SELECT count(*)::int FROM paper_chunks WHERE embedding IS NOT NULL) AS chunks_embedded, (SELECT count(*)::int FROM paper_chunks) AS chunks FROM papers WHERE status = 'published'`,
36 + ]);
37 + return { counts, queue, jobs, backup: backup ?? null, audit, searches, views, ai: ai[0] };
38 +}
added src/lib/ai/embeddings.ts +29 −0
@@ -0,0 +1,29 @@
1 +import { AI_PROVIDER } from "@/lib/env";
2 +/** Optional OpenAI embeddings. Model availability is verified at runtime (GET /v1/models/<id>) before first use; failures degrade to lexical only. */
3 +const MODEL = process.env.OPENAI_EMBEDDING_MODEL ?? "text-embedding-3-small";
4 +export const EMBEDDING_DIM = 1536;
5 +let verified: boolean | null = null;
6 +export function embeddingsEnabled() { return AI_PROVIDER === "openai" && !!process.env.OPENAI_API_KEY; }
7 +export async function verifyEmbeddingModel(): Promise<{ ok: boolean; model: string; reason?: string }> {
8 + if (!embeddingsEnabled()) return { ok: false, model: MODEL, reason: "AI provider disabled" };
9 + if (verified !== null) return { ok: verified, model: MODEL };
10 + try {
11 + const r = await fetch(`https://api.openai.com/v1/models/${encodeURIComponent(MODEL)}`, { headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}` }, signal: AbortSignal.timeout(15000) });
12 + verified = r.ok; return { ok: r.ok, model: MODEL, reason: r.ok ? undefined : `HTTP ${r.status}` };
13 + } catch (e) { verified = false; return { ok: false, model: MODEL, reason: (e as Error).message }; }
14 +}
15 +/** Embed up to 64 inputs per call; inputs are truncated to ~8,000 characters (model limit 8,191 tokens). Returns null when disabled. */
16 +export async function embed(inputs: string[]): Promise<number[][] | null> {
17 + if (!embeddingsEnabled()) return null;
18 + const v = await verifyEmbeddingModel(); if (!v.ok) return null;
19 + const out: number[][] = [];
20 + for (let i = 0; i < inputs.length; i += 64) {
21 + const batch = inputs.slice(i, i + 64).map((s) => s.slice(0, 8000));
22 + const r = await fetch("https://api.openai.com/v1/embeddings", { method: "POST", headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}`, "Content-Type": "application/json" }, body: JSON.stringify({ model: MODEL, input: batch, dimensions: EMBEDDING_DIM }), signal: AbortSignal.timeout(60000) });
23 + if (!r.ok) throw new Error(`embeddings HTTP ${r.status}: ${(await r.text()).slice(0, 200)}`);
24 + const j = (await r.json()) as { data: { index: number; embedding: number[] }[] };
25 + for (const d of j.data.sort((a, b) => a.index - b.index)) out.push(d.embedding);
26 + }
27 + return out;
28 +}
29 +export const EMBEDDING_MODEL = MODEL;
added src/lib/ai/lexvec.ts +17 −0
@@ -0,0 +1,17 @@
1 +/** Deterministic hashed TF-IDF-style vector (512 dims) — a provider-free similarity signal. Labelled "lexical (inferred)" in the UI. */
2 +const DIM = 512;
3 +const STOP = new Set("the of and in to a is for on with that this by as are from we an be at or it its which these those their not but also than into can has have was were our such more one two across between within using used use based paper results model models data".split(" "));
4 +function fnv(s: string): number { let h = 0x811c9dc5; for (let i = 0; i < s.length; i++) { h ^= s.charCodeAt(i); h = Math.imul(h, 0x01000193) >>> 0; } return h; }
5 +export function tokenize(text: string): string[] {
6 + return text.toLowerCase().normalize("NFD").replace(/[̀-ͯ]/g, "").replace(/[^a-z0-9 ]+/g, " ").split(/\s+/).filter((w) => w.length > 2 && !STOP.has(w) && !/^\d+$/.test(w));
7 +}
8 +export function lexvec(text: string): number[] {
9 + const v = new Float64Array(DIM); const toks = tokenize(text); const tf = new Map<string, number>();
10 + for (const t of toks) tf.set(t, (tf.get(t) ?? 0) + 1);
11 + for (let i = 0; i < toks.length - 1; i++) { const bg = toks[i] + "_" + toks[i + 1]; tf.set(bg, (tf.get(bg) ?? 0) + 0.5); }
12 + for (const [t, c] of tf) { const h = fnv(t); const sign = (h & 1) ? 1 : -1; v[h % DIM] += sign * (1 + Math.log(c)); }
13 + let n = 0; for (let i = 0; i < DIM; i++) n += v[i] * v[i]; n = Math.sqrt(n) || 1;
14 + return Array.from(v, (x) => x / n);
15 +}
16 +export function cosine(a: number[], b: number[]): number { let s = 0; for (let i = 0; i < a.length; i++) s += a[i] * b[i]; return s; }
17 +export const LEXVEC_DIM = DIM;
added src/lib/api.ts +8 −0
@@ -0,0 +1,8 @@
1 +import { rateLimit } from "@/lib/auth/ratelimit";
2 +export function ip(req: Request) { return (req.headers.get("x-forwarded-for")?.split(",")[0] ?? req.headers.get("x-real-ip") ?? "0.0.0.0").trim(); }
3 +export function limited(req: Request, bucket: string, max = 120, windowMs = 60_000) { return !rateLimit(`${bucket}:${ip(req)}`, max, windowMs); }
4 +export function json(data: unknown, init: ResponseInit & { maxAge?: number } = {}) {
5 + const { maxAge = 60, ...rest } = init;
6 + return Response.json(data, { ...rest, headers: { "Cache-Control": `public, max-age=${maxAge}, s-maxage=${maxAge}`, ...(rest.headers ?? {}) } });
7 +}
8 +export const tooMany = () => Response.json({ error: "Too many requests" }, { status: 429, headers: { "Retry-After": "60" } });
added src/lib/audit.ts +5 −0
@@ -0,0 +1,5 @@
1 +import { sql } from "@/db";
2 +export async function audit(actor: { id?: string | null; label: string; ip?: string | null }, action: string, entityType: string, entityId: string | null, before?: unknown, after?: unknown) {
3 + await sql`INSERT INTO audit_log (actor_id, actor_label, action, entity_type, entity_id, before, after, ip)
4 + VALUES (${actor.id ?? null}, ${actor.label}, ${action}, ${entityType}, ${entityId}, ${before === undefined ? null : sql.json(before as never)}, ${after === undefined ? null : sql.json(after as never)}, ${actor.ip ?? null})`;
5 +}
added src/lib/auth/attempts.ts +14 −0
@@ -0,0 +1,14 @@
1 +import { sql } from "@/db";
2 +/** Login limiter backed by the DB: 5 failures / 15 min per IP or per e-mail, exponential lockout up to 60 min. */
3 +export async function loginAllowed(ip: string, email: string): Promise<{ ok: true } | { ok: false; retryAfterSec: number }> {
4 + const [r] = await sql`SELECT
5 + (SELECT count(*) FROM login_attempts WHERE ip = ${ip} AND success = false AND created_at > now() - interval '15 minutes') AS by_ip,
6 + (SELECT count(*) FROM login_attempts WHERE email = ${email} AND success = false AND created_at > now() - interval '15 minutes') AS by_user`;
7 + const failures = Math.max(Number(r.by_ip), Number(r.by_user));
8 + if (failures < 5) return { ok: true };
9 + return { ok: false, retryAfterSec: Math.min(60, 2 ** (failures - 4)) * 60 };
10 +}
11 +export async function recordAttempt(ip: string, email: string | null, success: boolean) {
12 + await sql`INSERT INTO login_attempts (ip, email, success) VALUES (${ip}, ${email}, ${success})`;
13 + if (Math.random() < 0.05) await sql`DELETE FROM login_attempts WHERE created_at < now() - interval '2 days'`;
14 +}
added src/lib/auth/guard.ts +48 −0
@@ -0,0 +1,48 @@
1 +import "server-only";
2 +import { headers } from "next/headers";
3 +import { redirect } from "next/navigation";
4 +import { getSession, verifyCsrf, type SessionUser } from "./session";
5 +
6 +export class HttpError extends Error { constructor(public status: number, message: string) { super(message); } }
7 +
8 +/** Route handlers: 401 without session, 403 if MFA pending; mutating requests need same-origin + CSRF header. */
9 +export async function requireUser(req?: Request): Promise<SessionUser> {
10 + const s = await getSession(); if (!s) throw new HttpError(401, "Unauthenticated");
11 + if (s.totpEnabled && !s.mfaPassed) throw new HttpError(403, "MFA required");
12 + if (req && !["GET", "HEAD", "OPTIONS"].includes(req.method)) {
13 + await assertSameOrigin();
14 + if (!verifyCsrf(s, req.headers.get("x-csrf-token"))) throw new HttpError(403, "CSRF token invalid");
15 + }
16 + return s;
17 +}
18 +/** Server actions: session + same-origin + CSRF field from the form. */
19 +export async function requireAction(form: FormData): Promise<SessionUser> {
20 + const s = await getSession(); if (!s) throw new HttpError(401, "Unauthenticated");
21 + if (s.totpEnabled && !s.mfaPassed) throw new HttpError(403, "MFA required");
22 + await assertSameOrigin();
23 + if (!verifyCsrf(s, String(form.get("_csrf") ?? ""))) throw new HttpError(403, "CSRF token invalid");
24 + return s;
25 +}
26 +export async function requireUserPage(): Promise<SessionUser> {
27 + const s = await getSession(); if (!s) redirect("/login");
28 + if (s.totpEnabled && !s.mfaPassed) redirect("/login?step=totp");
29 + if (s.mustChangePassword && !((await headers()).get("x-pathname") ?? "").startsWith("/admin/account")) redirect("/admin/account?must_change=1");
30 + return s;
31 +}
32 +export async function assertSameOrigin() {
33 + const h = await headers();
34 + const sfs = h.get("sec-fetch-site");
35 + if (sfs && sfs !== "same-origin" && sfs !== "none") throw new HttpError(403, "Cross-site request blocked");
36 + const origin = h.get("origin");
37 + if (origin) {
38 + const host = h.get("x-forwarded-host") ?? h.get("host");
39 + const allowed = new Set([process.env.APP_URL?.replace(/\/$/, ""), `https://${host}`, `http://${host}`].filter(Boolean));
40 + if (!allowed.has(origin)) throw new HttpError(403, "Origin mismatch");
41 + }
42 +}
43 +export function jsonError(e: unknown) {
44 + if (e instanceof HttpError) return Response.json({ error: e.message }, { status: e.status });
45 + if (e && typeof e === "object" && "issues" in e) return Response.json({ error: "Validation failed", issues: (e as { issues: unknown }).issues }, { status: 400 });
46 + console.error(e);
47 + return Response.json({ error: "Internal error" }, { status: 500 });
48 +}
added src/lib/auth/password.ts +10 −0
@@ -0,0 +1,10 @@
1 +import { hash, verify } from "@node-rs/argon2";
2 +const OPTS = { algorithm: 2 /* Argon2id */, memoryCost: 65536, timeCost: 3, parallelism: 1, outputLen: 32 };
3 +export const hashPassword = (pw: string) => hash(pw, OPTS);
4 +export const verifyPassword = (h: string, pw: string) => verify(h, pw).catch(() => false);
5 +export function passwordPolicy(pw: string): string | null {
6 + if (pw.length < 14) return "Password must be at least 14 characters.";
7 + if (/^\d+$/.test(pw)) return "Password cannot be only digits.";
8 + if (!/[a-z]/.test(pw) || !/[A-Z]/.test(pw) || !/[0-9]/.test(pw)) return "Use upper-case and lower-case letters and a digit.";
9 + return null;
10 +}
added src/lib/auth/ratelimit.ts +9 −0
@@ -0,0 +1,9 @@
1 +/** In-memory sliding-window limiter (single-process deployment). Keys are scoped by the caller. */
2 +const buckets = new Map<string, number[]>();
3 +export function rateLimit(key: string, max: number, windowMs: number) {
4 + const now = Date.now(); const arr = (buckets.get(key) ?? []).filter((t) => now - t < windowMs);
5 + if (arr.length >= max) { buckets.set(key, arr); return false; }
6 + arr.push(now); buckets.set(key, arr);
7 + if (buckets.size > 10000) for (const [k, v] of buckets) if (!v.some((t) => now - t < windowMs)) buckets.delete(k);
8 + return true;
9 +}
added src/lib/auth/session.ts +53 −0
@@ -0,0 +1,53 @@
1 +import "server-only";
2 +import { cookies, headers } from "next/headers";
3 +import { createHash, createHmac, randomBytes, timingSafeEqual } from "node:crypto";
4 +import { cache } from "react";
5 +import { sql } from "@/db";
6 +
7 +export const COOKIE = process.env.NODE_ENV === "production" ? "__Host-wp_session" : "wp_session";
8 +const IDLE_DAYS = 7, ABS_DAYS = 30, ROTATE_HOURS = 12;
9 +const sha = (s: string) => createHash("sha256").update(s).digest("hex");
10 +export function clientIp(h: Headers) { return (h.get("x-forwarded-for")?.split(",")[0] ?? h.get("x-real-ip") ?? "0.0.0.0").trim(); }
11 +const cookieOpts = { httpOnly: true, secure: process.env.NODE_ENV === "production", sameSite: "strict" as const, path: "/" };
12 +
13 +export async function createSession(userId: string, opts: { mfaPassed?: boolean } = {}) {
14 + const token = randomBytes(32).toString("base64url"); const h = await headers(); const now = Date.now();
15 + await sql`INSERT INTO sessions (user_id, token_hash, csrf_secret, user_agent, ip, expires_at, absolute_expires_at, mfa_passed)
16 + VALUES (${userId}, ${sha(token)}, ${randomBytes(32).toString("base64url")}, ${h.get("user-agent")?.slice(0, 300) ?? null}, ${clientIp(h)},
17 + ${new Date(now + IDLE_DAYS * 864e5)}, ${new Date(now + ABS_DAYS * 864e5)}, ${opts.mfaPassed ?? false})`;
18 + return token;
19 +}
20 +export async function setSessionCookie(token: string) { (await cookies()).set(COOKIE, token, { ...cookieOpts, maxAge: ABS_DAYS * 86400 }); }
21 +export async function clearSessionCookie() { (await cookies()).set(COOKIE, "", { ...cookieOpts, maxAge: 0 }); }
22 +
23 +export type SessionUser = { id: string; email: string; displayName: string; role: string; totpEnabled: boolean; mustChangePassword: boolean; sessionId: string; mfaPassed: boolean; csrfSecret: string };
24 +
25 +export const getSession = cache(async (): Promise<SessionUser | null> => {
26 + const token = (await cookies()).get(COOKIE)?.value; if (!token) return null;
27 + const rows = await sql`SELECT s.id AS session_id, s.mfa_passed, s.csrf_secret, s.last_seen_at, u.id, u.email, u.display_name, u.role, u.totp_enabled, u.must_change_password
28 + FROM sessions s JOIN users u ON u.id = s.user_id
29 + WHERE s.token_hash = ${sha(token)} AND s.revoked_at IS NULL AND s.expires_at > now() AND s.absolute_expires_at > now() LIMIT 1`;
30 + if (!rows.length) return null;
31 + const r = rows[0];
32 + if (Date.now() - new Date(r.last_seen_at).getTime() > 5 * 60_000)
33 + sql`UPDATE sessions SET last_seen_at = now(), expires_at = now() + ${IDLE_DAYS + " days"}::interval WHERE id = ${r.session_id}`.catch(() => {});
34 + return { id: r.id, email: r.email, displayName: r.display_name, role: r.role, totpEnabled: r.totp_enabled, mustChangePassword: r.must_change_password, sessionId: r.session_id, mfaPassed: r.mfa_passed, csrfSecret: r.csrf_secret };
35 +});
36 +
37 +export async function rotateSession(sessionId: string): Promise<string> {
38 + const token = randomBytes(32).toString("base64url");
39 + await sql`UPDATE sessions SET token_hash = ${sha(token)}, rotated_at = now() WHERE id = ${sessionId}`;
40 + return token;
41 +}
42 +export async function maybeRotate(s: SessionUser) {
43 + const [row] = await sql`SELECT rotated_at FROM sessions WHERE id = ${s.sessionId}`;
44 + if (row && Date.now() - new Date(row.rotated_at).getTime() > ROTATE_HOURS * 3600e3) { const tk = await rotateSession(s.sessionId); await setSessionCookie(tk); }
45 +}
46 +export async function markMfaPassed(sessionId: string) { await sql`UPDATE sessions SET mfa_passed = true WHERE id = ${sessionId}`; }
47 +export async function revokeSession(sessionId: string) { await sql`UPDATE sessions SET revoked_at = now() WHERE id = ${sessionId}`; }
48 +export async function revokeOtherSessions(userId: string, keep: string) { await sql`UPDATE sessions SET revoked_at = now() WHERE user_id = ${userId} AND revoked_at IS NULL AND id <> ${keep}`; }
49 +
50 +export function csrfToken(s: SessionUser) { return createHmac("sha256", process.env.SESSION_SECRET ?? "dev").update(s.csrfSecret).digest("base64url"); }
51 +export function verifyCsrf(s: SessionUser, token: string | null | undefined) {
52 + if (!token) return false; const a = Buffer.from(csrfToken(s)); const b = Buffer.from(token); return a.length === b.length && timingSafeEqual(a, b);
53 +}
added src/lib/env.ts +7 −0
@@ -0,0 +1,7 @@
1 +export const APP_URL = (process.env.APP_URL ?? "http://localhost:8560").replace(/\/$/, "");
2 +export const IS_PROD = process.env.NODE_ENV === "production";
3 +export const SITE_NAME = "Research Working Papers";
4 +export const SITE_SECONDARY = "Simon-Pierre Boucher Research Observatory";
5 +export const INSTITUTION = "Université du Québec en Outaouais";
6 +export const STORAGE_DIR = process.env.STORAGE_DIR ?? `${process.env.HOME}/data/wp-spboucher-ai`;
7 +export const AI_PROVIDER = (process.env.AI_PROVIDER ?? "none") as "none" | "openai";
added src/lib/i18n/en.ts +22 −0
@@ -0,0 +1,22 @@
1 +export const en = {
2 + site: { title: "Research Working Papers", secondary: "Simon-Pierre Boucher Research Observatory", tagline: "Research, methods, and evidence at the frontier of quantitative science.", affiliation: "Université du Québec en Outaouais" },
3 + nav: { papers: "Papers", topics: "Topics", methods: "Methods", datasets: "Data", authors: "Authors", series: "Series", map: "Research map", timeline: "Timeline", search: "Search", about: "About", citation: "Cite & reuse", api: "API", admin: "Admin", menu: "Menu", close: "Close", skip: "Skip to content", theme: "Theme", language: "Language", lightDark: "Toggle light / dark" },
4 + home: { heroTitle: "Research Working Papers", heroLead: "An open collection of research in finance, economics, artificial intelligence, real estate, and quantitative science.", searchPlaceholder: "Search titles, abstracts, methods, data, full text…", explore: "Explore the papers", browseTopics: "Browse by topic", latest: "Latest working papers", latestLead: "Newly released or revised manuscripts, most recent first.", featured: "Featured research", featuredLead: "A selection chosen by the editor.", domains: "Research domains", domainsLead: "Navigate the collection by field.", stats: "The collection in numbers", statsLead: "Computed from the publication database at request time. Nothing here is estimated.", constellation: "Research constellation", constellationLead: "Papers connected by shared methods, data, topics, authors and textual similarity. Deterministic links are drawn from curated metadata; similarity links are inferred and labelled as such.", openMap: "Open the full research map", viewAll: "View all papers", newRelease: "New", revision: "Revised" },
5 + stats: { papers: "Working papers", manuscripts: "Public manuscripts", topics: "Research topics", authors: "Authors", datasets: "Datasets", methods: "Methods", latest: "Latest release", pages: "Pages", ssrn: "On SSRN", versions: "Versions archived" },
6 + papers: { title: "Working paper library", lead: "Every public working paper in the series, with full metadata, versions and citation tools.", results: "results", result: "result", noResults: "No paper matches these filters.", clear: "Clear filters", view: "View", sort: "Sort", display: "Display", filters: "Filters", list: "Academic list", compact: "Compact", cards: "Cards", table: "Table", sortRecent: "Most recent", sortRevised: "Recently revised", sortNumber: "Working paper number", sortTitle: "Title A–Z", sortYear: "Publication year", sortViews: "Most viewed", sortDownloads: "Most downloaded", year: "Year", field: "Research domain", topic: "Topic", author: "Author", keyword: "Keyword", method: "Methodology", dataset: "Dataset", status: "Paper status", pubStatus: "Publication status", fulltext: "Full text", language: "Language", series: "Series", onlyPdf: "With PDF", popularityNote: "Views = distinct daily visitors of a paper page since launch (bots excluded). Downloads = manuscript file requests, any version.", pdf: "PDF", pages: "pages", number: "No." },
7 + paper: { download: "Download PDF", read: "Read online", cite: "Cite", copyCitation: "Copy citation", bibtex: "BibTeX", ris: "RIS", share: "Share link", copied: "Copied", versions: "Version history", previousVersions: "View previous versions", abstract: "Abstract", question: "Research question", contributions: "Scientific contributions", methodology: "Methodology", data: "Research data", artifacts: "Research artifacts", related: "Related research", keywords: "Keywords", jel: "JEL codes", authors: "Authors", affiliation: "Affiliation", year: "Year", released: "First release", revised: "Latest revision", version: "Current version", status: "Status", pubStatus: "Publication status", identifier: "Working paper", series: "Series", pagesLabel: "Pages", sha: "SHA-256", doi: "DOI", ssrn: "SSRN", repository: "Repository", journal: "Journal", viewsLabel: "views", downloadsLabel: "downloads", whyRelated: "Why related:", inferred: "inferred", curated: "curated", versionTable: { version: "Version", date: "Date", description: "Description", file: "File", current: "current", withdrawn: "withdrawn" }, topics: "Topics", methodsTag: "Methods", datasetsTag: "Datasets", findings: "Key findings", reproducibility: "Reproducibility", frOnly: "This abstract is available in English only.", withdrawnNotice: "This working paper has been withdrawn. The record is kept for the integrity of the series.", supersededNotice: "This working paper has been superseded.", readerTitle: "Read online", readerFallback: "Your browser cannot display the PDF inline.", domain: "Domain", notAvailable: "Not available" },
8 + search: { title: "Advanced search", lead: "Lexical search is typo-tolerant and covers titles, abstracts, keywords, authors, methods, data and the full manuscript text. Semantic retrieval is used when embeddings are available and is labelled.", placeholder: "Search…", mode: "Mode", hybrid: "Hybrid", lexical: "Lexical", semantic: "Semantic", go: "Search", matches: "Matching passages", none: "No result.", took: "took", in: "in", fieldHit: "match in" },
9 + topics: { title: "Research topics", lead: "Domains and topics are editorial categories. Every assignment is made by the editor; inferred assignments are labelled.", papers: "papers", paper: "paper", domains: "Research domains", all: "All topics" },
10 + methods: { title: "Research methods", lead: "Econometric, computational and experimental methods used across the series, as documented in the manuscripts." },
11 + datasets: { title: "Research data catalog", lead: "Datasets used in the series with provider, coverage, frequency and access conditions as stated by the authors.", provider: "Provider", coverage: "Coverage", frequency: "Frequency", access: "Access", license: "License", public: "Public", restricted: "Restricted" },
12 + authors: { title: "Authors", lead: "Researchers contributing to the series.", papers: "papers", orcid: "ORCID", website: "Website", scholar: "Google Scholar", ssrn: "SSRN author page", github: "GitHub" },
13 + series: { title: "Research series", lead: "Each series has its own identifier scheme. Working paper numbers are permanent and never reassigned; gaps are allowed." },
14 + map: { title: "Research map", lead: "Interactive knowledge graph of the collection. Click a node to open the paper. Edge colours encode the kind of relationship; dashed edges are inferred from text similarity.", legend: "Legend", topic: "Shared topic", method: "Shared method", dataset: "Shared dataset", author: "Shared author", semantic: "Textual similarity (inferred)", curated: "Author-curated", nodes: "papers", edges: "links", reset: "Reset view", filter: "Relationship kinds" },
15 + timeline: { title: "Publication timeline", lead: "Releases and revisions over time, by month.", releases: "releases", revisions: "revisions" },
16 + about: { title: "About the observatory" },
17 + citation: { title: "Citation and reuse" },
18 + api: { title: "Public API" },
19 + footer: { rights: "All manuscripts are the intellectual property of their authors.", attribution: "Attribution", built: "Publication infrastructure operated on MacLustr.", feed: "RSS feed", oai: "OAI-PMH", sitemap: "Sitemap" },
20 + misc: { readMore: "Read more", back: "Back", loading: "Loading…", notFound: "Page not found", notFoundLead: "The page or working paper you requested does not exist. Working paper numbers are permanent; check the identifier.", home: "Home", all: "All", none: "None", yes: "Yes", no: "No", updated: "Updated", more: "more", less: "less", of: "of", page: "Page", next: "Next", previous: "Previous", status: { draft: "Draft", reserved: "Reserved", published: "Published", withdrawn: "Withdrawn", superseded: "Superseded" }, pubStatus: { working_paper: "Working paper", under_review: "Under review", forthcoming: "Forthcoming", published: "Published in a journal" } },
21 +};
22 +export type Dict = typeof en;
added src/lib/i18n/fr.ts +22 −0
@@ -0,0 +1,22 @@
1 +import type { Dict } from "./en";
2 +export const fr: Dict = {
3 + site: { title: "Cahiers de recherche", secondary: "Observatoire de recherche Simon-Pierre Boucher", tagline: "Recherche, méthodes et données probantes à la frontière des sciences quantitatives.", affiliation: "Université du Québec en Outaouais" },
4 + nav: { papers: "Cahiers", topics: "Thèmes", methods: "Méthodes", datasets: "Données", authors: "Auteurs", series: "Séries", map: "Carte de recherche", timeline: "Chronologie", search: "Recherche", about: "À propos", citation: "Citer et réutiliser", api: "API", admin: "Admin", menu: "Menu", close: "Fermer", skip: "Aller au contenu", theme: "Thème", language: "Langue", lightDark: "Basculer clair / sombre" },
5 + home: { heroTitle: "Cahiers de recherche", heroLead: "Une collection ouverte de travaux en finance, en économie, en intelligence artificielle, en immobilier et en sciences quantitatives.", searchPlaceholder: "Chercher dans les titres, résumés, méthodes, données, texte intégral…", explore: "Explorer les cahiers", browseTopics: "Parcourir par thème", latest: "Derniers cahiers", latestLead: "Manuscrits publiés ou révisés récemment, du plus récent au plus ancien.", featured: "Recherches en vedette", featuredLead: "Une sélection de l'éditeur.", domains: "Domaines de recherche", domainsLead: "Parcourir la collection par champ.", stats: "La collection en chiffres", statsLead: "Calculés à partir de la base de publications au moment de la requête. Rien ici n'est estimé.", constellation: "Constellation de recherche", constellationLead: "Cahiers reliés par des méthodes, des données, des thèmes, des auteurs et une similarité textuelle. Les liens déterministes viennent des métadonnées curatées ; les liens de similarité sont inférés et identifiés comme tels.", openMap: "Ouvrir la carte de recherche complète", viewAll: "Voir tous les cahiers", newRelease: "Nouveau", revision: "Révisé" },
6 + stats: { papers: "Cahiers de recherche", manuscripts: "Manuscrits publics", topics: "Thèmes de recherche", authors: "Auteurs", datasets: "Jeux de données", methods: "Méthodes", latest: "Dernière parution", pages: "Pages", ssrn: "Sur SSRN", versions: "Versions archivées" },
7 + papers: { title: "Bibliothèque des cahiers", lead: "Tous les cahiers publics de la série, avec métadonnées complètes, versions et outils de citation.", results: "résultats", result: "résultat", noResults: "Aucun cahier ne correspond à ces filtres.", clear: "Effacer les filtres", view: "Voir", sort: "Trier", display: "Affichage", filters: "Filtres", list: "Liste académique", compact: "Compacte", cards: "Cartes", table: "Tableau", sortRecent: "Plus récents", sortRevised: "Récemment révisés", sortNumber: "Numéro de cahier", sortTitle: "Titre A–Z", sortYear: "Année de publication", sortViews: "Plus consultés", sortDownloads: "Plus téléchargés", year: "Année", field: "Domaine", topic: "Thème", author: "Auteur", keyword: "Mot-clé", method: "Méthodologie", dataset: "Jeu de données", status: "État du cahier", pubStatus: "Statut de publication", fulltext: "Texte intégral", language: "Langue", series: "Série", onlyPdf: "Avec PDF", popularityNote: "Consultations = visiteurs quotidiens distincts d'une page de cahier depuis le lancement (robots exclus). Téléchargements = requêtes du fichier du manuscrit, toutes versions.", pdf: "PDF", pages: "pages", number: "No" },
8 + paper: { download: "Télécharger le PDF", read: "Lire en ligne", cite: "Citer", copyCitation: "Copier la citation", bibtex: "BibTeX", ris: "RIS", share: "Partager le lien", copied: "Copié", versions: "Historique des versions", previousVersions: "Voir les versions précédentes", abstract: "Résumé", question: "Question de recherche", contributions: "Contributions scientifiques", methodology: "Méthodologie", data: "Données de recherche", artifacts: "Artefacts de recherche", related: "Recherches liées", keywords: "Mots-clés", jel: "Codes JEL", authors: "Auteurs", affiliation: "Affiliation", year: "Année", released: "Première parution", revised: "Dernière révision", version: "Version courante", status: "État", pubStatus: "Statut de publication", identifier: "Cahier", series: "Série", pagesLabel: "Pages", sha: "SHA-256", doi: "DOI", ssrn: "SSRN", repository: "Dépôt", journal: "Revue", viewsLabel: "consultations", downloadsLabel: "téléchargements", whyRelated: "Pourquoi lié :", inferred: "inféré", curated: "curaté", versionTable: { version: "Version", date: "Date", description: "Description", file: "Fichier", current: "courante", withdrawn: "retirée" }, topics: "Thèmes", methodsTag: "Méthodes", datasetsTag: "Jeux de données", findings: "Résultats clés", reproducibility: "Reproductibilité", frOnly: "Ce résumé n'est disponible qu'en anglais.", withdrawnNotice: "Ce cahier a été retiré. La fiche est conservée pour l'intégrité de la série.", supersededNotice: "Ce cahier a été remplacé.", readerTitle: "Lire en ligne", readerFallback: "Votre navigateur ne peut pas afficher le PDF.", domain: "Domaine", notAvailable: "Non disponible" },
9 + search: { title: "Recherche avancée", lead: "La recherche lexicale tolère les fautes de frappe et couvre les titres, résumés, mots-clés, auteurs, méthodes, données et le texte intégral des manuscrits. La recherche sémantique est utilisée lorsque des plongements sont disponibles et est identifiée.", placeholder: "Chercher…", mode: "Mode", hybrid: "Hybride", lexical: "Lexical", semantic: "Sémantique", go: "Chercher", matches: "Passages correspondants", none: "Aucun résultat.", took: "en", in: "dans", fieldHit: "correspondance dans" },
10 + topics: { title: "Thèmes de recherche", lead: "Les domaines et thèmes sont des catégories éditoriales. Chaque attribution est faite par l'éditeur ; les attributions inférées sont identifiées.", papers: "cahiers", paper: "cahier", domains: "Domaines de recherche", all: "Tous les thèmes" },
11 + methods: { title: "Méthodes de recherche", lead: "Méthodes économétriques, computationnelles et expérimentales utilisées dans la série, telles que documentées dans les manuscrits." },
12 + datasets: { title: "Catalogue des données", lead: "Jeux de données utilisés dans la série, avec fournisseur, couverture, fréquence et conditions d'accès tels qu'énoncés par les auteurs.", provider: "Fournisseur", coverage: "Couverture", frequency: "Fréquence", access: "Accès", license: "Licence", public: "Public", restricted: "Restreint" },
13 + authors: { title: "Auteurs", lead: "Chercheuses et chercheurs contribuant à la série.", papers: "cahiers", orcid: "ORCID", website: "Site web", scholar: "Google Scholar", ssrn: "Page d'auteur SSRN", github: "GitHub" },
14 + series: { title: "Séries de recherche", lead: "Chaque série possède son propre schéma d'identifiants. Les numéros de cahier sont permanents et jamais réattribués ; les trous sont permis." },
15 + map: { title: "Carte de recherche", lead: "Graphe de connaissances interactif de la collection. Cliquez sur un nœud pour ouvrir le cahier. La couleur des liens encode le type de relation ; les liens pointillés sont inférés par similarité textuelle.", legend: "Légende", topic: "Thème partagé", method: "Méthode partagée", dataset: "Données partagées", author: "Auteur partagé", semantic: "Similarité textuelle (inférée)", curated: "Curaté par l'auteur", nodes: "cahiers", edges: "liens", reset: "Réinitialiser la vue", filter: "Types de relation" },
16 + timeline: { title: "Chronologie des publications", lead: "Parutions et révisions dans le temps, par mois.", releases: "parutions", revisions: "révisions" },
17 + about: { title: "À propos de l'observatoire" },
18 + citation: { title: "Citation et réutilisation" },
19 + api: { title: "API publique" },
20 + footer: { rights: "Tous les manuscrits sont la propriété intellectuelle de leurs auteurs.", attribution: "Attribution", built: "Infrastructure de publication opérée sur MacLustr.", feed: "Fil RSS", oai: "OAI-PMH", sitemap: "Plan du site" },
21 + misc: { readMore: "Lire la suite", back: "Retour", loading: "Chargement…", notFound: "Page introuvable", notFoundLead: "La page ou le cahier demandé n'existe pas. Les numéros de cahier sont permanents ; vérifiez l'identifiant.", home: "Accueil", all: "Tous", none: "Aucun", yes: "Oui", no: "Non", updated: "Mis à jour", more: "plus", less: "moins", of: "de", page: "Page", next: "Suivant", previous: "Précédent", status: { draft: "Brouillon", reserved: "Réservé", published: "Publié", withdrawn: "Retiré", superseded: "Remplacé" }, pubStatus: { working_paper: "Cahier de recherche", under_review: "En évaluation", forthcoming: "À paraître", published: "Publié dans une revue" } },
22 +};
added src/lib/i18n/index.ts +32 −0
@@ -0,0 +1,32 @@
1 +import { en, type Dict } from "./en";
2 +import { fr } from "./fr";
3 +export type Locale = "en" | "fr";
4 +export const LOCALES: Locale[] = ["en", "fr"];
5 +export const dict: Record<Locale, Dict> = { en, fr };
6 +export function t(locale: Locale): Dict { return dict[locale] ?? en; }
7 +/** Prefix a canonical path with the locale segment (FR only; EN is the canonical, unprefixed form). */
8 +export function href(locale: Locale, path: string): string {
9 + if (locale === "en") return path;
10 + if (path.startsWith("/api/") || path.startsWith("/admin") || path.startsWith("/login") || path.startsWith("http")) return path;
11 + return path === "/" ? "/fr" : `/fr${path}`;
12 +}
13 +export function pickLocale(v: string | null | undefined): Locale { return v === "fr" ? "fr" : "en"; }
14 +const MONTHS: Record<Locale, string[]> = {
15 + en: ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"],
16 + fr: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"],
17 +};
18 +/** Format a date with its declared precision (day / month / year) — never invents a day that the source did not state. */
19 +export function fmtDate(locale: Locale, d: string | Date | null | undefined, precision: "day" | "month" | "year" = "month"): string {
20 + if (!d) return "";
21 + const date = typeof d === "string" ? new Date(d.length === 10 ? d + "T12:00:00Z" : d) : d;
22 + if (Number.isNaN(date.getTime())) return "";
23 + const y = date.getUTCFullYear(), m = date.getUTCMonth(), day = date.getUTCDate();
24 + if (precision === "year") return String(y);
25 + if (precision === "month") return locale === "fr" ? `${MONTHS.fr[m]} ${y}` : `${MONTHS.en[m]} ${y}`;
26 + return locale === "fr" ? `${day} ${MONTHS.fr[m]} ${y}` : `${MONTHS.en[m]} ${day}, ${y}`;
27 +}
28 +export function fmtInt(locale: Locale, n: number): string { return new Intl.NumberFormat(locale === "fr" ? "fr-CA" : "en-CA").format(n); }
29 +export function pick<T extends Record<string, unknown>>(row: T, base: string, locale: Locale): string {
30 + const frv = row[`${base}_fr`] as string | null | undefined; const env = row[`${base}_en`] as string | null | undefined;
31 + return (locale === "fr" && frv) ? frv : (env ?? (row[base] as string) ?? "");
32 +}
added src/lib/i18n/server.ts +6 −0
@@ -0,0 +1,6 @@
1 +import "server-only";
2 +import { headers } from "next/headers";
3 +import { cache } from "react";
4 +import { pickLocale, t, type Locale } from "./index";
5 +export const getLocale = cache(async (): Promise<Locale> => pickLocale((await headers()).get("x-locale")));
6 +export async function getT() { const locale = await getLocale(); return { locale, t: t(locale) }; }
added src/lib/jobs/backup.ts +37 −0
@@ -0,0 +1,37 @@
1 +import { execFile } from "node:child_process";
2 +import { promisify } from "node:util";
3 +import { promises as fs } from "node:fs";
4 +import { join } from "node:path";
5 +import { createHash } from "node:crypto";
6 +import { sql } from "@/db";
7 +import { STORAGE_DIR } from "@/lib/env";
8 +const run = promisify(execFile);
9 +/** Nightly backup: pg_dump (custom format) + tar of the manuscript store, encrypted with AES-256 (openssl, pbkdf2), 14-day retention, optional rsync to a second node. */
10 +export async function runBackup() {
11 + const dir = process.env.BACKUP_DIR ?? join(process.env.HOME ?? "/tmp", "backups", "wp-spboucher-ai");
12 + await fs.mkdir(dir, { recursive: true, mode: 0o700 });
13 + const stamp = new Date().toISOString().replace(/[-:]/g, "").slice(0, 13);
14 + const pgBin = process.env.PG_BIN ?? "/opt/homebrew/opt/postgresql@17/bin";
15 + const adminUrl = process.env.DATABASE_URL_ADMIN ?? process.env.DATABASE_URL!;
16 + const pass = process.env.BACKUP_PASSPHRASE; if (!pass) throw new Error("BACKUP_PASSPHRASE missing");
17 + const log: string[] = []; let ok = true; let remoteOk: boolean | null = null;
18 + const dump = join(dir, `wp-${stamp}.dump`); const tar = join(dir, `wp-files-${stamp}.tar`);
19 + try {
20 + await run(join(pgBin, "pg_dump"), ["-Fc", "-f", dump, adminUrl], { maxBuffer: 1 << 24 }); log.push("pg_dump ok");
21 + await run("tar", ["-cf", tar, "-C", STORAGE_DIR, "files", "previews"], { maxBuffer: 1 << 24 }); log.push("tar ok");
22 + for (const f of [dump, tar]) { await run("openssl", ["enc", "-aes-256-cbc", "-pbkdf2", "-salt", "-in", f, "-out", f + ".enc", "-pass", "env:WP_BK"], { env: { ...process.env, WP_BK: pass } }); await fs.unlink(f); }
23 + log.push("encrypted");
24 + } catch (e) { ok = false; log.push("ERROR " + (e as Error).message.slice(0, 500)); }
25 + const outs = [dump + ".enc", tar + ".enc"]; let bytes = 0; const sums: string[] = [];
26 + for (const f of outs) { try { const b = await fs.readFile(f); bytes += b.length; sums.push(createHash("sha256").update(b).digest("hex")); } catch { /* missing on failure */ } }
27 + // Retention: 14 days.
28 + for (const n of await fs.readdir(/*turbopackIgnore: true*/ dir)) { const p = join(dir, n); const st = await fs.stat(p); if (st.isFile() && Date.now() - st.mtimeMs > 14 * 864e5) await fs.unlink(p); }
29 + if (ok && process.env.BACKUP_REMOTE) {
30 + try { await run("rsync", ["-az", "--delete", "-e", "ssh -o BatchMode=yes -o ConnectTimeout=15", dir + "/", process.env.BACKUP_REMOTE + "/"], { maxBuffer: 1 << 22 }); remoteOk = true; log.push("rsync ok → " + process.env.BACKUP_REMOTE); }
31 + catch (e) { remoteOk = false; log.push("rsync failed: " + (e as Error).message.slice(0, 300)); }
32 + }
33 + await sql`INSERT INTO backups (kind, path, bytes, sha256, ok, log, remote_ok) VALUES ('nightly', ${outs.join(";")}, ${bytes}, ${sums.join(";")}, ${ok}, ${log.join("\n")}, ${remoteOk})`;
34 + await sql`DELETE FROM backups WHERE created_at < now() - interval '120 days'`;
35 + if (!ok) throw new Error(log.join(" | "));
36 + return { ok, bytes, files: outs, remoteOk };
37 +}
added src/lib/jobs/handlers.ts +156 −0
@@ -0,0 +1,156 @@
1 +import { promises as fs } from "node:fs";
2 +import { join, basename } from "node:path";
3 +import { sql } from "@/db";
4 +import { STORAGE_DIR } from "@/lib/env";
5 +import { pdfInfo, pdfIntegrity, pdfText, pdfTextPages, pdfThumbnail, sha256File, storeFile, storagePath, ensureStorage } from "@/lib/pdf/process";
6 +import { proposeMetadata, titleSimilarity } from "@/lib/pdf/extract";
7 +import { chunkPages } from "@/lib/pdf/chunk";
8 +import { lexvec } from "@/lib/ai/lexvec";
9 +import { embed, embeddingsEnabled, EMBEDDING_MODEL } from "@/lib/ai/embeddings";
10 +import { enqueue } from "./queue";
11 +import { runBackup } from "./backup";
12 +import { rebuildRelations } from "./relations";
13 +
14 +const vec = (v: number[]) => `[${v.map((x) => x.toFixed(6)).join(",")}]`;
15 +
16 +export async function processPdf(p: { fileId: string; paperId?: string; versionId?: string }) {
17 + const [f] = await sql`SELECT * FROM files WHERE id = ${p.fileId}`; if (!f) throw new Error("file not found");
18 + const path = storagePath(f.storage_key);
19 + const integ = await pdfIntegrity(path);
20 + const info = integ.info ?? (integ.ok ? await pdfInfo(path) : null);
21 + await sql`UPDATE files SET integrity_ok = ${integ.ok}, pages = ${info?.pages ?? null}, pdf_meta = ${info ? sql.json(info.raw as never) : null} WHERE id = ${p.fileId}`;
22 + if (!integ.ok) { await sql`UPDATE files SET text_status = 'failed' WHERE id = ${p.fileId}`; throw new Error(`integrity: ${integ.reason}`); }
23 + // Preview (first page) — content-addressed by sha.
24 + const previewBase = join(/*turbopackIgnore: true*/ STORAGE_DIR, "previews", f.sha256);
25 + try { await fs.access(previewBase + ".png"); } catch { await pdfThumbnail(path, previewBase, 60); }
26 + if (p.versionId) await sql`UPDATE paper_versions SET pages = coalesce(pages, ${info?.pages ?? null}), sha256 = ${f.sha256}, bytes = ${Number(f.bytes)} WHERE id = ${p.versionId}`;
27 + // Full text → chunks (only for the current version of a paper).
28 + if (p.paperId) {
29 + const pages = await pdfTextPages(path);
30 + const chunks = chunkPages(pages);
31 + await sql.begin(async (tx) => {
32 + await tx`DELETE FROM paper_chunks WHERE paper_id = ${p.paperId!} AND (version_id = ${p.versionId ?? null} OR version_id IS NULL)`;
33 + for (let i = 0; i < chunks.length; i += 200) {
34 + const batch = chunks.slice(i, i + 200);
35 + await tx`INSERT INTO paper_chunks ${tx(batch.map((c) => ({ paper_id: p.paperId!, version_id: p.versionId ?? null, chunk_index: c.index, page_from: c.pageFrom, page_to: c.pageTo, text: c.text })))}`;
36 + }
37 + });
38 + const [paper] = await sql`SELECT title, subtitle, abstract, keywords FROM papers WHERE id = ${p.paperId}`;
39 + const base = `${paper.title} ${paper.subtitle ?? ""} ${(paper.keywords ?? []).join(" ")} ${(paper.abstract as string[]).join(" ")}`;
40 + const lv = lexvec(base + " " + chunks.map((c) => c.text).join(" ").slice(0, 60000));
41 + await sql`UPDATE papers SET lexvec = ${vec(lv)}::vector WHERE id = ${p.paperId}`;
42 + await sql`UPDATE files SET text_status = 'done' WHERE id = ${p.fileId}`;
43 + if (embeddingsEnabled()) await enqueue("embed_paper", { paperId: p.paperId }, { dedupe: true });
44 + await enqueue("relations", {}, { dedupe: true, runAfter: new Date(Date.now() + 30_000) });
45 + } else await sql`UPDATE files SET text_status = 'skipped' WHERE id = ${p.fileId}`;
46 +}
47 +
48 +/** Paper-level lexvec from metadata only (no manuscript yet). */
49 +export async function lexvecFromMetadata(paperId: string) {
50 + const [paper] = await sql`SELECT title, subtitle, abstract, keywords, research_question, contributions, methodology, data_sections FROM papers WHERE id = ${paperId}`;
51 + if (!paper) return;
52 + const text = [paper.title, paper.subtitle, (paper.keywords ?? []).join(" "), ...(paper.abstract as string[]), paper.research_question, JSON.stringify(paper.contributions), JSON.stringify(paper.methodology), JSON.stringify(paper.data_sections)].filter(Boolean).join(" ");
53 + await sql`UPDATE papers SET lexvec = ${vec(lexvec(text))}::vector WHERE id = ${paperId}`;
54 +}
55 +
56 +export async function embedPaper(p: { paperId: string }) {
57 + const [paper] = await sql`SELECT id, title, subtitle, abstract, keywords FROM papers WHERE id = ${p.paperId}`; if (!paper) return;
58 + const head = `${paper.title}${paper.subtitle ? ": " + paper.subtitle : ""}\n\n${(paper.abstract as string[]).join("\n\n")}\n\nKeywords: ${(paper.keywords ?? []).join(", ")}`;
59 + const e = await embed([head]); if (!e) return;
60 + await sql`UPDATE papers SET embedding = ${vec(e[0])}::vector, embedding_model = ${EMBEDDING_MODEL} WHERE id = ${p.paperId}`;
61 + const chunks = await sql`SELECT id, text FROM paper_chunks WHERE paper_id = ${p.paperId} AND embedding IS NULL ORDER BY chunk_index`;
62 + for (let i = 0; i < chunks.length; i += 64) {
63 + const batch = chunks.slice(i, i + 64); const vs = await embed(batch.map((c) => c.text)); if (!vs) return;
64 + await sql.begin(async (tx) => { for (let k = 0; k < batch.length; k++) await tx`UPDATE paper_chunks SET embedding = ${vec(vs[k])}::vector WHERE id = ${batch[k].id}`; });
65 + }
66 + await enqueue("relations", {}, { dedupe: true, runAfter: new Date(Date.now() + 30_000) });
67 +}
68 +
69 +export async function dailyStats() {
70 + await sql`INSERT INTO paper_stats_daily (paper_id, day, views, downloads)
71 + SELECT paper_id, day, count(*) FILTER (WHERE kind = 'view'), count(*) FILTER (WHERE kind = 'download')
72 + FROM events WHERE NOT is_bot AND day >= current_date - 2 GROUP BY paper_id, day
73 + ON CONFLICT (paper_id, day) DO UPDATE SET views = EXCLUDED.views, downloads = EXCLUDED.downloads`;
74 + await sql`UPDATE papers p SET view_count = s.v, download_count = s.d FROM (SELECT paper_id, sum(views) v, sum(downloads) d FROM paper_stats_daily GROUP BY paper_id) s WHERE s.paper_id = p.id AND (p.view_count <> s.v OR p.download_count <> s.d)`;
75 + await sql`DELETE FROM events WHERE day < current_date - 400`;
76 + await sql`DELETE FROM search_log WHERE at < now() - interval '180 days'`;
77 + await sql`DELETE FROM jobs WHERE status IN ('done', 'failed') AND finished_at < now() - interval '30 days'`;
78 +}
79 +
80 +export async function ingestScan(p: { dirs?: string[] } = {}) {
81 + await ensureStorage();
82 + const [extra] = await sql`SELECT value FROM settings WHERE key = 'ingest.extra_dirs'`;
83 + const dirs = [join(STORAGE_DIR, "inbox"), ...(p.dirs ?? []), ...((extra?.value as string[] | undefined) ?? [])];
84 + let found = 0, queued = 0;
85 + for (const dir of dirs) {
86 + let entries: string[] = [];
87 + try { entries = (await fs.readdir(dir)).filter((n) => n.toLowerCase().endsWith(".pdf")); } catch { continue; }
88 + for (const name of entries) {
89 + found++;
90 + const src = join(dir, name); const sha = await sha256File(src);
91 + const [known] = await sql`SELECT 1 FROM ingest_candidates WHERE sha256 = ${sha} LIMIT 1`;
92 + if (known) continue;
93 + const [existing] = await sql`SELECT f.id FROM files f WHERE f.sha256 = ${sha} LIMIT 1`;
94 + let fileId: string;
95 + if (existing) fileId = existing.id;
96 + else {
97 + const st = await fs.stat(src); const key = await storeFile(src, sha);
98 + const [f] = await sql`INSERT INTO files (kind, storage_key, original_name, mime, bytes, sha256) VALUES ('manuscript', ${key}, ${basename(src)}, 'application/pdf', ${st.size}, ${sha}) RETURNING id`;
99 + fileId = f.id;
100 + }
101 + const [c] = await sql`INSERT INTO ingest_candidates (source_path, file_id, sha256, status) VALUES (${src}, ${fileId}, ${sha}, 'queued') ON CONFLICT DO NOTHING RETURNING id`;
102 + if (c) { queued++; await enqueue("ingest_candidate", { candidateId: c.id }); }
103 + }
104 + }
105 + await sql`INSERT INTO settings (key, value) VALUES ('ingest.last_scan', ${sql.json({ at: new Date().toISOString(), dirs, found, queued } as never)}) ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, updated_at = now()`;
106 + return { found, queued };
107 +}
108 +
109 +export async function ingestCandidate(p: { candidateId: string }) {
110 + const [c] = await sql`SELECT c.*, f.storage_key FROM ingest_candidates c JOIN files f ON f.id = c.file_id WHERE c.id = ${p.candidateId}`; if (!c) return;
111 + await sql`UPDATE ingest_candidates SET status = 'processing' WHERE id = ${c.id}`;
112 + const path = storagePath(c.storage_key);
113 + const flags: { code: string; message: string }[] = [];
114 + const integ = await pdfIntegrity(path);
115 + if (!integ.ok) flags.push({ code: "integrity", message: integ.reason ?? "PDF failed integrity check" });
116 + const info = integ.info ?? null;
117 + await sql`UPDATE files SET integrity_ok = ${integ.ok}, pages = ${info?.pages ?? null}, pdf_meta = ${info ? sql.json(info.raw as never) : null} WHERE id = ${c.file_id}`;
118 + let proposal: Record<string, unknown> = {};
119 + if (integ.ok) {
120 + const first = await pdfText(path, { first: 1, last: 3 });
121 + const prop = proposeMetadata(first, info?.title, info?.author);
122 + proposal = { ...prop, pages: info?.pages ?? null, pdfMeta: { title: info?.title, author: info?.author, subject: info?.subject, keywords: info?.keywords, creationDate: info?.creationDate, modDate: info?.modDate, producer: info?.producer }, sourceFile: basename(c.source_path), sha256: c.sha256 };
123 + if (!prop.title) flags.push({ code: "title", message: "No title could be proposed" });
124 + if (!prop.authors.length) flags.push({ code: "authors", message: "No author line detected" });
125 + if (!prop.abstract) flags.push({ code: "abstract", message: "No abstract detected" });
126 + if (!prop.wp) flags.push({ code: "identifier", message: "No working paper number detected on the title page" });
127 + if (!prop.dateText) flags.push({ code: "date", message: "No release date detected; do not use the file timestamp" });
128 + if (first.trim().length < 200) flags.push({ code: "scanned", message: "Little or no extractable text (scanned PDF?)" });
129 + try { await pdfThumbnail(path, join(STORAGE_DIR, "previews", c.sha256), 60); } catch { flags.push({ code: "preview", message: "Preview generation failed" }); }
130 + // Duplicate detection: checksum, identifier, title similarity, author overlap.
131 + const dups: { paperId: string; identifier: string; title: string; reasons: string[]; score: number }[] = [];
132 + const papers = await sql`SELECT p.id, p.identifier, p.title, p.number, s.code, (SELECT array_agg(a.display_name) FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id) AS authors,
133 + EXISTS (SELECT 1 FROM paper_versions v WHERE v.paper_id = p.id AND v.sha256 = ${c.sha256}) AS same_file FROM papers p JOIN series s ON s.id = p.series_id`;
134 + for (const pr of papers) {
135 + const reasons: string[] = []; let score = 0;
136 + if (pr.same_file) { reasons.push("identical checksum"); score += 1; }
137 + if (prop.wp && pr.code === prop.wp.seriesCode && pr.number === prop.wp.number) { reasons.push(`same identifier ${pr.identifier}`); score += 0.6; }
138 + const ts = prop.title ? titleSimilarity(prop.title, pr.title) : 0; if (ts >= 0.6) { reasons.push(`title similarity ${(ts * 100).toFixed(0)}%`); score += ts * 0.5; }
139 + const ov = prop.authors.filter((a) => (pr.authors ?? []).some((b: string) => titleSimilarity(a, b) >= 0.5)).length; if (ov && ts >= 0.3) { reasons.push(`${ov} shared author(s)`); score += 0.1 * ov; }
140 + if (reasons.length) dups.push({ paperId: pr.id, identifier: pr.identifier, title: pr.title, reasons, score: Math.min(1, score) });
141 + }
142 + dups.sort((a, b) => b.score - a.score);
143 + if (dups.length) flags.push({ code: "duplicate", message: `Possible duplicate of ${dups[0].identifier} (${dups[0].reasons.join(", ")})` });
144 + await sql`UPDATE ingest_candidates SET proposal = ${sql.json(proposal as never)}, flags = ${sql.json(flags as never)}, duplicates = ${sql.json(dups.slice(0, 5) as never)}, status = 'ready' WHERE id = ${c.id}`;
145 + } else await sql`UPDATE ingest_candidates SET flags = ${sql.json(flags as never)}, status = 'ready' WHERE id = ${c.id}`;
146 +}
147 +
148 +export const handlers: Record<string, (payload: Record<string, unknown>) => Promise<unknown>> = {
149 + process_pdf: (p) => processPdf(p as { fileId: string; paperId?: string; versionId?: string }),
150 + embed_paper: (p) => embedPaper(p as { paperId: string }),
151 + relations: () => rebuildRelations(),
152 + daily_stats: () => dailyStats(),
153 + backup: () => runBackup(),
154 + ingest_scan: (p) => ingestScan(p as { dirs?: string[] }),
155 + ingest_candidate: (p) => ingestCandidate(p as { candidateId: string }),
156 +};
added src/lib/jobs/queue.ts +23 −0
@@ -0,0 +1,23 @@
1 +import { sql } from "@/db";
2 +export type JobKind = "process_pdf" | "embed_paper" | "relations" | "daily_stats" | "backup" | "ingest_scan" | "ingest_candidate";
3 +export async function enqueue(kind: JobKind, payload: Record<string, unknown> = {}, opts: { runAfter?: Date; dedupe?: boolean } = {}) {
4 + if (opts.dedupe) {
5 + const [dup] = await sql`SELECT id FROM jobs WHERE kind = ${kind} AND payload = ${sql.json(payload as never)} AND status IN ('queued', 'running') LIMIT 1`;
6 + if (dup) return Number(dup.id);
7 + }
8 + const [r] = await sql`INSERT INTO jobs (kind, payload, run_after) VALUES (${kind}, ${sql.json(payload as never)}, ${opts.runAfter ?? new Date()}) RETURNING id`;
9 + return Number(r.id);
10 +}
11 +export async function claimJob(): Promise<{ id: number; kind: JobKind; payload: Record<string, unknown>; attempts: number } | null> {
12 + const rows = await sql`UPDATE jobs SET status = 'running', locked_at = now(), attempts = attempts + 1
13 + WHERE id = (SELECT id FROM jobs WHERE (status = 'queued' AND run_after <= now()) OR (status = 'running' AND locked_at < now() - interval '30 minutes') ORDER BY run_after LIMIT 1 FOR UPDATE SKIP LOCKED)
14 + RETURNING id, kind, payload, attempts`;
15 + if (!rows.length) return null; const r = rows[0];
16 + return { id: Number(r.id), kind: r.kind, payload: r.payload, attempts: r.attempts };
17 +}
18 +export async function finishJob(id: number, ok: boolean, error?: string, retryable = true) {
19 + if (ok) { await sql`UPDATE jobs SET status = 'done', finished_at = now(), last_error = NULL WHERE id = ${id}`; return; }
20 + const [j] = await sql`SELECT attempts, max_attempts FROM jobs WHERE id = ${id}`;
21 + if (retryable && j && j.attempts < j.max_attempts) await sql`UPDATE jobs SET status = 'queued', run_after = now() + (${j.attempts} * interval '2 minutes'), last_error = ${error ?? null} WHERE id = ${id}`;
22 + else await sql`UPDATE jobs SET status = 'failed', finished_at = now(), last_error = ${error ?? null} WHERE id = ${id}`;
23 +}
added src/lib/jobs/relations.ts +41 −0
@@ -0,0 +1,41 @@
1 +import { sql } from "@/db";
2 +/**
3 + * Deterministic relations from curated metadata (topics, methods, datasets, co-authors other than the series' primary author)
4 + * plus inferred links from textual similarity (lexical hashed vectors; OpenAI embeddings when available). Curated relations are never touched.
5 + */
6 +export async function rebuildRelations() {
7 + const papers = await sql`SELECT id, identifier FROM papers WHERE status = 'published'`;
8 + const ids = papers.map((p) => p.id as string);
9 + const rows: { from: string; to: string; kind: string; weight: number; en: string; fr: string; inferred: boolean }[] = [];
10 + const shared = async (table: string, col: string, nameTable: string, kind: string, labelEn: string, labelFr: string, weightEach: number, extra = "") => {
11 + const r = await sql.unsafe(`SELECT a.paper_id AS f, b.paper_id AS t, array_agg(n.name_en ORDER BY n.name_en) AS en, array_agg(n.name_fr ORDER BY n.name_en) AS fr
12 + FROM ${table} a JOIN ${table} b ON a.${col} = b.${col} AND a.paper_id <> b.paper_id JOIN ${nameTable} n ON n.id = a.${col}
13 + JOIN papers pa ON pa.id = a.paper_id AND pa.status = 'published' JOIN papers pb ON pb.id = b.paper_id AND pb.status = 'published' ${extra} GROUP BY a.paper_id, b.paper_id`);
14 + for (const x of r) rows.push({ from: x.f, to: x.t, kind, weight: Math.min(1, weightEach * (x.en as string[]).length), en: `${labelEn}: ${(x.en as string[]).join(", ")}`, fr: `${labelFr} : ${(x.fr as string[]).join(", ")}`, inferred: false });
15 + };
16 + await shared("paper_topics", "topic_id", "topics", "topic", "Shared topics", "Thèmes partagés", 0.35, "AND n.kind = 'topic'");
17 + await shared("paper_methods", "method_id", "methods", "method", "Shared methods", "Méthodes partagées", 0.4);
18 + await shared("paper_datasets", "dataset_id", "datasets", "dataset", "Shared datasets", "Données partagées", 0.5);
19 + // Co-authors: the observatory's primary author is on (nearly) every paper, so only non-primary shared authors create a link.
20 + const au = await sql`SELECT a.paper_id AS f, b.paper_id AS t, array_agg(n.display_name ORDER BY n.display_name) AS names FROM paper_authors a JOIN paper_authors b ON a.author_id = b.author_id AND a.paper_id <> b.paper_id JOIN authors n ON n.id = a.author_id AND NOT n.is_primary
21 + JOIN papers pa ON pa.id = a.paper_id AND pa.status = 'published' JOIN papers pb ON pb.id = b.paper_id AND pb.status = 'published' GROUP BY a.paper_id, b.paper_id`;
22 + for (const x of au) rows.push({ from: x.f, to: x.t, kind: "author", weight: 0.6, en: `Shared co-authors: ${(x.names as string[]).join(", ")}`, fr: `Co-auteurs partagés : ${(x.names as string[]).join(", ")}`, inferred: false });
23 + // Lexical similarity (always available) — top 4 neighbours above threshold.
24 + const lex = await sql`SELECT a.id AS f, b.id AS t, 1 - (a.lexvec <=> b.lexvec) AS sim FROM papers a JOIN papers b ON a.id <> b.id WHERE a.status = 'published' AND b.status = 'published' AND a.lexvec IS NOT NULL AND b.lexvec IS NOT NULL`;
25 + const topk = (list: { f: string; t: string; sim: number }[], k: number, thr: number) => {
26 + const by = new Map<string, { f: string; t: string; sim: number }[]>();
27 + for (const x of list) { if (x.sim < thr) continue; const arr = by.get(x.f) ?? []; arr.push(x); by.set(x.f, arr); }
28 + const out: { f: string; t: string; sim: number }[] = []; for (const arr of by.values()) out.push(...arr.sort((a, b) => b.sim - a.sim).slice(0, k)); return out;
29 + };
30 + for (const x of topk(lex.map((r) => ({ f: r.f, t: r.t, sim: Number(r.sim) })), 4, 0.22)) rows.push({ from: x.f, to: x.t, kind: "lexical", weight: Number(x.sim.toFixed(3)), en: `Textual similarity ${(x.sim * 100).toFixed(0)}% (inferred from vocabulary of title, abstract and manuscript)`, fr: `Similarité textuelle ${(x.sim * 100).toFixed(0)} % (inférée du vocabulaire du titre, du résumé et du manuscrit)`, inferred: true });
31 + const sem = await sql`SELECT a.id AS f, b.id AS t, 1 - (a.embedding <=> b.embedding) AS sim FROM papers a JOIN papers b ON a.id <> b.id WHERE a.status = 'published' AND b.status = 'published' AND a.embedding IS NOT NULL AND b.embedding IS NOT NULL`;
32 + for (const x of topk(sem.map((r) => ({ f: r.f, t: r.t, sim: Number(r.sim) })), 4, 0.45)) rows.push({ from: x.f, to: x.t, kind: "semantic", weight: Number(x.sim.toFixed(3)), en: `Semantic similarity ${(x.sim * 100).toFixed(0)}% (inferred from embeddings of title and abstract)`, fr: `Similarité sémantique ${(x.sim * 100).toFixed(0)} % (inférée des plongements du titre et du résumé)`, inferred: true });
33 + await sql.begin(async (tx) => {
34 + await tx`DELETE FROM paper_relations WHERE kind <> 'curated'`;
35 + for (let i = 0; i < rows.length; i += 500) {
36 + const batch = rows.slice(i, i + 500).filter((r) => ids.includes(r.from) && ids.includes(r.to));
37 + if (batch.length) await tx`INSERT INTO paper_relations ${tx(batch.map((r) => ({ from_paper_id: r.from, to_paper_id: r.to, kind: r.kind, weight: r.weight, explanation: r.en, explanation_fr: r.fr, inferred: r.inferred })))} ON CONFLICT (from_paper_id, to_paper_id, kind) DO UPDATE SET weight = EXCLUDED.weight, explanation = EXCLUDED.explanation, explanation_fr = EXCLUDED.explanation_fr`;
38 + }
39 + });
40 + return { relations: rows.length, papers: ids.length };
41 +}
added src/lib/papers/citation.ts +58 −0
@@ -0,0 +1,58 @@
1 +export type CitablePaper = {
2 + identifier: string; slug: string; title: string; subtitle?: string | null; year?: number | null;
3 + authors: { givenName: string; familyName: string }[];
4 + seriesName: string; institution: string; url: string; versionLabel?: string | null;
5 + doi?: string | null; ssrnId?: string | null; keywords?: string[]; abstract?: string[]; releasedOn?: string | null; releasedPrecision?: string | null; pages?: number | null; jel?: string[];
6 +};
7 +const esc = (s: string) => s.replace(/[{}]/g, "").replace(/&/g, "\\&").replace(/%/g, "\\%");
8 +function monthOf(d?: string | null) { if (!d) return null; const m = Number(d.slice(5, 7)); return Number.isFinite(m) && m >= 1 && m <= 12 ? m : null; }
9 +const MONTHS = ["jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec"];
10 +function fullTitle(p: CitablePaper) { return p.subtitle ? `${p.title}: ${p.subtitle}` : p.title; }
11 +function authorsApa(a: CitablePaper["authors"]) {
12 + const parts = a.map((x) => `${x.familyName}, ${x.givenName.split(/[\s-]+/).map((g) => g[0] + ".").join("-")}`);
13 + if (parts.length <= 1) return parts[0] ?? "";
14 + if (parts.length === 2) return `${parts[0]}, & ${parts[1]}`;
15 + return parts.slice(0, -1).join(", ") + ", & " + parts[parts.length - 1];
16 +}
17 +function authorsChicago(a: CitablePaper["authors"]) {
18 + const first = a[0] ? `${a[0].familyName}, ${a[0].givenName}` : ""; const rest = a.slice(1).map((x) => `${x.givenName} ${x.familyName}`);
19 + if (!rest.length) return first; if (rest.length === 1) return `${first}, and ${rest[0]}`; return `${first}, ${rest.slice(0, -1).join(", ")}, and ${rest[rest.length - 1]}`;
20 +}
21 +export function citationApa(p: CitablePaper) {
22 + const y = p.year ?? "n.d.";
23 + return `${authorsApa(p.authors)} (${y}). ${fullTitle(p)} (${p.seriesName} ${p.identifier}${p.versionLabel ? ", " + p.versionLabel : ""}). ${p.institution}. ${p.doi ? `https://doi.org/${p.doi}` : p.url}`;
24 +}
25 +export function citationChicago(p: CitablePaper) {
26 + const y = p.year ?? "n.d.";
27 + return `${authorsChicago(p.authors)}. ${y}. “${fullTitle(p)}.” ${p.seriesName} ${p.identifier}. ${p.institution}. ${p.doi ? `https://doi.org/${p.doi}` : p.url}.`;
28 +}
29 +export function citationKey(p: CitablePaper) {
30 + const fam = (p.authors[0]?.familyName ?? "anon").normalize("NFD").replace(/[\u0300-\u036f]/g, "").replace(/[^A-Za-z]/g, "").toLowerCase();
31 + return `${fam}${p.year ?? ""}${p.slug}`;
32 +}
33 +export function bibtex(p: CitablePaper) {
34 + const m = monthOf(p.releasedOn); const lines = [
35 + `@techreport{${citationKey(p)},`,
36 + ` title = {${esc(fullTitle(p))}},`,
37 + ` author = {${p.authors.map((a) => `${esc(a.familyName)}, ${esc(a.givenName)}`).join(" and ")}},`,
38 + p.year ? ` year = {${p.year}},` : null,
39 + m && p.releasedPrecision !== "year" ? ` month = ${MONTHS[m - 1]},` : null,
40 + ` institution = {${esc(p.institution)}},`,
41 + ` type = {${esc(p.seriesName)}},`,
42 + ` number = {${p.identifier}},`,
43 + p.versionLabel ? ` note = {Version ${p.versionLabel}},` : null,
44 + p.doi ? ` doi = {${p.doi}},` : null,
45 + ` url = {${p.url}},`,
46 + p.keywords?.length ? ` keywords = {${esc(p.keywords.join(", "))}},` : null,
47 + p.abstract?.length ? ` abstract = {${esc(p.abstract.join(" "))}},` : null,
48 + `}`,
49 + ].filter(Boolean);
50 + return lines.join("\n");
51 +}
52 +export function ris(p: CitablePaper) {
53 + const lines = ["TY - RPRT", `TI - ${fullTitle(p)}`, ...p.authors.map((a) => `AU - ${a.familyName}, ${a.givenName}`), p.year ? `PY - ${p.year}` : null,
54 + p.releasedOn ? `DA - ${p.releasedOn.slice(0, 4)}/${p.releasedPrecision === "year" ? "" : p.releasedOn.slice(5, 7)}/${p.releasedPrecision === "day" ? p.releasedOn.slice(8, 10) : ""}/` : null,
55 + `PB - ${p.institution}`, `T2 - ${p.seriesName}`, `M1 - ${p.identifier}`, p.versionLabel ? `ET - ${p.versionLabel}` : null,
56 + p.doi ? `DO - ${p.doi}` : null, `UR - ${p.url}`, ...(p.keywords ?? []).map((k) => `KW - ${k}`), p.abstract?.length ? `AB - ${p.abstract.join(" ")}` : null, "ER - "].filter(Boolean);
57 + return lines.join("\r\n");
58 +}
added src/lib/papers/events.ts +14 −0
@@ -0,0 +1,14 @@
1 +import "server-only";
2 +import { createHash } from "node:crypto";
3 +import { sqlPublic as sql } from "@/db";
4 +const BOT = /bot|crawl|spider|slurp|facebookexternalhit|preview|curl\/|wget|python-requests|httpx|go-http-client|headless|lighthouse|pingdom|uptime|monitor|scrapy|semrush|ahrefs|mj12|dotbot|petalbot|bytespider|gptbot|claudebot|ccbot|anthropic|openai/i;
5 +export function isBot(ua: string | null) { return !ua || BOT.test(ua); }
6 +/** Visitor hash: sha256(ip, user-agent, daily salt) — not reversible, rotates daily, one event per visitor per paper per day. */
7 +export function visitorHash(ip: string, ua: string | null) {
8 + const salt = process.env.SESSION_SECRET ?? "salt"; const day = new Date().toISOString().slice(0, 10);
9 + return createHash("sha256").update(`${salt}|${day}|${ip}|${ua ?? ""}`).digest("hex").slice(0, 32);
10 +}
11 +export async function recordEvent(kind: "view" | "download", paperId: string, h: { ip: string; ua: string | null; referrer?: string | null; locale?: string; versionId?: string | null }) {
12 + let host: string | null = null; try { host = h.referrer ? new URL(h.referrer).hostname : null; } catch { host = null; }
13 + await sql`INSERT INTO events (paper_id, version_id, kind, visitor_hash, referrer_host, locale, is_bot) VALUES (${paperId}, ${h.versionId ?? null}, ${kind}, ${visitorHash(h.ip, h.ua)}, ${host}, ${h.locale ?? null}, ${isBot(h.ua)}) ON CONFLICT DO NOTHING`;
14 +}
added src/lib/papers/identifiers.ts +27 −0
@@ -0,0 +1,27 @@
1 +/** Working paper identifiers: permanent, never renumbered, gaps allowed. `WP38` ⇄ slug `wp38`; `FIRC0002` ⇄ slug `firc2`. */
2 +export type ParsedId = { seriesCode: string; number: number; identifier: string; slug: string };
3 +export function makeIdentifier(seriesCode: string, number: number): ParsedId {
4 + const code = seriesCode.toUpperCase();
5 + const identifier = code === "WP" ? `WP${String(number).padStart(2, "0")}` : `${code}${String(number).padStart(4, "0")}`;
6 + return { seriesCode: code, number, identifier, slug: `${code.toLowerCase()}${number}` };
7 +}
8 +/** Accepts `wp38`, `WP38`, `WP038`, `wp-38`, `firc2`, `FIRC0002`, `38` (defaults to WP). */
9 +export function parseIdentifier(input: string): ParsedId | null {
10 + const s = input.trim().toLowerCase().replace(/[\s_-]+/g, "");
11 + const m = /^([a-z]*)0*(\d{1,5})$/.exec(s); if (!m) return null;
12 + const code = m[1] === "" ? "wp" : m[1]; const n = Number(m[2]); if (!Number.isFinite(n) || n <= 0) return null;
13 + return makeIdentifier(code, n);
14 +}
15 +/** Version labels: `v2`, `v1.1`, `2.0` → version number 2 / label "v2.0". Minor versions are kept in the label only. */
16 +export function parseVersionLabel(v: string): { versionNo: number; label: string } | null {
17 + const m = /^v?(\d+)(?:\.(\d+))?$/i.exec(v.trim()); if (!m) return null;
18 + return { versionNo: Number(m[1]), label: `v${m[1]}${m[2] ? "." + m[2] : ""}` };
19 +}
20 +/** Detect a working paper number in text extracted from a title page. */
21 +export function detectWpNumber(text: string): { seriesCode: string; number: number } | null {
22 + const t = text.replace(/\s+/g, " ");
23 + const firc = /FIRC[^0-9]{0,40}?(?:No\.?|N[o°]|Paper)\s*0*(\d{1,4})/i.exec(t); if (firc) return { seriesCode: "FIRC", number: Number(firc[1]) };
24 + const wp = /Working\s+Paper\s+(?:No\.?|N[o°]|Number|#)\s*0*(\d{1,4})/i.exec(t) ?? /\bWP\s*0*(\d{1,4})\b/.exec(t);
25 + if (wp) return { seriesCode: "WP", number: Number(wp[1]) };
26 + return null;
27 +}
added src/lib/papers/meta.ts +33 −0
@@ -0,0 +1,33 @@
1 +import { APP_URL, INSTITUTION } from "@/lib/env";
2 +import type { PaperFull } from "./types";
3 +import type { CitablePaper } from "./citation";
4 +export function citable(p: PaperFull, locale: "en" | "fr" = "en"): CitablePaper {
5 + return { identifier: p.identifier, slug: p.slug, title: p.title, subtitle: p.subtitle, year: p.year, authors: p.authors.map((a) => ({ givenName: a.given_name, familyName: a.family_name })),
6 + seriesName: locale === "fr" ? p.series_name_fr : p.series_name_en, institution: INSTITUTION, url: `${APP_URL}/papers/${p.slug}`, versionLabel: p.version_label, doi: p.doi, ssrnId: p.ssrn_id, keywords: p.keywords, abstract: p.abstract, releasedOn: p.first_released_on, releasedPrecision: p.first_released_precision, pages: p.pages, jel: p.jel };
7 +}
8 +/** Highwire Press / Google Scholar + Dublin Core meta tags. Only verified fields are emitted. */
9 +export function scholarMeta(p: PaperFull): Record<string, string | string[]> {
10 + const date = p.first_released_on ? (p.first_released_precision === "year" ? p.first_released_on.slice(0, 4) : p.first_released_precision === "month" ? p.first_released_on.slice(0, 7).replace("-", "/") : p.first_released_on.replace(/-/g, "/")) : undefined;
11 + const m: Record<string, string | string[]> = {
12 + citation_title: p.subtitle ? `${p.title}: ${p.subtitle}` : p.title, citation_author: p.authors.map((a) => `${a.family_name}, ${a.given_name}`),
13 + citation_technical_report_number: p.identifier, citation_technical_report_institution: INSTITUTION, citation_abstract_html_url: `${APP_URL}/papers/${p.slug}`, citation_language: p.language,
14 + "DC.title": p.title, "DC.creator": p.authors.map((a) => a.display_name), "DC.identifier": `${APP_URL}/papers/${p.slug}`, "DC.type": "Text", "DC.language": p.language, "DC.publisher": INSTITUTION, "DC.rights": "All rights reserved by the authors; working paper circulated for discussion.",
15 + };
16 + if (date) { m.citation_publication_date = date; m.citation_date = date; m["DC.date"] = p.first_released_on!.slice(0, p.first_released_precision === "month" ? 7 : p.first_released_precision === "year" ? 4 : 10); }
17 + if (p.has_pdf && p.file_id) m.citation_pdf_url = `${APP_URL}/api/files/${p.file_id}/${p.slug}-${p.version_label ?? "current"}.pdf`;
18 + if (p.doi) m.citation_doi = p.doi;
19 + if (p.keywords.length) { m.citation_keywords = p.keywords.join("; "); m["DC.subject"] = p.keywords; }
20 + if (p.abstract.length) m["DC.description"] = p.abstract.join(" ").slice(0, 1000);
21 + return m;
22 +}
23 +export function jsonLd(p: PaperFull) {
24 + return {
25 + "@context": "https://schema.org", "@type": "ScholarlyArticle", "@id": `${APP_URL}/papers/${p.slug}`, headline: p.title, alternativeHeadline: p.subtitle ?? undefined, name: p.title, identifier: p.identifier,
26 + author: p.authors.map((a) => ({ "@type": "Person", name: a.display_name, givenName: a.given_name, familyName: a.family_name, url: `${APP_URL}/authors/${a.slug}` })),
27 + datePublished: p.first_released_on ?? undefined, dateModified: p.last_revised_on ?? undefined, inLanguage: p.language, abstract: p.abstract.join("\n\n") || undefined, keywords: p.keywords.join(", ") || undefined,
28 + publisher: { "@type": "Organization", name: INSTITUTION }, isPartOf: { "@type": "Periodical", name: p.series_name_en }, url: `${APP_URL}/papers/${p.slug}`,
29 + encoding: p.has_pdf && p.file_id ? [{ "@type": "MediaObject", encodingFormat: "application/pdf", contentUrl: `${APP_URL}/api/files/${p.file_id}/${p.slug}-${p.version_label ?? "current"}.pdf` }] : undefined,
30 + sameAs: [p.ssrn_id ? `https://ssrn.com/abstract=${p.ssrn_id}` : null, p.doi ? `https://doi.org/${p.doi}` : null].filter(Boolean),
31 + version: p.version_label ?? undefined,
32 + };
33 +}
added src/lib/papers/openapi.ts +23 −0
@@ -0,0 +1,23 @@
1 +import { APP_URL } from "@/lib/env";
2 +export function openapi() {
3 + const q = (name: string, description: string, schema: Record<string, unknown> = { type: "string" }) => ({ name, in: "query", description, schema });
4 + return {
5 + openapi: "3.1.0",
6 + info: { title: "Research Working Papers API", version: "1.0.0", description: "Public, read-only API of the Simon-Pierre Boucher Research Observatory (wp.spboucher.ai).", contact: { url: `${APP_URL}/about` } },
7 + servers: [{ url: `${APP_URL}/api/v1` }],
8 + paths: {
9 + "/papers": { get: { summary: "List published working papers", description: "Filters are multi-valued (repeat the parameter or separate values with commas). Paginated.", parameters: [q("q", "Typo-tolerant query over titles, abstracts, keywords, authors, identifiers"), q("year", "Publication year(s)"), q("domain", "Research domain slug(s)"), q("topic", "Topic slug(s)"), q("author", "Author slug(s)"), q("keyword", "Keyword(s)"), q("method", "Method slug(s)"), q("dataset", "Dataset slug(s)"), q("series", "Series code (WP, FIRC)"), q("pub", "Publication status", { type: "string", enum: ["working_paper", "under_review", "forthcoming", "published"] }), q("pdf", "1 = only papers with a downloadable manuscript"), q("sort", "Sort order", { type: "string", enum: ["recent", "revised", "number", "title", "year", "views", "downloads"] }), q("page", "Page number", { type: "integer" }), q("per_page", "Items per page (max 100)", { type: "integer" })] } },
10 + "/papers/{id}": { get: { summary: "Get one working paper", description: "Full record: metadata, authors, topics, methods, datasets, versions, related papers (with explanations and an `inferred` flag).", parameters: [{ name: "id", in: "path", description: "Identifier or slug (wp38, WP38, firc2)", schema: { type: "string" } }] } },
11 + "/papers/{id}/citation.{format}": { get: { summary: "Citation export", parameters: [{ name: "id", in: "path", schema: { type: "string" } }, { name: "format", in: "path", schema: { type: "string", enum: ["bib", "ris", "json"] } }] } },
12 + "/search": { get: { summary: "Hybrid search", description: "Lexical (metadata + full text, typo-tolerant) fused with semantic retrieval when embeddings are available; results carry the signals that matched and a highlighted passage.", parameters: [q("q", "Query"), q("mode", "Retrieval mode", { type: "string", enum: ["hybrid", "lexical", "semantic"] }), q("limit", "Max results (≤ 50)", { type: "integer" })] } },
13 + "/stats": { get: { summary: "Collection statistics", description: "Computed live from the publication database." } },
14 + "/topics": { get: { summary: "Domains and topics with paper counts" } },
15 + "/methods": { get: { summary: "Methods with paper counts" } },
16 + "/datasets": { get: { summary: "Research data catalog" } },
17 + "/authors": { get: { summary: "Authors with paper counts" } },
18 + "/series": { get: { summary: "Research series" } },
19 + "/graph": { get: { summary: "Research knowledge graph", description: "Nodes = published papers; edges = relations (topic, method, dataset, author, semantic, lexical, curated). Inferred edges are flagged.", parameters: [q("kinds", "Comma-separated relation kinds to include")] } },
20 + "/timeline": { get: { summary: "Releases and revisions by date" } },
21 + },
22 + };
23 +}
added src/lib/papers/params.ts +7 −0
@@ -0,0 +1,7 @@
1 +import type { ListParams } from "./types";
2 +export type SP = Record<string, string | string[] | undefined>;
3 +const arr = (v: string | string[] | undefined) => (v === undefined ? [] : Array.isArray(v) ? v : [v]).flatMap((x) => x.split(",")).map((x) => x.trim()).filter(Boolean);
4 +export function parseListParams(sp: SP): ListParams {
5 + const sort = ["recent", "revised", "number", "title", "year", "views", "downloads"].includes(String(sp.sort)) ? (sp.sort as ListParams["sort"]) : "recent";
6 + return { q: typeof sp.q === "string" ? sp.q : undefined, year: arr(sp.year).map(Number).filter(Number.isFinite), domain: arr(sp.domain), topic: arr(sp.topic), author: arr(sp.author), keyword: arr(sp.keyword), method: arr(sp.method), dataset: arr(sp.dataset), pubStatus: arr(sp.pub), series: arr(sp.series), lang: arr(sp.lang), pdf: sp.pdf === "1", sort, page: Number(sp.page ?? 1) || 1, perPage: Math.min(100, Number(sp.per_page ?? 20) || 20) };
7 +}
added src/lib/papers/queries.ts +129 −0
@@ -0,0 +1,129 @@
1 +import "server-only";
2 +import { sqlPublic as sql } from "@/db";
3 +import type { ListParams, PaperFull, PaperSummary } from "./types";
4 +import { parseIdentifier } from "./identifiers";
5 +
6 +const SUMMARY_SELECT = sql`
7 + p.id, p.identifier, p.slug, p.number, s.code AS series_code, s.name_en AS series_name_en, s.name_fr AS series_name_fr,
8 + p.title, p.subtitle, p.title_fr, p.abstract, p.year, p.status, p.publication_status, p.notice, p.language,
9 + p.first_released_on::text, p.first_released_precision, p.last_revised_on::text, p.last_revised_precision,
10 + v.label AS version_label, v.version_no, v.pages, (v.file_id IS NOT NULL AND NOT v.withdrawn) AS has_pdf, v.file_id, v.sha256,
11 + coalesce((SELECT json_agg(json_build_object('slug', a.slug, 'display_name', a.display_name, 'given_name', a.given_name, 'family_name', a.family_name) ORDER BY pa.position) FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id), '[]'::json) AS authors,
12 + (SELECT json_build_object('slug', t.slug, 'name_en', t.name_en, 'name_fr', t.name_fr, 'accent', t.accent) FROM topics t WHERE t.id = p.primary_domain_id) AS domain,
13 + p.keywords, p.ssrn_id, p.doi, p.view_count::int, p.download_count::int, p.featured, p.featured_rank, p.published_at::text, p.updated_at::text,
14 + (SELECT count(*)::int FROM public_paper_versions pv WHERE pv.paper_id = p.id) AS versions_count`;
15 +const FROM = sql`FROM public_papers p JOIN series s ON s.id = p.series_id LEFT JOIN public_paper_versions v ON v.id = p.current_version_id`;
16 +
17 +export async function listPapers(params: ListParams): Promise<{ items: PaperSummary[]; total: number; page: number; perPage: number }> {
18 + const perPage = Math.min(100, Math.max(1, params.perPage ?? 20)); const page = Math.max(1, params.page ?? 1);
19 + const conds = [sql`p.status = 'published'`];
20 + if (params.featured) conds.push(sql`p.featured`);
21 + if (params.year?.length) conds.push(sql`p.year = ANY(${params.year})`);
22 + if (params.series?.length) conds.push(sql`s.code = ANY(${params.series.map((x) => x.toUpperCase())})`);
23 + if (params.status?.length) conds.push(sql`p.status = ANY(${params.status})`);
24 + if (params.pubStatus?.length) conds.push(sql`p.publication_status = ANY(${params.pubStatus})`);
25 + if (params.lang?.length) conds.push(sql`p.language = ANY(${params.lang})`);
26 + if (params.pdf) conds.push(sql`v.file_id IS NOT NULL AND NOT v.withdrawn`);
27 + if (params.domain?.length) conds.push(sql`EXISTS (SELECT 1 FROM paper_topics pt JOIN topics t ON t.id = pt.topic_id WHERE pt.paper_id = p.id AND t.kind = 'domain' AND t.slug = ANY(${params.domain}))`);
28 + if (params.topic?.length) conds.push(sql`EXISTS (SELECT 1 FROM paper_topics pt JOIN topics t ON t.id = pt.topic_id WHERE pt.paper_id = p.id AND t.slug = ANY(${params.topic}))`);
29 + if (params.author?.length) conds.push(sql`EXISTS (SELECT 1 FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id AND a.slug = ANY(${params.author}))`);
30 + if (params.method?.length) conds.push(sql`EXISTS (SELECT 1 FROM paper_methods pm JOIN methods m ON m.id = pm.method_id WHERE pm.paper_id = p.id AND m.slug = ANY(${params.method}))`);
31 + if (params.dataset?.length) conds.push(sql`EXISTS (SELECT 1 FROM paper_datasets pd JOIN datasets d ON d.id = pd.dataset_id WHERE pd.paper_id = p.id AND d.slug = ANY(${params.dataset}))`);
32 + if (params.keyword?.length) conds.push(sql`p.keywords && ${params.keyword}::text[]`);
33 + const q = params.q?.trim();
34 + if (q) conds.push(sql`(p.fts @@ websearch_to_tsquery('english', f_unaccent(${q})) OR p.search_blob % f_unaccent(lower(${q})) OR word_similarity(f_unaccent(lower(${q})), p.search_blob) > 0.45 OR p.identifier ILIKE ${"%" + q.replace(/\s+/g, "") + "%"} OR EXISTS (SELECT 1 FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id AND f_unaccent(lower(a.display_name)) LIKE '%' || f_unaccent(lower(${q})) || '%'))`);
35 + const where = conds.reduce((acc, c) => sql`${acc} AND ${c}`);
36 + const order = {
37 + recent: sql`p.first_released_on DESC NULLS LAST, p.number DESC`, revised: sql`p.last_revised_on DESC NULLS LAST, p.number DESC`, number: sql`s.sort, p.number DESC`,
38 + title: sql`p.title ASC`, year: sql`p.year DESC NULLS LAST, p.number DESC`, views: sql`p.view_count DESC, p.number DESC`, downloads: sql`p.download_count DESC, p.number DESC`,
39 + }[params.sort ?? "recent"];
40 + const rank = q ? sql`ts_rank_cd(p.fts, websearch_to_tsquery('english', f_unaccent(${q}))) + similarity(f_unaccent(lower(${q})), p.search_blob) DESC,` : sql``;
41 + const [{ total }] = await sql`SELECT count(*)::int AS total ${FROM} WHERE ${where}`;
42 + const items = await sql`SELECT ${SUMMARY_SELECT} ${FROM} WHERE ${where} ORDER BY ${rank} ${order} LIMIT ${perPage} OFFSET ${(page - 1) * perPage}`;
43 + return { items: items as unknown as PaperSummary[], total, page, perPage };
44 +}
45 +
46 +export async function getPaper(idOrSlug: string): Promise<PaperFull | null> {
47 + const parsed = parseIdentifier(idOrSlug); const slug = parsed?.slug ?? idOrSlug.toLowerCase();
48 + const rows = await sql`SELECT ${SUMMARY_SELECT}, p.abstract_fr, p.structured_abstract, p.research_question, p.research_question_fr, p.hero_headline, p.hero_subheadline, p.contributions, p.methodology, p.data_sections, p.findings, p.reproducibility, p.artifacts, p.external_links, p.jel, p.journal_ref, p.internal_code, p.embedding_model,
49 + coalesce((SELECT json_agg(json_build_object('slug', t.slug, 'name_en', t.name_en, 'name_fr', t.name_fr, 'kind', t.kind, 'assigned_by', pt.assigned_by) ORDER BY t.kind, t.sort) FROM paper_topics pt JOIN topics t ON t.id = pt.topic_id WHERE pt.paper_id = p.id), '[]'::json) AS topics,
50 + coalesce((SELECT json_agg(json_build_object('slug', m.slug, 'name_en', m.name_en, 'name_fr', m.name_fr, 'assigned_by', pm.assigned_by, 'note', pm.note) ORDER BY m.sort) FROM paper_methods pm JOIN methods m ON m.id = pm.method_id WHERE pm.paper_id = p.id), '[]'::json) AS methods,
51 + coalesce((SELECT json_agg(json_build_object('slug', d.slug, 'name_en', d.name_en, 'name_fr', d.name_fr, 'provider', d.provider, 'assigned_by', pd.assigned_by, 'role', pd.role) ORDER BY d.sort) FROM paper_datasets pd JOIN datasets d ON d.id = pd.dataset_id WHERE pd.paper_id = p.id), '[]'::json) AS datasets,
52 + coalesce((SELECT json_agg(json_build_object('id', pv.id, 'version_no', pv.version_no, 'label', pv.label, 'released_on', pv.released_on::text, 'released_precision', pv.released_precision, 'description', pv.description, 'description_fr', pv.description_fr, 'file_id', CASE WHEN pv.withdrawn THEN NULL ELSE pv.file_id END, 'pages', pv.pages, 'sha256', pv.sha256, 'bytes', pv.bytes, 'is_current', pv.is_current, 'withdrawn', pv.withdrawn, 'withdrawn_reason', pv.withdrawn_reason) ORDER BY pv.version_no DESC) FROM public_paper_versions pv WHERE pv.paper_id = p.id), '[]'::json) AS versions,
53 + coalesce((SELECT json_agg(x) FROM (SELECT q.slug, q.identifier, q.title, q.year, array_agg(r.kind ORDER BY r.kind) AS kinds, array_agg(r.explanation ORDER BY r.kind) AS explanations, array_agg(r.explanation_fr ORDER BY r.kind) AS explanations_fr, bool_and(r.inferred) AS inferred, sum(r.weight)::real AS weight
54 + FROM public_paper_relations r JOIN public_papers q ON q.id = r.to_paper_id AND q.status = 'published' WHERE r.from_paper_id = p.id GROUP BY q.slug, q.identifier, q.title, q.year, q.number ORDER BY sum(r.weight) DESC, q.number DESC LIMIT 8) x), '[]'::json) AS related
55 + ${FROM} WHERE p.slug = ${slug} LIMIT 1`;
56 + return (rows[0] as unknown as PaperFull) ?? null;
57 +}
58 +
59 +export async function facets() {
60 + const [years, domains, topics, authors, methods, datasets, series, keywords] = await Promise.all([
61 + sql`SELECT year, count(*)::int AS n FROM public_papers WHERE status = 'published' AND year IS NOT NULL GROUP BY year ORDER BY year DESC`,
62 + sql`SELECT t.slug, t.name_en, t.name_fr, t.accent, count(pt.paper_id)::int AS n FROM topics t LEFT JOIN paper_topics pt ON pt.topic_id = t.id LEFT JOIN public_papers p ON p.id = pt.paper_id AND p.status = 'published' WHERE t.kind = 'domain' AND t.is_active GROUP BY t.id ORDER BY t.sort`,
63 + sql`SELECT t.slug, t.name_en, t.name_fr, count(p.id)::int AS n FROM topics t JOIN paper_topics pt ON pt.topic_id = t.id JOIN public_papers p ON p.id = pt.paper_id AND p.status = 'published' WHERE t.kind = 'topic' GROUP BY t.id HAVING count(p.id) > 0 ORDER BY t.name_en`,
64 + sql`SELECT a.slug, a.display_name AS name, count(p.id)::int AS n FROM authors a JOIN paper_authors pa ON pa.author_id = a.id JOIN public_papers p ON p.id = pa.paper_id AND p.status = 'published' GROUP BY a.id ORDER BY a.sort`,
65 + sql`SELECT m.slug, m.name_en, m.name_fr, count(p.id)::int AS n FROM methods m JOIN paper_methods pm ON pm.method_id = m.id JOIN public_papers p ON p.id = pm.paper_id AND p.status = 'published' GROUP BY m.id HAVING count(p.id) > 0 ORDER BY m.name_en`,
66 + sql`SELECT d.slug, d.name_en, d.name_fr, count(p.id)::int AS n FROM datasets d JOIN paper_datasets pd ON pd.dataset_id = d.id JOIN public_papers p ON p.id = pd.paper_id AND p.status = 'published' GROUP BY d.id HAVING count(p.id) > 0 ORDER BY d.name_en`,
67 + sql`SELECT s.code AS slug, s.name_en, s.name_fr, count(p.id)::int AS n FROM series s LEFT JOIN public_papers p ON p.series_id = s.id AND p.status = 'published' GROUP BY s.id ORDER BY s.sort`,
68 + sql`SELECT k AS keyword, count(*)::int AS n FROM public_papers p, unnest(p.keywords) k WHERE p.status = 'published' GROUP BY k HAVING count(*) >= 2 ORDER BY n DESC, k LIMIT 60`,
69 + ]);
70 + type N = { slug: string; name_en: string; name_fr: string; n: number; accent?: string | null };
71 + return { years: years as unknown as { year: number; n: number }[], domains: domains as unknown as N[], topics: topics as unknown as N[], authors: authors as unknown as { slug: string; name: string; n: number }[], methods: methods as unknown as N[], datasets: datasets as unknown as N[], series: series as unknown as N[], keywords: keywords as unknown as { keyword: string; n: number }[] };
72 +}
73 +
74 +export async function stats() {
75 + const [r] = await sql`SELECT
76 + (SELECT count(*)::int FROM public_papers WHERE status = 'published') AS papers,
77 + (SELECT count(*)::int FROM public_papers p JOIN public_paper_versions v ON v.id = p.current_version_id WHERE p.status = 'published' AND v.file_id IS NOT NULL AND NOT v.withdrawn) AS manuscripts,
78 + (SELECT count(DISTINCT t.id)::int FROM topics t JOIN paper_topics pt ON pt.topic_id = t.id JOIN public_papers p ON p.id = pt.paper_id AND p.status = 'published' WHERE t.kind = 'topic') AS topics,
79 + (SELECT count(DISTINCT t.id)::int FROM topics t JOIN paper_topics pt ON pt.topic_id = t.id JOIN public_papers p ON p.id = pt.paper_id AND p.status = 'published' WHERE t.kind = 'domain') AS domains,
80 + (SELECT count(DISTINCT a.id)::int FROM authors a JOIN paper_authors pa ON pa.author_id = a.id JOIN public_papers p ON p.id = pa.paper_id AND p.status = 'published') AS authors,
81 + (SELECT count(DISTINCT d.id)::int FROM datasets d JOIN paper_datasets pd ON pd.dataset_id = d.id JOIN public_papers p ON p.id = pd.paper_id AND p.status = 'published') AS datasets,
82 + (SELECT count(DISTINCT m.id)::int FROM methods m JOIN paper_methods pm ON pm.method_id = m.id JOIN public_papers p ON p.id = pm.paper_id AND p.status = 'published') AS methods,
83 + (SELECT max(greatest(first_released_on, last_revised_on))::text FROM public_papers WHERE status = 'published') AS latest,
84 + (SELECT max(first_released_precision) FROM public_papers WHERE status = 'published') AS latest_precision,
85 + (SELECT coalesce(sum(v.pages), 0)::int FROM public_papers p JOIN public_paper_versions v ON v.id = p.current_version_id WHERE p.status = 'published') AS pages,
86 + (SELECT count(*)::int FROM public_papers WHERE status = 'published' AND ssrn_id IS NOT NULL) AS ssrn,
87 + (SELECT count(*)::int FROM public_paper_versions v JOIN public_papers p ON p.id = v.paper_id WHERE p.status = 'published') AS versions`;
88 + return r as unknown as { papers: number; manuscripts: number; topics: number; domains: number; authors: number; datasets: number; methods: number; latest: string | null; latest_precision: "day" | "month" | "year"; pages: number; ssrn: number; versions: number };
89 +}
90 +
91 +export async function domainsWithCounts() {
92 + return sql`SELECT t.slug, t.name_en, t.name_fr, t.description_en, t.description_fr, t.accent, count(pt.paper_id)::int AS n FROM topics t LEFT JOIN paper_topics pt ON pt.topic_id = t.id LEFT JOIN public_papers p ON p.id = pt.paper_id AND p.status = 'published' WHERE t.kind = 'domain' AND t.is_active GROUP BY t.id ORDER BY t.sort`;
93 +}
94 +export async function topicsWithCounts() {
95 + return sql`SELECT t.slug, t.name_en, t.name_fr, t.description_en, t.description_fr, t.kind, t.accent, count(p.id)::int AS n FROM topics t LEFT JOIN paper_topics pt ON pt.topic_id = t.id LEFT JOIN public_papers p ON p.id = pt.paper_id AND p.status = 'published' WHERE t.is_active GROUP BY t.id ORDER BY t.kind, t.sort`;
96 +}
97 +export async function getTopic(slug: string) { const [t] = await sql`SELECT slug, name_en, name_fr, description_en, description_fr, kind, accent FROM topics WHERE slug = ${slug} AND is_active`; return t ?? null; }
98 +export async function methodsWithCounts() {
99 + return sql`SELECT m.slug, m.name_en, m.name_fr, m.description_en, m.description_fr, m.category, count(p.id)::int AS n, coalesce(json_agg(json_build_object('slug', p.slug, 'identifier', p.identifier, 'title', p.title, 'year', p.year) ORDER BY p.number DESC) FILTER (WHERE p.id IS NOT NULL), '[]'::json) AS papers
100 + FROM methods m LEFT JOIN paper_methods pm ON pm.method_id = m.id LEFT JOIN public_papers p ON p.id = pm.paper_id AND p.status = 'published' GROUP BY m.id ORDER BY m.category, m.name_en`;
101 +}
102 +export async function getMethod(slug: string) { const [m] = await sql`SELECT slug, name_en, name_fr, description_en, description_fr, category FROM methods WHERE slug = ${slug}`; return m ?? null; }
103 +export async function datasetsWithCounts() {
104 + return sql`SELECT d.slug, d.name_en, d.name_fr, d.provider, d.coverage, d.frequency, d.access_conditions, d.license, d.url, d.description_en, d.description_fr, d.is_public, count(p.id)::int AS n, coalesce(json_agg(json_build_object('slug', p.slug, 'identifier', p.identifier, 'title', p.title, 'year', p.year) ORDER BY p.number DESC) FILTER (WHERE p.id IS NOT NULL), '[]'::json) AS papers
105 + FROM datasets d LEFT JOIN paper_datasets pd ON pd.dataset_id = d.id LEFT JOIN public_papers p ON p.id = pd.paper_id AND p.status = 'published' GROUP BY d.id ORDER BY n DESC, d.name_en`;
106 +}
107 +export async function getDataset(slug: string) { const [d] = await sql`SELECT * FROM datasets WHERE slug = ${slug}`; return d ?? null; }
108 +export async function authorsWithCounts() {
109 + return sql`SELECT a.*, count(p.id)::int AS n FROM authors a LEFT JOIN paper_authors pa ON pa.author_id = a.id LEFT JOIN public_papers p ON p.id = pa.paper_id AND p.status = 'published' GROUP BY a.id ORDER BY a.sort, a.family_name`;
110 +}
111 +export async function getAuthor(slug: string) { const [a] = await sql`SELECT * FROM authors WHERE slug = ${slug}`; return a ?? null; }
112 +export async function seriesWithCounts() {
113 + return sql`SELECT s.*, count(p.id)::int AS n, min(p.number)::int AS min_no, max(p.number)::int AS max_no FROM series s LEFT JOIN public_papers p ON p.series_id = s.id AND p.status = 'published' WHERE s.is_active GROUP BY s.id ORDER BY s.sort`;
114 +}
115 +export async function timeline() {
116 + const rows = await sql`SELECT p.slug, p.identifier, p.title, p.year, p.first_released_on::text AS released, p.first_released_precision AS rp, v.released_on::text AS vdate, v.version_no, v.label FROM public_papers p JOIN public_paper_versions v ON v.paper_id = p.id WHERE p.status = 'published' ORDER BY coalesce(v.released_on, p.first_released_on) DESC, p.number DESC`;
117 + return rows as unknown as { slug: string; identifier: string; title: string; year: number | null; released: string | null; rp: string; vdate: string | null; version_no: number; label: string }[];
118 +}
119 +export async function graph(kinds?: string[]) {
120 + const nodes = await sql`SELECT p.slug, p.identifier, p.title, p.year, t.slug AS domain, t.accent, p.view_count::int AS views FROM public_papers p LEFT JOIN topics t ON t.id = p.primary_domain_id WHERE p.status = 'published' ORDER BY p.number`;
121 + const edges = kinds?.length
122 + ? await sql`SELECT a.slug AS source, b.slug AS target, r.kind, r.weight, r.inferred, r.explanation, r.explanation_fr FROM public_paper_relations r JOIN public_papers a ON a.id = r.from_paper_id JOIN public_papers b ON b.id = r.to_paper_id WHERE a.slug < b.slug AND r.kind = ANY(${kinds})`
123 + : await sql`SELECT a.slug AS source, b.slug AS target, r.kind, r.weight, r.inferred, r.explanation, r.explanation_fr FROM public_paper_relations r JOIN public_papers a ON a.id = r.from_paper_id JOIN public_papers b ON b.id = r.to_paper_id WHERE a.slug < b.slug`;
124 + return { nodes, edges };
125 +}
126 +export async function siteSetting<T = unknown>(key: string): Promise<T | null> { const [r] = await sql`SELECT value FROM public_settings WHERE key = ${key}`; return (r?.value as T) ?? null; }
127 +export async function findRedirect(path: string) { const [r] = await sql`SELECT to_path, status FROM redirects WHERE from_path = ${path}`; return r ?? null; }
128 +export async function publicFile(fileId: string) { const [f] = await sql`SELECT f.*, v.paper_id, v.id AS version_id, p.slug, p.identifier, v.label FROM public_files f JOIN public_paper_versions v ON v.file_id = f.id JOIN public_papers p ON p.id = v.paper_id WHERE f.id = ${fileId} AND NOT v.withdrawn LIMIT 1`; return f ?? null; }
129 +export async function previewFor(slug: string) { const [r] = await sql`SELECT f.sha256 FROM public_papers p JOIN public_paper_versions v ON v.id = p.current_version_id JOIN public_files f ON f.id = v.file_id WHERE p.slug = ${slug}`; return r?.sha256 ?? null; }
added src/lib/papers/search.ts +50 −0
@@ -0,0 +1,50 @@
1 +import "server-only";
2 +import { sqlPublic as sql } from "@/db";
3 +import { embed, embeddingsEnabled } from "@/lib/ai/embeddings";
4 +export type SearchMode = "hybrid" | "lexical" | "semantic";
5 +export type SearchHit = { slug: string; identifier: string; title: string; subtitle: string | null; year: number | null; authors: string[]; score: number; signals: string[]; snippets: { text: string; page: number | null; kind: "chunk" | "abstract" }[] };
6 +const vec = (v: number[]) => `[${v.map((x) => x.toFixed(6)).join(",")}]`;
7 +
8 +/** Hybrid retrieval: lexical (tsvector over metadata + full-text chunks, trigram typo tolerance) fused with semantic (pgvector) by reciprocal rank fusion. */
9 +export async function search(q: string, mode: SearchMode = "hybrid", limit = 20): Promise<{ hits: SearchHit[]; mode: SearchMode; semanticUsed: boolean; tookMs: number }> {
10 + const t0 = Date.now(); const query = q.trim().slice(0, 300); if (!query) return { hits: [], mode, semanticUsed: false, tookMs: 0 };
11 + const rr = new Map<string, { score: number; signals: Set<string> }>();
12 + const add = (id: string, rank: number, signal: string, w = 1) => { const e = rr.get(id) ?? { score: 0, signals: new Set() }; e.score += w / (60 + rank); e.signals.add(signal); rr.set(id, e); };
13 + let semanticUsed = false;
14 + if (mode !== "semantic") {
15 + const meta = await sql`SELECT p.id, ts_rank_cd(p.fts, websearch_to_tsquery('english', f_unaccent(${query}))) AS r, similarity(f_unaccent(lower(${query})), p.search_blob) AS s,
16 + EXISTS (SELECT 1 FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id AND f_unaccent(lower(a.display_name)) LIKE '%' || f_unaccent(lower(${query})) || '%') AS by_author,
17 + p.identifier ILIKE ${"%" + query.replace(/\s+/g, "") + "%"} AS by_id
18 + FROM public_papers p WHERE p.status = 'published' AND (p.fts @@ websearch_to_tsquery('english', f_unaccent(${query})) OR p.search_blob % f_unaccent(lower(${query})) OR word_similarity(f_unaccent(lower(${query})), p.search_blob) > 0.45
19 + OR EXISTS (SELECT 1 FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id AND f_unaccent(lower(a.display_name)) LIKE '%' || f_unaccent(lower(${query})) || '%') OR p.identifier ILIKE ${"%" + query.replace(/\s+/g, "") + "%"})
20 + ORDER BY (CASE WHEN p.identifier ILIKE ${"%" + query.replace(/\s+/g, "") + "%"} THEN 10 ELSE 0 END) + ts_rank_cd(p.fts, websearch_to_tsquery('english', f_unaccent(${query}))) * 10 + similarity(f_unaccent(lower(${query})), p.search_blob) DESC LIMIT 40`;
21 + meta.forEach((m, i) => add(m.id, i, m.by_id ? "identifier" : m.by_author ? "author" : Number(m.r) > 0 ? "metadata" : "fuzzy-title", 1.2));
22 + const chunks = await sql`SELECT DISTINCT ON (c.paper_id) c.paper_id AS id, c.page_from, ts_headline('english', c.text, websearch_to_tsquery('english', f_unaccent(${query})), 'MaxWords=40, MinWords=20, StartSel=<mark>, StopSel=</mark>, MaxFragments=2, FragmentDelimiter=" … "') AS snippet, ts_rank_cd(c.fts, websearch_to_tsquery('english', f_unaccent(${query}))) AS r
23 + FROM public_paper_chunks c WHERE c.fts @@ websearch_to_tsquery('english', f_unaccent(${query})) ORDER BY c.paper_id, r DESC`;
24 + const sorted = [...chunks].sort((a, b) => Number(b.r) - Number(a.r)).slice(0, 40);
25 + sorted.forEach((c, i) => { add(c.id, i, "full-text", 1); const e = rr.get(c.id)!; (e as unknown as { snip?: { text: string; page: number | null } }).snip = { text: c.snippet, page: c.page_from }; });
26 + }
27 + if (mode !== "lexical" && embeddingsEnabled()) {
28 + try {
29 + const e = await embed([query]);
30 + if (e) {
31 + semanticUsed = true; const v = vec(e[0]);
32 + const sem = await sql`SELECT id, 1 - (embedding <=> ${v}::vector) AS sim FROM public_papers WHERE status = 'published' AND embedding IS NOT NULL ORDER BY embedding <=> ${v}::vector LIMIT 20`;
33 + sem.forEach((s, i) => { if (Number(s.sim) > 0.25) add(s.id, i, "semantic", 1); });
34 + const semc = await sql`SELECT DISTINCT ON (paper_id) paper_id AS id, text, page_from, 1 - (embedding <=> ${v}::vector) AS sim FROM public_paper_chunks WHERE embedding IS NOT NULL ORDER BY paper_id, embedding <=> ${v}::vector`;
35 + [...semc].sort((a, b) => Number(b.sim) - Number(a.sim)).slice(0, 20).forEach((c, i) => { if (Number(c.sim) > 0.35) { add(c.id, i, "semantic-passage", 0.8); const e2 = rr.get(c.id) as unknown as { snip?: { text: string; page: number | null } }; if (!e2.snip) e2.snip = { text: String(c.text).slice(0, 320) + "…", page: c.page_from }; } });
36 + }
37 + } catch (err) { console.warn("semantic search unavailable:", (err as Error).message); }
38 + }
39 + const ids = [...rr.entries()].sort((a, b) => b[1].score - a[1].score).slice(0, limit).map(([id]) => id);
40 + if (!ids.length) return { hits: [], mode, semanticUsed, tookMs: Date.now() - t0 };
41 + const rows = await sql`SELECT p.id, p.slug, p.identifier, p.title, p.subtitle, p.year, p.abstract, (SELECT array_agg(a.display_name ORDER BY pa.position) FROM paper_authors pa JOIN authors a ON a.id = pa.author_id WHERE pa.paper_id = p.id) AS authors FROM public_papers p WHERE p.id = ANY(${ids})`;
42 + const by = new Map(rows.map((r) => [r.id, r]));
43 + const hits: SearchHit[] = ids.map((id) => { const r = by.get(id)!; const e = rr.get(id)! as { score: number; signals: Set<string>; snip?: { text: string; page: number | null } };
44 + const snippets = e.snip ? [{ text: e.snip.text, page: e.snip.page, kind: "chunk" as const }] : [{ text: String((r.abstract as string[])[0] ?? "").slice(0, 300) + "…", page: null, kind: "abstract" as const }];
45 + return { slug: r.slug, identifier: r.identifier, title: r.title, subtitle: r.subtitle, year: r.year, authors: r.authors ?? [], score: Number(e.score.toFixed(4)), signals: [...e.signals], snippets }; });
46 + return { hits, mode, semanticUsed, tookMs: Date.now() - t0 };
47 +}
48 +export async function logSearch(query: string, mode: string, results: number, locale: string, tookMs: number) {
49 + try { await sql`INSERT INTO search_log (query, mode, results, locale, took_ms) VALUES (${query.slice(0, 300)}, ${mode}, ${results}, ${locale}, ${tookMs})`; } catch { /* ignore */ }
50 +}
added src/lib/papers/types.ts +25 −0
@@ -0,0 +1,25 @@
1 +export type PaperSummary = {
2 + id: string; identifier: string; slug: string; number: number; series_code: string; series_name_en: string; series_name_fr: string;
3 + title: string; subtitle: string | null; title_fr: string | null; abstract: string[]; year: number | null; status: string; publication_status: string; notice: string | null;
4 + first_released_on: string | null; first_released_precision: "day" | "month" | "year"; last_revised_on: string | null; last_revised_precision: "day" | "month" | "year";
5 + version_label: string | null; version_no: number | null; pages: number | null; has_pdf: boolean; file_id: string | null; sha256: string | null;
6 + authors: { slug: string; display_name: string; given_name: string; family_name: string }[];
7 + domain: { slug: string; name_en: string; name_fr: string; accent: string | null } | null;
8 + keywords: string[]; ssrn_id: string | null; doi: string | null; view_count: number; download_count: number; featured: boolean; featured_rank: number | null; published_at: string | null; updated_at: string;
9 + versions_count: number; language: string;
10 +};
11 +export type PaperFull = PaperSummary & {
12 + abstract_fr: string[] | null; structured_abstract: unknown; research_question: string | null; research_question_fr: string | null; hero_headline: string | null; hero_subheadline: string | null;
13 + contributions: { kind?: string; title: string; description: string }[]; methodology: { title: string; description: string }[]; data_sections: { title: string; description: string }[];
14 + findings: { value: string; label: string }[]; reproducibility: string[]; artifacts: { kind: string; label: string; url: string }[]; external_links: { kind: string; label: string; url: string }[];
15 + jel: string[]; journal_ref: string | null; internal_code: string | null; embedding_model: string | null;
16 + topics: { slug: string; name_en: string; name_fr: string; kind: string; assigned_by: string }[];
17 + methods: { slug: string; name_en: string; name_fr: string; assigned_by: string; note: string | null }[];
18 + datasets: { slug: string; name_en: string; name_fr: string; provider: string | null; assigned_by: string; role: string | null }[];
19 + versions: { id: string; version_no: number; label: string; released_on: string | null; released_precision: string; description: string | null; description_fr: string | null; file_id: string | null; pages: number | null; sha256: string | null; bytes: number | null; is_current: boolean; withdrawn: boolean; withdrawn_reason: string | null }[];
20 + related: { slug: string; identifier: string; title: string; year: number | null; kinds: string[]; explanations: string[]; explanations_fr: string[]; inferred: boolean; weight: number }[];
21 +};
22 +export type ListParams = {
23 + q?: string; year?: number[]; domain?: string[]; topic?: string[]; author?: string[]; keyword?: string[]; method?: string[]; dataset?: string[];
24 + status?: string[]; pubStatus?: string[]; series?: string[]; lang?: string[]; pdf?: boolean; sort?: "recent" | "revised" | "number" | "title" | "year" | "views" | "downloads"; page?: number; perPage?: number; featured?: boolean;
25 +};
added src/lib/pdf/chunk.ts +16 −0
@@ -0,0 +1,16 @@
1 +/** Split page texts into overlapping chunks of ~1,200 characters aligned on paragraph boundaries; keeps page ranges. */
2 +export type Chunk = { index: number; pageFrom: number; pageTo: number; text: string };
3 +export function chunkPages(pages: string[], target = 1200, overlap = 150): Chunk[] {
4 + const out: Chunk[] = []; let buf = ""; let from = 1; let idx = 0;
5 + const flush = (pageTo: number) => { const t = buf.replace(/\s+/g, " ").trim(); if (t.length > 80) out.push({ index: idx++, pageFrom: from, pageTo, text: t }); buf = buf.slice(-overlap); };
6 + pages.forEach((page, i) => {
7 + const pno = i + 1; if (!buf.trim()) from = pno;
8 + for (const para of page.split(/\n{2,}/)) {
9 + const p = para.replace(/-\n/g, "").replace(/\n/g, " ").trim(); if (!p) continue;
10 + if (buf.length + p.length > target && buf.length > target / 2) { flush(pno); from = pno; }
11 + buf += (buf ? "\n" : "") + p;
12 + }
13 + });
14 + if (buf.trim()) flush(pages.length);
15 + return out;
16 +}
added src/lib/pdf/extract.ts +57 −0
@@ -0,0 +1,57 @@
1 +import { detectWpNumber } from "@/lib/papers/identifiers";
2 +/** Heuristic metadata proposal from title-page text. Every field is a suggestion for human review, never a fact. */
3 +export type Proposal = { title: string | null; authors: string[]; abstract: string | null; keywords: string[]; jel: string[]; wp: { seriesCode: string; number: number } | null; version: string | null; dateText: string | null; confidence: Record<string, number> };
4 +const STOP = /^(abstract|résumé|keywords?|jel|working paper|université|university|department|département|version|draft|preliminary|email|e-mail|tel|http)/i;
5 +export function proposeMetadata(firstPages: string, pdfMetaTitle?: string | null, pdfMetaAuthor?: string | null): Proposal {
6 + const text = firstPages.replace(/\r/g, "");
7 + const lines = text.split("\n").map((l) => l.trim()).filter(Boolean);
8 + const confidence: Record<string, number> = {};
9 + // Title: longest run of up to 3 consecutive Title-Case-ish lines near the top, before "Abstract", not a stop line.
10 + let title: string | null = null;
11 + const abstractIdx = lines.findIndex((l) => /^abstract\b/i.test(l));
12 + const head = lines.slice(0, abstractIdx > 0 ? abstractIdx : 40);
13 + const cands: string[] = [];
14 + for (let i = 0; i < head.length; i++) {
15 + const l = head[i]; if (STOP.test(l) || l.length < 12 || /@|\d{4}-\d{2}/.test(l)) continue;
16 + if (/^[A-Z]/.test(l) && !/[.;]$/.test(l) && !/^(by|and)\b/i.test(l)) {
17 + let acc = l; let j = i + 1;
18 + while (j < head.length && head[j].length > 8 && !STOP.test(head[j]) && !/@/.test(head[j]) && !/^(by|and)\b/i.test(head[j]) && acc.length < 300 && !/^[A-Z][a-z]+ [A-Z]/.test(head[j]) /* author-like */) { acc += " " + head[j]; j++; if (/[?:.]$/.test(head[j - 1])) break; }
19 + cands.push(acc);
20 + }
21 + }
22 + cands.sort((a, b) => b.length - a.length);
23 + title = cands[0] ?? null;
24 + if (pdfMetaTitle && pdfMetaTitle.length > 12 && !/^(untitled|main|paper|subject)/i.test(pdfMetaTitle)) { if (!title || pdfMetaTitle.length > 20) { title = title && title.length > pdfMetaTitle.length + 20 ? title : pdfMetaTitle; } }
25 + confidence.title = title ? (pdfMetaTitle && title === pdfMetaTitle ? 0.8 : 0.5) : 0;
26 + // Authors: lines with 2–4 capitalised words (allowing accents/hyphens), before Abstract, excluding title lines.
27 + const authors: string[] = [];
28 + const nameRe = /^(?:[A-ZÀ-Ý][a-zà-ÿ'’-]+(?:\s|-)){1,3}[A-ZÀ-Ý][a-zà-ÿ'’-]+(?:\s*[,*†‡§¶0-9]+)?$/;
29 + for (const l of head) { const cleaned = l.replace(/[*†‡§¶0-9,]+$/g, "").trim(); if (nameRe.test(cleaned) && !(title ?? "").includes(cleaned) && !/university|université|department|école|school/i.test(cleaned)) authors.push(cleaned); }
30 + if (!authors.length && pdfMetaAuthor) authors.push(...pdfMetaAuthor.split(/\s*(?:,|;|&|\band\b)\s*/).filter((a) => a.length > 3));
31 + confidence.authors = authors.length ? 0.5 : 0;
32 + // Abstract: text after "Abstract" until Keywords/JEL/1 Introduction.
33 + let abstract: string | null = null;
34 + if (abstractIdx >= 0) {
35 + const rest = lines.slice(abstractIdx + 1); const end = rest.findIndex((l) => /^(keywords?|jel|1\.?\s+introduction|introduction$|mots-clés)/i.test(l));
36 + abstract = rest.slice(0, end > 0 ? end : 40).join(" ").replace(/^abstract[:.]?\s*/i, "").replace(/\s+/g, " ").trim() || null;
37 + if (abstract && abstract.length < 80) abstract = null;
38 + }
39 + confidence.abstract = abstract ? 0.7 : 0;
40 + const kwLine = lines.find((l) => /^(keywords?|mots-clés)\s*[:.]/i.test(l));
41 + const keywords = kwLine ? kwLine.replace(/^(keywords?|mots-clés)\s*[:.]\s*/i, "").split(/[;,·•]/).map((k) => k.trim().replace(/\.$/, "")).filter((k) => k.length > 1 && k.length < 60) : [];
42 + const jelLine = lines.find((l) => /^jel\b/i.test(l));
43 + const jel = jelLine ? (jelLine.match(/\b[A-Z]\d{1,2}\b/g) ?? []) : [];
44 + const wp = detectWpNumber(text.slice(0, 4000));
45 + const vm = /\bversion\s*v?(\d+(?:\.\d+)?)/i.exec(text.slice(0, 6000)); const version = vm ? vm[1] : null;
46 + const dm = /\b(January|February|March|April|May|June|July|August|September|October|November|December|janvier|février|mars|avril|mai|juin|juillet|août|septembre|octobre|novembre|décembre)\s+(\d{4})\b/i.exec(text.slice(0, 6000));
47 + const dateText = dm ? `${dm[1]} ${dm[2]}` : null;
48 + confidence.wp = wp ? 0.9 : 0; confidence.date = dateText ? 0.6 : 0;
49 + return { title, authors, abstract, keywords, jel, wp, version, dateText, confidence };
50 +}
51 +/** Rough duplicate scoring between a candidate and an existing paper. */
52 +export function titleSimilarity(a: string, b: string): number {
53 + const norm = (s: string) => s.toLowerCase().normalize("NFD").replace(/[̀-ͯ]/g, "").replace(/[^a-z0-9 ]+/g, " ").split(/\s+/).filter((w) => w.length > 2);
54 + const A = new Set(norm(a)), B = new Set(norm(b)); if (!A.size || !B.size) return 0;
55 + let inter = 0; for (const w of A) if (B.has(w)) inter++;
56 + return inter / Math.max(A.size, B.size);
57 +}
added src/lib/pdf/process.ts +54 −0
@@ -0,0 +1,54 @@
1 +import { execFile } from "node:child_process";
2 +import { promisify } from "node:util";
3 +import { createHash } from "node:crypto";
4 +import { createReadStream, promises as fs } from "node:fs";
5 +import { join, dirname } from "node:path";
6 +import { STORAGE_DIR } from "@/lib/env";
7 +const run = promisify(execFile);
8 +const BIN = (name: string) => process.env.POPPLER_BIN ? join(process.env.POPPLER_BIN, name) : name;
9 +const PATH_ENV = { ...process.env, PATH: `/opt/homebrew/bin:/usr/local/bin:${process.env.PATH ?? ""}` };
10 +
11 +export async function sha256File(path: string): Promise<string> {
12 + return new Promise((res, rej) => { const h = createHash("sha256"); createReadStream(path).on("data", (d) => h.update(d)).on("end", () => res(h.digest("hex"))).on("error", rej); });
13 +}
14 +export type PdfInfo = { pages: number | null; title: string | null; author: string | null; subject: string | null; keywords: string | null; creator: string | null; producer: string | null; creationDate: string | null; modDate: string | null; pdfVersion: string | null; encrypted: boolean; pageSize: string | null; raw: Record<string, string> };
15 +export async function pdfInfo(path: string): Promise<PdfInfo> {
16 + const { stdout } = await run(BIN("pdfinfo"), [path], { env: PATH_ENV, maxBuffer: 1 << 20 });
17 + const raw: Record<string, string> = {};
18 + for (const line of stdout.split("\n")) { const i = line.indexOf(":"); if (i > 0) raw[line.slice(0, i).trim()] = line.slice(i + 1).trim(); }
19 + const g = (k: string) => raw[k] ?? null;
20 + return { pages: raw.Pages ? Number(raw.Pages) : null, title: g("Title"), author: g("Author"), subject: g("Subject"), keywords: g("Keywords"), creator: g("Creator"), producer: g("Producer"), creationDate: g("CreationDate"), modDate: g("ModDate"), pdfVersion: g("PDF version"), encrypted: (raw.Encrypted ?? "no").toLowerCase() !== "no", pageSize: g("Page size"), raw };
21 +}
22 +/** Integrity: the file must parse, report ≥ 1 page, not be encrypted, and the first page must yield text (or be flagged scanned). */
23 +export async function pdfIntegrity(path: string): Promise<{ ok: boolean; reason?: string; info?: PdfInfo }> {
24 + try {
25 + const info = await pdfInfo(path);
26 + if (!info.pages || info.pages < 1) return { ok: false, reason: "no pages", info };
27 + if (info.encrypted) return { ok: false, reason: "encrypted", info };
28 + return { ok: true, info };
29 + } catch (e) { return { ok: false, reason: (e as Error).message.slice(0, 200) }; }
30 +}
31 +export async function pdfText(path: string, opts: { first?: number; last?: number } = {}): Promise<string> {
32 + const args = ["-layout", "-enc", "UTF-8"]; if (opts.first) args.push("-f", String(opts.first)); if (opts.last) args.push("-l", String(opts.last)); args.push(path, "-");
33 + const { stdout } = await run(BIN("pdftotext"), args, { env: PATH_ENV, maxBuffer: 64 << 20 });
34 + return stdout;
35 +}
36 +/** Full text split per page (form-feed separated by pdftotext). */
37 +export async function pdfTextPages(path: string): Promise<string[]> {
38 + const t = await pdfText(path); return t.split("\f").map((p) => p.replace(/[ \t]+\n/g, "\n").trim());
39 +}
40 +export async function pdfThumbnail(path: string, outPngBase: string, dpi = 50): Promise<string> {
41 + await fs.mkdir(dirname(outPngBase), { recursive: true });
42 + await run(BIN("pdftoppm"), ["-png", "-r", String(dpi), "-f", "1", "-l", "1", "-singlefile", path, outPngBase], { env: PATH_ENV });
43 + return `${outPngBase}.png`;
44 +}
45 +export function storagePath(key: string) { return join(/*turbopackIgnore: true*/ STORAGE_DIR, key); }
46 +export async function ensureStorage() { for (const d of ["files", "previews", "inbox", "tmp"]) await fs.mkdir(join(/*turbopackIgnore: true*/ STORAGE_DIR, d), { recursive: true }); }
47 +/** Content-addressed storage key: files/<2 hex>/<sha256>.pdf — the same manuscript is never stored twice. */
48 +export function manuscriptKey(sha: string, ext = "pdf") { return `files/${sha.slice(0, 2)}/${sha}.${ext}`; }
49 +export async function storeFile(srcPath: string, sha: string, ext = "pdf"): Promise<string> {
50 + const key = manuscriptKey(sha, ext); const dest = storagePath(key);
51 + await fs.mkdir(dirname(dest), { recursive: true });
52 + try { await fs.access(dest); } catch { await fs.copyFile(srcPath, dest); await fs.chmod(dest, 0o640); }
53 + return key;
54 +}
added src/proxy.ts +24 −0
@@ -0,0 +1,24 @@
1 +import { NextResponse, type NextRequest } from "next/server";
2 +/**
3 + * Locale routing. Canonical public URLs are unprefixed (English); French lives under `/fr/...`.
4 + * Internally every public page sits under an `[locale]` segment so that cached renders never leak across languages:
5 + * /papers/wp38 → rewrite → /en/papers/wp38 (x-locale: en)
6 + * /fr/papers/wp38 → as is → /fr/papers/wp38 (x-locale: fr)
7 + * /en/... → 308 redirect to the canonical unprefixed URL
8 + * Also exposes the pathname to server components (auth guard) and keeps a `lang` cookie for the UI.
9 + */
10 +const PASS = /^\/(api(?!\/docs$)|admin|login|_next|feed\.xml|sitemap\.xml|robots\.txt|favicon\.ico|brand|fonts|icon|apple-icon|opengraph-image)(\/|$)/;
11 +export function proxy(req: NextRequest) {
12 + const { pathname, search } = req.nextUrl;
13 + const h = new Headers(req.headers); h.set("x-pathname", pathname);
14 + // A rewritten request re-enters the proxy with the internal `/en/...` path: let it through untouched.
15 + if (req.headers.get("x-wp-rewritten") === "1" || (pathname.startsWith("/en/") && req.headers.get("x-locale") === "en")) return NextResponse.next({ request: { headers: h } });
16 + if (PASS.test(pathname)) { h.set("x-locale", pathname.startsWith("/fr") ? "fr" : "en"); return NextResponse.next({ request: { headers: h } }); }
17 + if (pathname === "/en" || pathname.startsWith("/en/")) return NextResponse.redirect(new URL((pathname.slice(3) || "/") + search, req.url), 308);
18 + const fr = pathname === "/fr" || pathname.startsWith("/fr/");
19 + h.set("x-locale", fr ? "fr" : "en"); h.set("x-wp-rewritten", "1");
20 + const res = fr ? NextResponse.next({ request: { headers: h } }) : NextResponse.rewrite(new URL(`/en${pathname === "/" ? "" : pathname}${search}`, req.url), { request: { headers: h } });
21 + const want = fr ? "fr" : "en"; if (req.cookies.get("lang")?.value !== want) res.cookies.set("lang", want, { path: "/", maxAge: 31536000, sameSite: "lax" });
22 + return res;
23 +}
24 +export const config = { matcher: ["/((?!_next/static|_next/image|favicon.ico).*)"] };
added src/worker/index.ts +36 −0
@@ -0,0 +1,36 @@
1 +import { config } from "dotenv";
2 +import { existsSync } from "node:fs";
3 +import { join } from "node:path";
4 +for (const f of [".env.local", ".env"]) if (existsSync(join(process.cwd(), f))) config({ path: join(process.cwd(), f), quiet: true } as never);
5 +const { sql } = await import("@/db");
6 +const { claimJob, finishJob, enqueue } = await import("@/lib/jobs/queue");
7 +const { handlers } = await import("@/lib/jobs/handlers");
8 +
9 +const POLL = Number(process.env.WORKER_POLL_MS ?? 5000);
10 +const log = (level: string, msg: string, extra: Record<string, unknown> = {}) => console.log(JSON.stringify({ t: new Date().toISOString(), level, component: "worker", msg, ...extra }));
11 +let stopping = false;
12 +process.on("SIGTERM", () => { stopping = true; log("info", "SIGTERM received, finishing current job"); });
13 +process.on("SIGINT", () => { stopping = true; });
14 +
15 +async function schedule() {
16 + // Local time of the node (America/Toronto on the cluster). Runs once per day each.
17 + const now = new Date(); const hhmm = now.getHours() * 100 + now.getMinutes(); const today = now.toISOString().slice(0, 10);
18 + const mark = async (key: string) => { await sql`INSERT INTO settings (key, value) VALUES (${key}, ${sql.json({ day: today } as never)}) ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value, updated_at = now()`; };
19 + const done = async (key: string) => { const [r] = await sql`SELECT value FROM settings WHERE key = ${key}`; return r?.value?.day === today; };
20 + if (hhmm >= 10 && !(await done("private.worker.daily_stats"))) { await enqueue("daily_stats", {}, { dedupe: true }); await mark("private.worker.daily_stats"); }
21 + if (hhmm >= 440 && !(await done("private.worker.backup"))) { await enqueue("backup", {}, { dedupe: true }); await mark("private.worker.backup"); }
22 + if (hhmm >= 300 && !(await done("private.worker.relations"))) { await enqueue("relations", {}, { dedupe: true }); await mark("private.worker.relations"); }
23 +}
24 +
25 +log("info", "worker started", { poll_ms: POLL });
26 +while (!stopping) {
27 + try {
28 + await schedule();
29 + const job = await claimJob();
30 + if (!job) { await new Promise((r) => setTimeout(r, POLL)); continue; }
31 + const t0 = Date.now(); log("info", "job start", { id: job.id, kind: job.kind, attempt: job.attempts });
32 + try { const result = await handlers[job.kind]?.(job.payload); await finishJob(job.id, true); log("info", "job done", { id: job.id, kind: job.kind, ms: Date.now() - t0, result }); }
33 + catch (e) { const msg = (e as Error).message ?? String(e); await finishJob(job.id, false, msg); log("error", "job failed", { id: job.id, kind: job.kind, error: msg.slice(0, 500) }); }
34 + } catch (e) { log("error", "loop error", { error: (e as Error).message }); await new Promise((r) => setTimeout(r, POLL)); }
35 +}
36 +await sql.end(); log("info", "worker stopped");
added tests/e2e/admin.spec.ts +17 −0
@@ -0,0 +1,17 @@
1 +import { expect, test } from "@playwright/test";
2 +/** Admin flows run only when E2E_ADMIN_EMAIL / E2E_ADMIN_PASSWORD are provided (desktop project). */
3 +const email = process.env.E2E_ADMIN_EMAIL, pw = process.env.E2E_ADMIN_PASSWORD;
4 +test.describe("admin @desktop-only", () => {
5 + test("unauthenticated access is refused", async ({ request }) => {
6 + for (const p of ["/admin", "/admin/papers", "/admin/settings"]) { const r = await request.get(p, { maxRedirects: 0 }); expect([302, 307]).toContain(r.status()); expect(r.headers().location).toContain("/login"); }
7 + const u = await request.post("/api/admin/upload", { multipart: { paper_id: "x" } }); expect(u.status()).toBe(401);
8 + });
9 + test("login, dashboard, paper edit, audit", async ({ page }) => {
10 + test.skip(!email || !pw, "no admin credentials in env");
11 + await page.goto("/login"); await page.getByLabel("E-mail").fill(email!); await page.getByLabel("Password").fill(pw!); await page.getByRole("button", { name: "Sign in" }).click();
12 + await page.waitForURL(/\/admin/); await expect(page.getByRole("heading", { name: /Dashboard|Account/ })).toBeVisible();
13 + await page.goto("/admin/papers"); await expect(page.locator("table.data tbody tr").first()).toBeVisible();
14 + await page.goto("/admin/ingest"); await expect(page.getByRole("heading", { name: "Manuscript ingestion" })).toBeVisible();
15 + await page.goto("/admin/audit"); await expect(page.locator("table.data tbody tr").first()).toBeVisible();
16 + });
17 +});
added tests/e2e/public.spec.ts +48 −0
@@ -0,0 +1,48 @@
1 +import { expect, test } from "@playwright/test";
2 +test.describe("public site", () => {
3 + test("home shows live statistics, latest papers and constellation", async ({ page }) => {
4 + await page.goto("/"); await expect(page).toHaveTitle(/Research Working Papers/); await expect(page.getByRole("heading", { level: 1 })).toHaveText("Research Working Papers");
5 + const stats = await page.request.get("/api/v1/stats").then((r) => r.json()); await expect(page.locator("dl dd").first()).toHaveText(String(stats.papers));
6 + await expect(page.getByRole("heading", { name: "Latest working papers" })).toBeVisible(); await expect(page.locator("svg[role=img]").first()).toBeVisible();
7 + });
8 + test("library filters, sorts and switches display modes", async ({ page }) => {
9 + await page.goto("/papers?domain=real-estate-economics&sort=number"); await expect(page.getByText(/results?$/).first()).toBeVisible();
10 + const n = await page.locator("article").count(); expect(n).toBeGreaterThan(3);
11 + await page.goto("/papers?view=table"); await expect(page.locator("table.data")).toBeVisible();
12 + await page.goto("/papers?view=cards&q=hedonic"); expect(await page.locator("article").count()).toBeGreaterThan(0);
13 + await page.goto("/papers?q=zzzzqqqq-nothing"); await expect(page.getByText("No paper matches these filters.")).toBeVisible();
14 + });
15 + test("paper page: metadata, citation tools, versions, related, download", async ({ page }) => {
16 + await page.goto("/papers/wp38"); await expect(page.getByRole("heading", { level: 1 })).toContainText("Economic Geography");
17 + await expect(page.locator('meta[name="citation_title"]')).toHaveAttribute("content", /Economic Geography/); await expect(page.locator('meta[name="citation_pdf_url"]')).toHaveAttribute("content", /\.pdf$/);
18 + await expect(page.locator('script[type="application/ld+json"]')).toHaveCount(1);
19 + await page.getByRole("button", { name: /Cite/ }).click(); await expect(page.getByText("Boucher, S.-P. (2026)")).toBeVisible();
20 + await expect(page.getByRole("heading", { name: "Version history" })).toBeVisible(); await expect(page.getByRole("heading", { name: "Related research" })).toBeVisible(); await expect(page.getByText(/Why related:/).first()).toBeVisible();
21 + const href = await page.getByRole("link", { name: /Download PDF/ }).first().getAttribute("href"); const r = await page.request.head(href!); expect(r.status()).toBe(200); expect(r.headers()["content-type"]).toContain("application/pdf");
22 + });
23 + test("identifier aliases redirect to the canonical slug; unknown and reserved numbers 404", async ({ page }) => {
24 + const r1 = await page.request.get("/papers/WP38", { maxRedirects: 0 }); expect(r1.status()).toBe(308); expect(r1.headers().location).toMatch(/\/papers\/wp38$/);
25 + expect((await page.request.get("/papers/wp999")).status()).toBe(404); expect((await page.request.get("/papers/wp39")).status()).toBe(404);
26 + const v = await page.request.get("/papers/wp38/v1"); expect(v.status()).toBe(200);
27 + });
28 + test("french locale renders translated chrome on the same canonical path", async ({ page }) => {
29 + await page.goto("/fr/papers/wp38"); await expect(page.locator("html")).toHaveAttribute("lang", "fr"); await expect(page.getByRole("heading", { name: "Résumé" })).toBeVisible(); await expect(page.getByRole("link", { name: /Télécharger le PDF/ }).first()).toBeVisible();
30 + await page.goto("/fr"); await expect(page.getByRole("heading", { level: 1 })).toHaveText("Cahiers de recherche");
31 + });
32 + test("search returns highlighted passages", async ({ page }) => {
33 + await page.goto("/search?q=hedonic+quebec&mode=lexical"); await expect(page.locator("ol li").first()).toBeVisible(); await expect(page.locator("mark").first()).toBeVisible();
34 + });
35 + test("discovery pages render", async ({ page }) => {
36 + for (const p of ["/topics", "/topics/real-estate-economics", "/methods", "/datasets", "/authors", "/authors/simon-pierre-boucher", "/series", "/timeline", "/research-map", "/about", "/citation", "/api/docs"]) { const r = await page.goto(p); expect(r?.status(), p).toBe(200); await expect(page.getByRole("heading", { level: 1 })).toBeVisible(); }
37 + });
38 + test("machine endpoints", async ({ request }) => {
39 + const bib = await request.get("/api/v1/papers/wp7/citation.bib"); expect(bib.status()).toBe(200); expect(await bib.text()).toContain("@techreport{");
40 + const oai = await request.get("/api/oai?verb=Identify"); expect(await oai.text()).toContain("<repositoryName>");
41 + const feed = await request.get("/feed.xml"); expect(feed.headers()["content-type"]).toContain("rss"); const sm = await request.get("/sitemap.xml"); expect(await sm.text()).toContain("/papers/wp38");
42 + const g = await request.get("/api/v1/graph").then((r) => r.json()); expect(g.nodes.length).toBeGreaterThan(30); expect(g.edges.some((e: { inferred: boolean }) => e.inferred)).toBe(true);
43 + const h = await request.get("/api/health").then((r) => r.json()); expect(h.ok).toBe(true);
44 + });
45 + test("mobile navigation works", async ({ page, isMobile }) => {
46 + test.skip(!isMobile, "mobile only"); await page.goto("/"); await page.getByRole("button", { name: "Menu" }).click(); await page.getByRole("link", { name: "Papers" }).first().click(); await expect(page).toHaveURL(/\/papers$/);
47 + });
48 +});
added tests/unit/citation.test.ts +13 −0
@@ -0,0 +1,13 @@
1 +import { describe, expect, it } from "vitest";
2 +import { bibtex, citationApa, citationChicago, citationKey, ris, type CitablePaper } from "@/lib/papers/citation";
3 +const p: CitablePaper = { identifier: "WP38", slug: "wp38", title: "The Economic Geography of Digital Retail Supply", subtitle: "Evidence from Quebec", year: 2026, authors: [{ givenName: "Simon-Pierre", familyName: "Boucher" }], seriesName: "UQO Working Papers", institution: "Université du Québec en Outaouais", url: "https://wp.spboucher.ai/papers/wp38", versionLabel: "v1.0", keywords: ["economic geography", "Quebec"], abstract: ["Para one.", "Para two."], releasedOn: "2026-11-01", releasedPrecision: "month" };
4 +describe("citations", () => {
5 + it("APA", () => { expect(citationApa(p)).toBe("Boucher, S.-P. (2026). The Economic Geography of Digital Retail Supply: Evidence from Quebec (UQO Working Papers WP38, v1.0). Université du Québec en Outaouais. https://wp.spboucher.ai/papers/wp38"); });
6 + it("APA with three authors", () => { const q = { ...p, authors: [{ givenName: "Manel", familyName: "Kammoun" }, { givenName: "Djerrycharli", familyName: "Tandjambianda" }, { givenName: "Simon-Pierre", familyName: "Boucher" }] }; expect(citationApa(q)).toContain("Kammoun, M., Tandjambianda, D., & Boucher, S.-P. (2026)"); });
7 + it("Chicago", () => { expect(citationChicago(p)).toContain("Boucher, Simon-Pierre. 2026. “The Economic Geography"); });
8 + it("BibTeX has month, number, type and escaped fields", () => { const b = bibtex(p); expect(b).toContain("@techreport{boucher2026wp38,"); expect(b).toContain("month = nov,"); expect(b).toContain("number = {WP38},"); expect(b).toContain("type = {UQO Working Papers},"); expect(b).toContain("keywords = {economic geography, Quebec},"); expect(b.endsWith("}")).toBe(true); });
9 + it("BibTeX omits month when precision is year", () => { expect(bibtex({ ...p, releasedPrecision: "year" })).not.toContain("month ="); });
10 + it("RIS", () => { const r = ris(p); expect(r.startsWith("TY - RPRT")).toBe(true); expect(r).toContain("AU - Boucher, Simon-Pierre"); expect(r).toContain("DA - 2026/11//"); expect(r).toContain("M1 - WP38"); expect(r.trim().endsWith("ER -")).toBe(true); });
11 + it("DOI wins over URL", () => { expect(citationApa({ ...p, doi: "10.1000/xyz" })).toContain("https://doi.org/10.1000/xyz"); });
12 + it("citation key strips accents", () => { expect(citationKey({ ...p, authors: [{ givenName: "É", familyName: "Côté-Léger" }] })).toBe("coteleger2026wp38"); });
13 +});
added tests/unit/extract.test.ts +28 −0
@@ -0,0 +1,28 @@
1 +import { describe, expect, it } from "vitest";
2 +import { proposeMetadata, titleSimilarity } from "@/lib/pdf/extract";
3 +import { chunkPages } from "@/lib/pdf/chunk";
4 +import { cosine, lexvec, LEXVEC_DIM, tokenize } from "@/lib/ai/lexvec";
5 +const page = `UQO Working Paper No. 41
6 +Version 1.0 — November 2026
7 +
8 +The Invisible Balance Sheet: Intraday Information and Accounting Deterioration
9 +
10 +Simon-Pierre Boucher
11 +Université du Québec en Outaouais
12 +
13 +Abstract
14 +This paper asks whether one-minute bars contain information about future accounting deterioration beyond what daily data provides. We assemble a point-in-time panel of 4,568 firms and 194,817 firm-quarters and compare information sets at matched model budgets.
15 +Keywords: high-frequency data; accounting quality; point-in-time; LightGBM
16 +JEL: G12, G14, M41
17 +1 Introduction`;
18 +describe("metadata proposal", () => {
19 + const p = proposeMetadata(page, "The Invisible Balance Sheet", "Simon-Pierre Boucher");
20 + it("detects identifier, version, date", () => { expect(p.wp).toEqual({ seriesCode: "WP", number: 41 }); expect(p.version).toBe("1.0"); expect(p.dateText).toBe("November 2026"); });
21 + it("proposes title, authors, abstract, keywords, jel", () => { expect(p.title).toContain("The Invisible Balance Sheet"); expect(p.authors).toContain("Simon-Pierre Boucher"); expect(p.abstract).toMatch(/^This paper asks/); expect(p.keywords).toEqual(["high-frequency data", "accounting quality", "point-in-time", "LightGBM"]); expect(p.jel).toEqual(["G12", "G14", "M41"]); });
22 + it("never fills fields it cannot find", () => { const q = proposeMetadata("Some random text without structure", null, null); expect(q.wp).toBeNull(); expect(q.abstract).toBeNull(); expect(q.dateText).toBeNull(); expect(q.confidence.abstract).toBe(0); expect(q.confidence.wp).toBe(0); });
23 + it("title similarity is symmetric and bounded", () => { expect(titleSimilarity("Hedonic Models for Quebec", "Quebec hedonic models")).toBeGreaterThan(0.6); expect(titleSimilarity("a", "completely different")).toBe(0); });
24 +});
25 +describe("chunking and lexical vectors", () => {
26 + it("chunks pages with page ranges and overlap", () => { const pages = [Array(40).fill("Sentence one of the paragraph is here.").join(" ") + "\n\n" + Array(40).fill("Another paragraph with content.").join(" "), Array(60).fill("Second page text.").join(" ")]; const c = chunkPages(pages, 600, 80); expect(c.length).toBeGreaterThan(2); expect(c[0].pageFrom).toBe(1); expect(c[c.length - 1].pageTo).toBe(2); expect(c.every((x) => x.text.length > 80)).toBe(true); });
27 + it("lexvec is deterministic, unit-norm and discriminative", () => { const a = lexvec("hedonic house prices quebec spatial regression"); const b = lexvec("hedonic house prices quebec spatial regression"); const c = lexvec("limit order book liquidity llm agents"); expect(a).toEqual(b); expect(a.length).toBe(LEXVEC_DIM); expect(Math.abs(cosine(a, a) - 1)).toBeLessThan(1e-9); expect(cosine(a, c)).toBeLessThan(0.3); expect(tokenize("The Of and Quebec 2026")).toEqual(["quebec"]); });
28 +});
added tests/unit/i18n.test.ts +12 −0
@@ -0,0 +1,12 @@
1 +import { describe, expect, it } from "vitest";
2 +import { en } from "@/lib/i18n/en";
3 +import { fr } from "@/lib/i18n/fr";
4 +import { fmtDate, href, pickLocale } from "@/lib/i18n";
5 +const keys = (o: Record<string, unknown>, p = ""): string[] => Object.entries(o).flatMap(([k, v]) => (v && typeof v === "object" ? keys(v as Record<string, unknown>, `${p}${k}.`) : [`${p}${k}`]));
6 +describe("i18n", () => {
7 + it("French covers every English key and vice versa", () => { expect(keys(fr).sort()).toEqual(keys(en).sort()); });
8 + it("no empty strings", () => { for (const d of [en, fr]) for (const k of keys(d)) { const v = k.split(".").reduce((a: unknown, s) => (a as Record<string, unknown>)[s], d); expect(String(v).length, k).toBeGreaterThan(0); } });
9 + it("href keeps canonical EN and prefixes FR; API/admin untouched", () => { expect(href("en", "/papers")).toBe("/papers"); expect(href("fr", "/papers")).toBe("/fr/papers"); expect(href("fr", "/")).toBe("/fr"); expect(href("fr", "/api/docs")).toBe("/api/docs"); expect(href("fr", "/admin")).toBe("/admin"); });
10 + it("date formatting honours precision and never invents a day", () => { expect(fmtDate("en", "2026-10-01", "month")).toBe("October 2026"); expect(fmtDate("fr", "2026-10-01", "month")).toBe("octobre 2026"); expect(fmtDate("en", "2026-10-01", "year")).toBe("2026"); expect(fmtDate("en", "2026-10-17", "day")).toBe("October 17, 2026"); expect(fmtDate("fr", "2026-10-17", "day")).toBe("17 octobre 2026"); expect(fmtDate("en", null)).toBe(""); });
11 + it("locale picking is strict", () => { expect(pickLocale("fr")).toBe("fr"); expect(pickLocale("de")).toBe("en"); expect(pickLocale(null)).toBe("en"); });
12 +});
added tests/unit/identifiers.test.ts +23 −0
@@ -0,0 +1,23 @@
1 +import { describe, expect, it } from "vitest";
2 +import { detectWpNumber, makeIdentifier, parseIdentifier, parseVersionLabel } from "@/lib/papers/identifiers";
3 +describe("identifiers", () => {
4 + it("builds permanent identifiers and slugs", () => {
5 + expect(makeIdentifier("WP", 7)).toEqual({ seriesCode: "WP", number: 7, identifier: "WP07", slug: "wp7" });
6 + expect(makeIdentifier("wp", 38).identifier).toBe("WP38");
7 + expect(makeIdentifier("FIRC", 2)).toEqual({ seriesCode: "FIRC", number: 2, identifier: "FIRC0002", slug: "firc2" });
8 + });
9 + it("parses user input leniently", () => {
10 + for (const s of ["wp38", "WP38", "WP038", "wp-38", "WP_38", "38", " wp 38 "]) expect(parseIdentifier(s)?.slug).toBe("wp38");
11 + expect(parseIdentifier("firc2")?.identifier).toBe("FIRC0002"); expect(parseIdentifier("FIRC0002")?.slug).toBe("firc2");
12 + expect(parseIdentifier("hello")).toBeNull(); expect(parseIdentifier("wp0")).toBeNull(); expect(parseIdentifier("")).toBeNull();
13 + });
14 + it("parses version labels", () => {
15 + expect(parseVersionLabel("v2")).toEqual({ versionNo: 2, label: "v2" }); expect(parseVersionLabel("1.1")).toEqual({ versionNo: 1, label: "v1.1" }); expect(parseVersionLabel("x")).toBeNull();
16 + });
17 + it("detects working paper numbers on title pages", () => {
18 + expect(detectWpNumber("UQO Working Paper No. 38\nNovember 2026")).toEqual({ seriesCode: "WP", number: 38 });
19 + expect(detectWpNumber("Working Paper Number 7")).toEqual({ seriesCode: "WP", number: 7 });
20 + expect(detectWpNumber("FIRC Research Paper No. 0002")).toEqual({ seriesCode: "FIRC", number: 2 });
21 + expect(detectWpNumber("No working paper here")).toBeNull();
22 + });
23 +});
added tsconfig.json +45 −0
@@ -0,0 +1,45 @@
1 +{
2 + "compilerOptions": {
3 + "target": "ES2022",
4 + "lib": [
5 + "dom",
6 + "dom.iterable",
7 + "esnext"
8 + ],
9 + "allowJs": false,
10 + "skipLibCheck": true,
11 + "strict": true,
12 + "noEmit": true,
13 + "esModuleInterop": true,
14 + "module": "esnext",
15 + "moduleResolution": "bundler",
16 + "resolveJsonModule": true,
17 + "isolatedModules": true,
18 + "jsx": "react-jsx",
19 + "incremental": true,
20 + "types": [
21 + "node"
22 + ],
23 + "plugins": [
24 + {
25 + "name": "next"
26 + }
27 + ],
28 + "paths": {
29 + "@/*": [
30 + "./src/*"
31 + ]
32 + }
33 + },
34 + "include": [
35 + "next-env.d.ts",
36 + "**/*.ts",
37 + "**/*.tsx",
38 + ".next/types/**/*.ts",
39 + ".next/dev/types/**/*.ts"
40 + ],
41 + "exclude": [
42 + "node_modules",
43 + ".next"
44 + ]
45 +}
added vitest.config.ts +6 −0
@@ -0,0 +1,6 @@
1 +import { defineConfig } from "vitest/config";
2 +import { fileURLToPath } from "node:url";
3 +export default defineConfig({
4 + test: { include: ["tests/unit/**/*.test.ts"], environment: "node" },
5 + resolve: { alias: { "@": fileURLToPath(new URL("./src", import.meta.url)) } },
6 +});
7
Menu

Theme