Replication guide (WP47)
Computation runs on the MacLustr node M3U96a in ~/wp47_uqo (FinData OS API on http://127.0.0.1:8600); the laptop folder
~/Desktop/wp47-nelson-siegel-intraday holds the code, the public inputs, the results and the LaTeX. Code goes node-ward with
rsync -a --exclude data/cache --exclude .git --exclude paper . M3U96a:~/wp47_uqo/; results come back with
rsync -a M3U96a:~/wp47_uqo/results/ results/.
Environment (node)
cd ~/wp47_uqo && uv venv -p 3.12 .venv
uv pip install -p .venv/bin/python numpy pandas scipy statsmodels polars pyarrow matplotlib requests pyyaml archAPI key: ~/findata-secrets/api-key-wp47.txt on the node (laptop copy ~/Desktop/Cluster/secrets/findata/api-key-wp47.txt),
or $FINDATA_API_KEY; scopes read:market, query, export. On the laptop set FINDATA_URL=https://findata.spboucher.ai.
Never print or commit the key.
Public inputs (data/external/, download timestamp in DOWNLOADED_AT.txt = 2026-11-09T19:08:56Z)
fred/DGS1MO … DGS30.csv,fred/DFF.csv— FRED H.15 constant-maturity yields and effective fed funds (fredgraph CSV).fed/feds200628.csv— Gürkaynak–Sack–Wright zero-coupon curve and Svensson parameters (Federal Reserve Board).fomc/fomchistorical2008–2020.html,fomc/fomccalendars.html→fomc_dates_parsed.json(meeting end dates, statement dates).bls/cpi_release_dates_alfred_rid10.txt,bls/empsit_release_dates_alfred_rid50.txt— release-date lists read from the ALFRED release pages (rid 10 = CPI, rid 50 = Employment Situation) on 2026-11-09; revision releases are filtered inevents.py.
Pipeline (run in this order on the node; times measured on M3U96a, 2026-11-09)
| Script | Output | Time |
|---|---|---|
01_extract_futures.py |
data/cache/futures_1m/{ROOT}_{a}_{b}.parquet (vendor continuous series, kept for the EDA and SR3 roll check), futures_1d/continuous_{raw,ratio}_*.parquet |
~2 s per export block |
01b_extract_sr3.py |
futures_1m/SR3_2018_2026.parquet, futures_1d/*_SR3_*, returns_1m_SR3.parquet, sr3_daily.parquet |
~10 s |
01c_extract_etf.py |
etf_1m/etf_{2010_2018,2019_2026}.parquet (SHY IEI IEF TLH TLT, split/dividend adjusted) |
~5 s |
01d_extract_contracts.py |
data/cache/contracts_1m/{ROOT}_{a}_{b}.parquet — all individual contracts of the six roots (18 exports, 326 MB; dataset market_bars.futures_contracts.1m.raw) |
~5.6 min |
02_daily_factors.py |
results/metrics/daily_factors_{cmt,cmt_lambdahat,gsw}.csv, daily_factors_summary.json |
~5 s |
03_returns.py |
data/cache/returns_1m.parquet (most-traded contract per session, act_* flags), returns_daily.parquet, results/metrics/returns_diagnostics.json |
~15 s |
04_mimicking.py |
mimicking_weights_quarterly.json, mimicking_fidelity.json, data/cache/mimicking_oos_daily.parquet |
~2 s |
04b_fidelity_gsw_and_sofr.py |
mimicking_fidelity_robustness.json, daily_factors_cmt_1y_plus.csv |
~3 s |
04c_targets_and_spanning.py |
targets_and_spanning.json, daily_factors_cmt_2y_plus.csv |
~5 s |
04d_robustness_checks.py |
robustness_checks.json (expanding window, 15:00/16:00 boundaries, 5-min denominators, FOMC release-minute check) |
~2 s |
05_events_list.py |
results/metrics/events.csv (runs anywhere) |
<1 s |
06_intraday_factors.py |
data/cache/factors_1m.parquet, factors_daily_realized.parquet (+ csv), realized_correlations.json, intraday_periodicity.csv, fidelity_cumulated_intraday.json |
~10 s |
06b_etf_crossval.py |
data/cache/factors_1m_etf.parquet, etf_crossval.json |
~60 s |
06c_noise.py |
noise_benchmark.json |
<1 s |
07_events.py |
event_windows.csv, event_windows_placebo.csv, event_summary.json, event_paths.json |
~8 s |
08_har.py |
har_forecasts.json, data/cache/har_preds_{L,S,C}_h{1,5,22}.parquet |
~30 s |
13_irf.py |
irf.json, data/cache/lp_events.parquet, results/figdata/{lp_irf,var_girf,vol_connectedness_rolling}.csv, paper/tables/tab_lp.tex, tab_connect.tex (copy the two .tex files back to the laptop) |
— (not timed) |
09a_export_figure_data.py |
results/figdata/*.csv (sample days, fidelity by year with bands, event paths with bootstrap bands, heatmap matrices, ETF pairs) |
~10 s |
laptop 11_tables.py (python3 -I) |
paper/tables/*.tex (except tab_lp/tab_connect), paper/numbers.tex, docs/NUMBERS_INDEX.md |
— |
laptop 12_text_macros.py (python3 -I) |
paper/numbers_text.tex (text macros, \providecommand), results/metrics/implied_yield_fidelity.json |
— |
laptop 10_figures.py (--audit renders colour + gray PNGs) and 14_irf_figures.py |
paper/figures/*.pdf (12 figures) |
— |
Then cd paper && latexmk -pdf main.tex on the laptop (45 pages for v1.0-rc). Macro names in numbers*.tex must not contain digits.
00_eda_futures.py is an optional data-quality scan (data/cache/eda_futures.json). Then cd paper && latexmk on the laptop.
Determinism: the only random element is the bootstrap/placebo sampling in 07_events.py (numpy.random.default_rng(47)).
Dataset versions are pinned by the FinData export (version field in data/cache/extract_log.jsonl: futures continuous 1m v132, daily v133; contracts 1m v19699 printed by 01d). Random elements: bootstrap/placebo sampling in 07_events.py (default_rng(47)), 13_irf.py (default_rng(13)), 09a bands.
Licence constraint
FirstRate Data (FinData OS) is licensed for private use. data/cache/ (raw and derived minute data, including contracts_1m/) stays on the node and is
git-ignored; only aggregates (daily realized measures, event-window sums, weights, figure data) leave the node and enter the paper.