data/ — empirical inputs for WP 24 (“From Labor to Capital: AGI, General Equilibrium, and Asset Valuation”)
Retrieved 2026-10-27, key-less endpoints only, no number typed from memory. Two scripts rebuild everything:
bash
python3 -I scripts/01_download_data.py # raw downloads -> data/raw/<source>/ + data/raw/manifest.json (idempotent; --force to refetch)
python3 -I scripts/02_build_calibration_targets.py # -> data/processed/calibration_targets.csv, data/processed/series/*.csv, bea_io_value_added_2023.csv, epoch_price_performance_fit.csvDependencies: requests, pandas, numpy, openpyxl, xlrd (installed 2026-10-27 for the Shiller .xls files), PyMuPDF (fitz, to read the LBNL PDF).
Raw files are untrusted data and are never executed. data/raw/ and data/processed/ are git-ignored (regenerate them); only the two READMEs are versioned.
Layout
| Path | Content |
|---|---|
raw/README.md |
one block per source: what, URL, size, vintage, licence/terms |
raw/manifest.json |
per-file URL, retrieval date, size, SHA-256, HTTP content type, status (93 entries, all RETRIEVED) |
raw/fred/_metadata.json, raw/fred/_RETRIEVAL_NOTE.md |
FRED titles/units/vintages; why the FRED csv files were fetched with WebFetch |
processed/calibration_targets.csv |
162 rows: target_id, description, value, unit, year_or_period, geography, frequency, source, series_id, url, raw_file, retrieved, status, notes |
processed/series/*.csv |
92 tidy date,value series (every FRED series, Shiller, DFA, Treasury, Cleveland Fed, StatCan, BLS API) |
processed/bea_io_value_added_2023.csv |
71 BEA summary industries: compensation, taxes, gross operating surplus, value added, intermediates, gross output, labor share, intermediate share |
processed/epoch_price_performance_fit.csv |
60 accelerators used in the FLOP/s-per-dollar regression |
Targets and transformations (US unless noted; “avg” = mean over 1990–latest full year of calendar-year means)
| Block | target_id prefix | Construction |
|---|---|---|
| 1 Labor share | T01_labor_share_BLS_nonfarm_index_* |
BLS PRS85006173, index 2017=100 (not a level). Latest 2026Q2 = 93.4; 2025 = 96.6; avg 103.9 |
T01_labor_share_COE_GDI_* |
COE / GDI (both $bn SAAR). Latest 0.499 (2026Q2); 2025 0.508; avg 0.542 | |
T01_labor_share_COE_NDI_* |
COE / net domestic income W256RC1Q027SBEA. Latest 0.597; avg 0.642 | |
T01_labor_share_PWT_US_*, T12_labor_share_PWT_CA_* |
PWT labsh via FRED (PWT 11.0, to 2023): US 0.568 (2023), avg 0.604; Canada 0.659, avg 0.663. PWT 10.01 spreadsheet cross-check (*_PWT1001_*, to 2019): US 0.597, CA 0.655 |
|
T01_labor_share_BEA_IO_2023_* |
2023 summary use table: Σ compensation / Σ value added = 0.513 (all 71 industries incl. government); compensation/(compensation+GOS) = 0.548; intermediates / gross output = 0.427; selected industries (utilities 0.21, computer systems design 0.91, publishing incl. software 0.23, computer & electronic products 0.57, construction 0.61, professional services 0.74, ambulatory health 0.77) | |
| 2 Profits | T02_profits_after_tax_noIVA_GDP_* |
FRED CP = after tax without IVA/CCAdj: 0.131 (2026Q2), 2025 0.114, avg 0.086 |
T02_profits_after_tax_IVA_CCAdj_GDP_* |
FRED CPATAX = with IVA & CCAdj: 0.119, 2025 0.110, avg 0.084 | |
T02_net_dividends_GDP_* |
DIVIDEND / GDP: 0.0675 (2026Q2), 2025 0.070, avg 0.051; annual B056RC1A027NBEA/GDPA 2025 0.070 | |
T02_GDP_*, T02_real_GDP_growth_avg |
GDPA 2025 = 30,861 $bn; GDP 2026Q2 = 32,563 $bn SAAR; GDPC1 2026Q2 = 24,408; real growth 1990–2025 = 2.48 %/yr (log) | |
| 3 Household wealth | T03_household_net_worth_GDP_* |
TNWBSHNO ($mn, end of quarter) / GDP ($bn SAAR) ×1e-3: 6.02 (2026Q2), 2025 5.72, avg 4.58 |
T03_household_real_estate_GDP_*, T03_owners_equity_real_estate_GDP_*, T03_owners_equity_share_of_real_estate_*, T03_real_estate_share_of_net_worth_* |
HNOREMQ027S/GDP 1.66 (avg 1.47); OEHRENWBSHNO/GDP 1.10 (avg 0.81); HOEREPHRE 71.9 % (avg 60.7 %); real estate/net worth 0.276 (avg 0.322) | |
| 4 Wealth concentration | T04_wealth_share_* |
DFA dfa-networth-shares.csv, Net worth column, 2026Q2: top 0.1 % 15.0, top 1 % 32.5, top 10 % 68.9, p50–90 28.8, bottom 50 % 2.3 (percent); 1990–2025 averages 11.3 / 28.0 / 65.3 / 32.4 / 2.3. FRED mirrors WFRBST01134 / WFRBSN09161 / WFRBSN40188 / WFRBSB50215 agree. Top 10 % hold 88.1 % of corporate equities & mutual fund shares |
| 5 Investment | T05_*_GDP_* |
GPDI/GDP 0.179 (avg 0.175); PNFI/GDP 0.145 (avg 0.130); information-processing equipment Y034RC1Q027SBEA/GDP 0.0257 (avg 0.0214); software B985RC1Q027SBEA/GDP 0.0249 (avg 0.0154); IPE+software A679RC1Q027SBEA/GDP 0.0505; IPP Y001RC1Q027SBEA/GDP 0.0577 |
| 6 Capital | T06_net_stock_private_fixed_assets_GDP_* |
current-cost net stock K1PTOTL1ES000 (end-2024, $mn)/GDPA ×1e-3 = 2.39 (avg 2.25); nonresidential K1NTOTL1ES000: 1.22 (avg 1.20) |
T06_depreciation_rate_* |
current-cost depreciation / end-of-year current-cost net stock: private total M1PTOTL1ES000/K1PTOTL1ES000 = 0.0568 (2024; avg 0.0559); nonresidential 0.0890 (avg 0.0843); NIPA private CFC/net stock 0.0569 | |
T06_CFC_* |
COFC/GDP 0.165 (avg 0.155); private CFC A024RC1A027NBEA/GDPA 0.137 (2025) | |
| 7 Real rate | T07_TIPS10_* |
DFII10 daily: 2.66 % (2026-10-23), 2025 mean 1.96 %, 2003–2025 mean 0.97 %; Treasury.gov daily real curve 10 YR identical (1.963 / 0.973); Cleveland Fed REAINTRATREARAT10Y 2.47 % (2026-10), 2025 1.71 %, 1990–2025 avg 1.78 %; GS10 nominal 4.99 % (2026-09), 2025 4.29 % |
| 8 Equity valuation | T08_* |
Shiller ie_data.xls: CAPE 30.8 (2023-09), 1990–2022 avg 26.4, 1881–2022 avg 17.3; D/P 0.0158 (2023-06), avg 0.0203; E/P 0.0417, avg 0.0461. Vintage caveat: the Yale file ends 2023-09 |
| 9 Housing | T09_* |
Shiller real HPI (1890=100) 209.3 (2023-01); real growth 1.54 %/yr 1990–2023, 0.56 %/yr 1890–2023. Case-Shiller national nominal growth 4.15 %/yr 1990–2025, real (CPI-U) 1.63 %/yr 1990–2024; CPI rent growth 3.27 %/yr. Rent-price proxy = (CPI rent / Case-Shiller), both rebased Jan-2000 = 100: 74.8 (2026-07), avg 91.1 — an index of the change in the rent-price ratio since 2000, not a level |
| 10 Energy | T10_* |
LBNL 2024 report (PDF p. 7 [printed 5] and p. 53 [printed 52]): data centers 176 TWh in 2023 = 4.4 % of US electricity consumption; 76 TWh = 1.9 % in 2018; 2028 scenarios 325–580 TWh = 6.7–12.0 % (p. 8 [6], p. 53 [52]). EIA Table 1.1: net generation 4,425 TWh (2025), 4,309 (2024); retail sales 4,080 TWh (2025), 3,975 (2024) |
| 11 Compute | T11_* |
Epoch ml_hardware.csv, column Price-performance (= Max performance FLOP/s ÷ release price USD), 60 accelerators 2008–2026: OLS of log2 on release year gives a doubling time of 2.02 years (R² in notes), 2.02 years on 2015+. Caveat row: Max performance mixes precisions (FP32 for old GPUs, tensor FP16/FP8 for recent ones) |
| 12 Canada | T12_* |
StatCan 36-10-0103-01: COE / GDP at market prices (SAAR) 0.492 (2026Q2), avg 0.506; 36-10-0580-01: households+NPISH net worth (market value) / GDP 5.65 (2026Q2), avg 4.25 |
| 13 Electricity price | T13_* |
BLS APU000072610: 0.196 $/kWh (2026-09), 2025 mean 0.185, 1990–2025 avg 0.118 |
| cross-checks | T00_crosscheck_BLS_API_* |
same three BLS series pulled through the BLS API v1: identical to the FRED copies |
Conventions: ratios of a quarterly stock (end of quarter) to quarterly GDP use SAAR GDP of the same quarter; annual Fixed-Assets stocks (end of year) are divided by annual GDP of the same year; depreciation rates divide depreciation during year t by the end-of-year-t stock (not the lagged stock). “Latest full calendar year” requires 4 quarterly / 11–12 monthly / ≥240 daily observations (11 because BLS published no October-2025 CPI/APU values).
Explicit list of what could NOT be retrieved
- No calibration target is NOT_RETRIEVED (161 RETRIEVED, 1 INFO row for the Epoch caveat).
- Access paths that failed and were replaced (documented in
raw/README.md,raw/fred/_RETRIEVAL_NOTE.md,raw/manifest.json):- FRED by direct HTTP (
requests/urllib/curl) — fred.stlouisfed.org timed out from this network on 2026-10-27 (also from the OVH server); the 43 csv files are verbatim copies obtained with the Claude Code WebFetch tool;01_download_data.py --forcere-downloads them when FRED answers. - LBNL host for the data-center report (HTTP 202 challenge page) — replaced by the eScholarship copy of the same report (DOI 10.71468/P1WC7Q).
- BEA per-file xlsx URLs (
apps.bea.gov/industry/xls/io-annual/...) return “page not found” — replaced by BEA's ownAllTablesIO.zip, which contains the sameIOUse_After_Redefinitions_PRO_1997-2023_Summary.xlsx. - US Treasury “all years” csv returns 403 — fetched year by year instead (2003–2026).
- FRED by direct HTTP (
- Known limitations rather than failures: Shiller
ie_data.xlsandFig3-1.xlson the Yale site end in 2023 (CAPE, D/P, E/P and the long real home-price index are therefore 2023 vintages; Case-Shiller via FRED runs to 2026-07); the rent-price ratio is a proxy index (no key-less level series); EIA direct consumption/losses are not in Table 1.1 (retail sales used); the Epoch doubling time is our own regression on Epoch's columns, not Epoch's published estimate; CSUSHPISA values are not redistributed in the targets (ratios only).