SPB Git

spb/wp17_uqo

Public

FIRC0002-ECON-01 EDIT-01 ECON-02 ADVE-09 THEO-08 EDIT-10 DATA-04 (L3), EDIT-08 THEO-16 DATA-12 (L2), L4/L5/L6/L7/L9/L10 scripts: v2 chain (panel, core, extended, event study, survivorship, figures, tables), run_all.sh, regeneration test, lock file

Market cap = FinData raw close x point-in-time SEC shares (split-carried, unit-error screens: whole-series 30x, +/-4 filings 5x,
IWV-weight consistency, USD 5M floor, 6e12 ceiling, 3-month staleness); Yahoo-implied raw as fallback (98.9% within 2%).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Nov 5, 2026) parent cf83735 Browse files

10 changed files +661 −21

added data/processed_r3k_v2/README.md +17 −0
@@ -0,0 +1,17 @@
1 +# data/processed_r3k_v2 — derived files of the Stage D review (FIRC-0002 v2.1), regenerated by `scripts/run_all.sh`
2 +
3 +Not committed (size); every file is produced by a script in `scripts/` from the staged author inputs (`data/raw_russell3000/`,
4 +`data/processed_r3k/`, 2026-10-28 vintage, sha256 in `UQO-WP-REVIEW/reproducibility/FIRC-0002/environment/inputs.sha256`) and
5 +public sources. Deposit (author action): archive this directory + `data/processed_r3k/` + `data/raw_russell3000/` (≈ 420 MB) as an
6 +spbgit release or an M3U96b archive and cite its sha256 in the README; the EDGAR chunk texts and embeddings are not redistributed.
7 +
8 +| file | producer | content | source / licence |
9 +|---|---|---|---|
10 +| prices_monthly_unadj.parquet, prices_daily_v2.parquet, benchmarks_*_v2.parquet, splits.parquet | r3k_10 | Yahoo Finance via yfinance (Close is split-adjusted, dividend-unadjusted; Adj Close total return); download date in download_v2.log | Yahoo Finance terms of use (personal/research use, no redistribution) |
11 +| ff_daily.parquet | r3k_10 | Ken French daily FF5 + MOM | Kenneth R. French Data Library (free, cite) |
12 +| sec_shares_raw.parquet, shares_history_v2.parquet | r3k_12 | SEC XBRL companyfacts (dei:EntityCommonStockSharesOutstanding + us-gaap fallbacks), per-class rows and the cleaned per-filing count | SEC EDGAR (public domain), fair-use rate |
13 +| prices_monthly_findata_raw.parquet | r3k_13b | FinData OS (FirstRate) unadjusted month-end closes, read-only SQL | FirstRate licence (private; not redistributable — document access) |
14 +| iwv_holdings/*.json | manual (manifest results/r3k_v2/iwv_holdings_manifest.json) | dated iShares IWV holdings files captured by the Internet Archive | iShares/BlackRock terms; archive.org |
15 +| monthly_returns_v2.parquet, merged_panel_v2.parquet, analysis_panel_v2.parquet, panel_meta_v2.json | r3k_14, r3k_15 | rebuilt monthly return file (hygiene flags, corrected caps) and firm-year panels | derived |
16 +| ls_ts_v2_*.parquet, event_car_v2.parquet, event_car_paths_v2.parquet | r3k_15, r3k_17 | long–short monthly series; event-study firm CARs and paths | derived |
17 +| spid_sample_sections_L8.parquet | r3k_22 (texts read from the N1 copy of filing_sections on M4M64a) | source texts of the 200 sampled SPID mentions | EDGAR filings (public) |
added requirements.lock +50 −0
@@ -0,0 +1,50 @@
1 +anyio==4.16.0
2 +beautifulsoup4==4.15.0
3 +certifi==2026.10.23
4 +cffi==2.1.1
5 +charset-normalizer==3.5.2
6 +contourpy==1.4.0
7 +curl-cffi==0.17.0
8 +cycler==0.12.1
9 +cython==3.3.1
10 +duckdb==2.0.0
11 +fonttools==4.68.0
12 +formulaic==1.2.2
13 +frozendict==2.5.1
14 +h11==0.16.0
15 +httpcore==1.0.9
16 +httpx==0.28.1
17 +idna==3.20
18 +interface-meta==2.0.1
19 +kiwisolver==1.5.1
20 +linearmodels==6.1
21 +matplotlib==3.9.4
22 +multitasking==0.0.13
23 +mypy-extensions==1.2.0
24 +narwhals==2.28.0
25 +numpy==2.1.3
26 +packaging==26.3
27 +pandas==2.2.3
28 +patsy==1.0.1
29 +peewee==4.5.3
30 +pillow==13.0.0
31 +platformdirs==4.14.5
32 +protobuf==7.36.2
33 +pyarrow==17.0.0
34 +pycparser==3.12
35 +pyhdfe==0.2.1
36 +pyparsing==3.3.3
37 +python-dateutil==4.0.0
38 +pytz==2026.6
39 +requests==2.32.5
40 +scipy==1.14.1
41 +setuptools==84.2.1
42 +setuptools-scm==8.3.1
43 +soupsieve==4.0.1
44 +statsmodels==0.14.6
45 +typing-extensions==4.16.0
46 +tzdata==2026.6
47 +urllib3==2.8.0
48 +websockets==17.4
49 +wrapt==2.5.2
50 +yfinance==0.2.65
modified scripts/r3k_12_sec_shares_v2.py +12 −3
@@ -45,19 +45,28 @@ def one(cik):
45 45 rows = [r for r in rows if r[8] == "shares" and r[4] is not None and r[4] > 0]
46 46 if rows: out = rows; src = tax + ":" + concept; break
47 47 return cik, out, src
48 +import sys
49 +REAGG = "--reaggregate" in sys.argv
48 50 rows, fail, done, srcs = [], 0, 0, {}
51 +if REAGG: raw = pd.read_parquet(f"{OUT}/sec_shares_raw.parquet"); srcs = raw.drop_duplicates("cik").set_index("cik")["concept"].to_dict(); print("re-aggregating", len(raw), "stored rows (no network)", flush=True)
49 52 with ThreadPoolExecutor(4) as ex:
53 + if not REAGG:
50 54 for f in as_completed([ex.submit(one, c) for c in ciks]):
51 55 cik, rr, src = f.result(); done += 1
52 56 if rr is None: fail += 1
53 57 else: rows += rr; srcs[cik] = src
54 58 if done % 250 == 0: print(f"{done}/{len(ciks)} fail={fail}", dt.datetime.now().strftime("%H:%M:%S"), flush=True)
55 −raw = pd.DataFrame(rows, columns=["cik", "concept", "filed", "end", "shares", "form", "fy", "fp", "unit"]); raw["filed"] = pd.to_datetime(raw["filed"]); raw["end"] = pd.to_datetime(raw["end"])
56 −raw.to_parquet(f"{OUT}/sec_shares_raw.parquet", index=False)
59 +if not REAGG:
60 + raw = pd.DataFrame(rows, columns=["cik", "concept", "filed", "end", "shares", "form", "fy", "fp", "unit"]); raw["filed"] = pd.to_datetime(raw["filed"]); raw["end"] = pd.to_datetime(raw["end"])
61 + raw.to_parquet(f"{OUT}/sec_shares_raw.parquet", index=False)
57 62 # aggregation: per filing, classes at the latest `end`
58 63 raw = raw.sort_values(["cik", "filed", "end"])
59 64 latest_end = raw.groupby(["cik", "filed"])["end"].transform("max"); le = raw[raw["end"] == latest_end]
60 −agg = le.groupby(["cik", "filed"]).agg(shares_sum=("shares", lambda s: float(np.unique(s.values).sum())), n_classes=("shares", lambda s: int(len(np.unique(s.values)))), concept=("concept", "first")).reset_index()
65 +# classes are summed ONLY for the instant cover-page concept (dei, one value per share class); the us-gaap fallbacks are duration
66 +# facts (quarter and year-to-date values share the same `end`) -> max, never sum
67 +le = le.copy(); le["is_dei"] = le["concept"].str.startswith("dei:")
68 +agg = le.groupby(["cik", "filed"]).agg(shares_sum=("shares", lambda s: float(np.unique(s.values).sum())), shares_mx=("shares", "max"), n_classes=("shares", lambda s: int(len(np.unique(s.values)))), concept=("concept", "first"), is_dei=("is_dei", "first")).reset_index()
69 +agg.loc[~agg.is_dei, "shares_sum"] = agg.loc[~agg.is_dei, "shares_mx"]; agg.loc[~agg.is_dei, "n_classes"] = 1; agg = agg.drop(columns=["shares_mx", "is_dei"])
61 70 mx = raw.groupby(["cik", "filed"])["shares"].max().rename("shares_max").reset_index()
62 71 agg = agg.merge(mx, on=["cik", "filed"]).merge(m, on="cik", how="inner").rename(columns={"filed": "date"})
63 72 agg["shares_source"] = agg["concept"]; agg = agg[["ticker", "cik", "date", "shares_sum", "shares_max", "n_classes", "shares_source"]].sort_values(["ticker", "date"])
modified scripts/r3k_14_build_panel_v2.py +77 −13
@@ -28,7 +28,8 @@ unadj = pd.read_parquet(f"{V}/prices_monthly_unadj.parquet"); sh = pd.read_parqu
28 28 mc_v1 = pd.read_parquet(f"{D}/marketcap.parquet"); sh_v1 = pd.read_parquet(f"{D}/shares_history.parquet")
29 29 hyg = pd.read_csv(f"{T}/hygiene_log.csv"); spid = pd.read_csv(f"{R}/spid_firm_year_signals.csv"); v1 = pd.read_parquet(f"{D}/merged_panel.parquet")
30 30 META = {}
31 −def mend(s): return pd.to_datetime(s).dt.to_period("M").dt.to_timestamp("M")
31 +def mend(s): return pd.to_datetime(s).dt.to_period("M").dt.to_timestamp("M").astype("datetime64[ns]")
32 +def ns(s): return pd.to_datetime(s).astype("datetime64[ns]")
32 33
33 34 # ---------------- monthly returns (author's construction) + hygiene [B1] ----------------
34 35 px = px.sort_values(["ticker", "date"]).copy(); px["date"] = mend(px["date"]); px = px.drop_duplicates(["ticker", "date"], keep="last")
@@ -41,19 +42,82 @@ px.loc[px.splice, "ret"] = np.nan; META["n_monthly_removed_second_source"] = int
41 42 unadj["date"] = mend(unadj["date"]); unadj = unadj.drop_duplicates(["ticker", "date"], keep="last")[["ticker", "date", "close_unadj"]]
42 43 px = px.merge(unadj, on=["ticker", "date"], how="left"); META["share_months_with_unadj_close"] = round(float(px.close_unadj.notna().mean()), 4)
43 44 # shares [A1]: point-in-time, backward as-of on filing date
44 −sh = sh.sort_values(["ticker", "date"]); sh["date"] = pd.to_datetime(sh["date"]); sh = sh.drop_duplicates(["ticker", "date"], keep="last")
45 +sh = sh.sort_values(["ticker", "date"]); sh["date"] = ns(sh["date"]); sh = sh.drop_duplicates(["ticker", "date"], keep="last")
46 +# [A1e] XBRL unit errors in the cover-page count (LAMR 2013-14, TXNM 2020, CPK 2025 report shares x 1,000): a count that jumps by more
47 +# than 5x (or falls below 0.2x) relative to the previous filing AND reverts at the next filing is an error, as is an unsupported jump
48 +# at the end of the series (no split event within +/- 6 months, splits.parquet); erroneous counts are set to missing so that the as-of
49 +# merge carries the previous valid count. Permanent jumps (splits, large issuances) are kept.
50 +_sp = pd.read_parquet(f"{V}/splits.parquet"); _sp["date"] = ns(_sp["date"]); _spd = {t: g["date"].values for t, g in _sp.groupby("ticker")}
51 +# rules (applied in this order, each with a split guard = a split event of ratio >= 2 or <= 0.5 within 6 months of the filing):
52 +# (1) a count more than 30x above or below the ticker's whole-series median count is a unit error (x 1,000 reports: LAMR 2013-14,
53 +# TXNM 2020, CPK 2025, ENSG / MEI 2010) — runs of several consecutive erroneous filings are caught because they stay a
54 +# minority of the series; (2) a one-off count more than 5x above/below the median of the surrounding +/- 4 filings;
55 +# (3) series-level check: the October-2026 cap implied by the cleaned count vs the firm's IWV weight (companies.russell_weight_pct,
56 +# 2026-10-23): a ratio to the cross-sectional median outside [1/20, 20] marks the whole series as unusable (shares -> missing).
57 +def near_split(row):
58 + d = _spd.get(row.ticker); return bool(d is not None and (np.abs((d - row.date.to_datetime64()).astype("timedelta64[D]").astype(int)) <= 183).any())
59 +_sp_big = _sp[(_sp.ratio >= 2) | (_sp.ratio <= 0.5)]; _spd = {t: g["date"].values for t, g in _sp_big.groupby("ticker")}
60 +sh["near_split"] = sh.apply(near_split, axis=1)
61 +med_all = sh.groupby("ticker")["shares_sum"].transform("median"); r_all = sh["shares_sum"] / med_all; err1 = ((r_all > 30) | (r_all < 1 / 30)) & ~sh["near_split"]
62 +sh.loc[err1, "shares_sum"] = np.nan
63 +def rolling_med_excl(g):
64 + v = g.values; out = np.full(len(v), np.nan)
65 + for i in range(len(v)):
66 + win = np.r_[v[max(0, i - 4):i], v[i + 1:i + 5]]; win = win[~np.isnan(win)]
67 + if len(win) >= 2: out[i] = np.median(win)
68 + return pd.Series(out, index=g.index)
69 +med4 = sh.groupby("ticker")["shares_sum"].transform(rolling_med_excl); r4 = sh["shares_sum"] / med4; err2 = ((r4 > 5) | (r4 < 0.2)) & ~sh["near_split"] & med4.notna() & sh["shares_sum"].notna()
70 +sh.loc[err2, "shares_sum"] = np.nan
71 +META["n_sec_share_counts_flagged_unit_error"] = int(err1.sum() + err2.sum()); META["tickers_sec_share_unit_error"] = sorted(sh.loc[err1 | err2, "ticker"].unique().tolist())
72 +sh = sh.dropna(subset=["shares_sum"]).drop(columns=["near_split"])
45 73 px = pd.merge_asof(px.sort_values("date"), sh[["ticker", "date", "shares_sum", "shares_max", "shares_source"]].rename(columns={"date": "shares_date"}).sort_values("shares_date"), left_on="date", right_on="shares_date", by="ticker", direction="backward").sort_values(["ticker", "date"])
46 74 px["shares_age_m"] = (px["date"] - px["shares_date"]).dt.days / 30.44
47 −px["mktcap"] = px["shares_sum"] * px["close_unadj"]
75 +# [A1b] carry the point-in-time count through splits that occur between the cover-page date and the price date (yfinance split
76 +# history, r3k_10): shares_pit = shares_sum x cumsplit(price date) / cumsplit(cover date). Without this a 1-for-k reverse split
77 +# leaves a k-fold overstated cap (pre-split count x post-split price) until the next 10-K/10-Q cover page (SLS 2014, ON 2012 ...).
78 +sp = pd.read_parquet(f"{V}/splits.parquet"); sp["date"] = ns(sp["date"]); sp = sp[(sp.ratio > 0) & np.isfinite(sp.ratio)].sort_values(["ticker", "date"]); sp["cum"] = sp.groupby("ticker")["ratio"].cumprod()
79 +def cum_at(dates, t, when):
80 + g = sp[sp.ticker == t]
81 + if g.empty: return np.ones(len(dates))
82 + idx = np.searchsorted(g["date"].values, dates.values, side="right"); c = np.r_[1.0, g["cum"].values]; return c[idx]
83 +px["cum_price"] = 1.0; px["cum_shares"] = 1.0
84 +for t, g in px.groupby("ticker"):
85 + if t in set(sp.ticker):
86 + px.loc[g.index, "cum_price"] = cum_at(g["date"], t, "price"); sd = g["shares_date"].fillna(g["date"]); px.loc[g.index, "cum_shares"] = cum_at(sd, t, "shares")
87 +# Yahoo "Close" (auto_adjust=False) is split-adjusted to the download date and only dividend-unadjusted (verified: AAPL June 2011
88 +# Close 11.99 = 335 / 28). Market cap on that basis = Close_yahoo x shares on the current split basis
89 +# = Close_yahoo x shares_cover x cum_total / cum_at(cover date) == raw close x point-in-time shares (cross-checked against FinData raw bars in r3k_13b).
90 +tot = sp.groupby("ticker")["cum"].last(); px["cum_total"] = px["ticker"].map(tot).fillna(1.0)
91 +px["shares_pit"] = px["shares_sum"] * px["cum_price"] / px["cum_shares"] # point-in-time count on the price-date split basis (for reference)
92 +px["shares_nowbasis"] = px["shares_sum"] * px["cum_total"] / px["cum_shares"] # count on the current split basis (matches Yahoo Close)
93 +# [A1c] backfill for months before the first cover-page count (pre-XBRL years of smaller filers): the first SEC count carried back
94 +# through splits, flagged shares_backfilled (the author's fallback was the constant 2026 count)
95 +first = sh.sort_values("date").groupby("ticker").first().rename(columns={"date": "first_date", "shares_sum": "first_shares"})[["first_date", "first_shares"]]
96 +px = px.merge(first, left_on="ticker", right_index=True, how="left"); px["shares_backfilled"] = px["shares_sum"].isna() & px["first_shares"].notna()
97 +for t, g in px[px.shares_backfilled].groupby("ticker"):
98 + cb = cum_at(g["first_date"], t, "shares") if t in set(sp.ticker) else np.ones(len(g)); px.loc[g.index, "shares_pit"] = g["first_shares"].values * g["cum_price"].values / cb; px.loc[g.index, "shares_nowbasis"] = g["first_shares"].values * g["cum_total"].values / cb
99 +# [A1d] canonical price = FinData OS raw (unadjusted) month-end close (r3k_13b) x point-in-time shares on the price-date basis;
100 +# the Yahoo-implied version (split-adjusted Close x shares on the current basis) is the fallback where FinData has no bar and the
101 +# cross-check elsewhere (mktcap_src, mktcap_yahoo kept).
102 +fdr = pd.read_parquet(f"{V}/prices_monthly_findata_raw.parquet"); fdr["date"] = ns(fdr["date"]); fdr = fdr.drop_duplicates(["ticker", "date"], keep="last")[["ticker", "date", "close_raw"]]
103 +px = px.merge(fdr, on=["ticker", "date"], how="left")
104 +px["mktcap_yahoo"] = px["shares_nowbasis"] * px["close_unadj"]; px["mktcap_findata"] = px["shares_pit"] * px["close_raw"]
105 +px["mktcap"] = px["mktcap_findata"].where(px["mktcap_findata"].notna(), px["mktcap_yahoo"]); px["mktcap_src"] = np.where(px["mktcap_findata"].notna(), "findata_raw", np.where(px["mktcap_yahoo"].notna(), "yahoo_implied", "none"))
106 +both = px.dropna(subset=["mktcap_findata", "mktcap_yahoo"]); META["share_months_findata_price"] = round(float(px.mktcap_src.eq("findata_raw").mean()), 4); META["share_months_two_sources_disagree_gt5pct"] = round(float(((both.mktcap_findata / both.mktcap_yahoo - 1).abs() > 0.05).mean()), 4)
48 107 # author's variable for the before/after: adjusted close x max-class SEC shares (history + constant), exactly as r3k_02
49 −cur = mc_v1.set_index("ticker")["shares"].to_dict(); sh_v1 = sh_v1.sort_values(["ticker", "date"]); sh_v1["date"] = pd.to_datetime(sh_v1["date"]); sh_v1 = sh_v1.drop_duplicates(["ticker", "date"], keep="last")
108 +cur = mc_v1.set_index("ticker")["shares"].to_dict(); sh_v1 = sh_v1.sort_values(["ticker", "date"]); sh_v1["date"] = ns(sh_v1["date"]); sh_v1 = sh_v1.drop_duplicates(["ticker", "date"], keep="last")
50 109 tmp = pd.merge_asof(px[["ticker", "date", "adj_close"]].sort_values("date"), sh_v1.rename(columns={"shares": "sh1", "date": "d1"}).sort_values("d1"), left_on="date", right_on="d1", by="ticker", direction="backward")
51 110 tmp["sh1"] = tmp["sh1"].fillna(tmp["ticker"].map(cur)); px = px.merge(tmp[["ticker", "date", "sh1"]], on=["ticker", "date"], how="left"); px["mktcap_v1"] = px["sh1"] * px["adj_close"]
111 +# [A1f] series-level check vs the IWV weight (2026-10-23): implied-cap ratio r_i = cap_i(Oct 2026) / weight_i; firms with r_i outside
112 +# [1/20, 20] x median(r) have a share series that is wrong throughout (or a non-matching listing) -> cap set to missing for the firm
113 +wts = comp.drop_duplicates("ticker").set_index("ticker")["russell_weight_pct"]; last = px[px.date == px.date.max()].drop_duplicates("ticker").set_index("ticker")["mktcap"]
114 +rr = (last / wts.reindex(last.index)).dropna(); rr = rr[np.isfinite(rr) & (rr > 0)]; ratio = rr / rr.median(); bad_series = set(ratio[(ratio > 20) | (ratio < 1 / 20)].index)
115 +META["n_tickers_cap_inconsistent_with_iwv_weight"] = len(bad_series); META["tickers_cap_inconsistent_with_iwv_weight"] = sorted(bad_series); px.loc[px.ticker.isin(bad_series), "mktcap"] = np.nan
52 116 # cap screen [A2]
53 117 px["cap_med12"] = px.groupby("ticker")["mktcap"].transform(lambda s: s.rolling(12, min_periods=3).median().shift(1))
54 118 flag = ((px["mktcap"] > 10 * px["cap_med12"]) | (px["mktcap"] < 0.1 * px["cap_med12"])) & px["cap_med12"].notna()
55 −flag |= (px["mktcap"] > 5e12) & (px["date"] < "2023-01-01")
56 −px["mktcap_flag"] = flag; px.loc[flag, "mktcap"] = np.nan; META["n_months_cap_flagged"] = int(flag.sum()); META["n_tickers_cap_flagged"] = int(px[flag].ticker.nunique())
119 +flag |= ((px["mktcap"] > 5e12) & (px["date"] < "2023-01-01")) | (px["mktcap"] > 6e12) | (px["mktcap"] < 5e6) # absolute floor: shares reported in thousands (MBUU, RCAT, HY 2015) give caps below USD 5 M
120 +px["mktcap_flag"] = flag; px.loc[flag, "mktcap"] = np.nan; META["n_months_cap_flagged"] = int(flag.sum()); META["n_tickers_cap_flagged"] = int(px[flag].ticker.nunique()); META["n_months_shares_backfilled"] = int(px.shares_backfilled.sum()); META["n_months_split_carry_ne_1"] = int((px.cum_price != px.cum_shares).sum())
57 121 px = px.sort_values(["ticker", "date"]).reset_index(drop=True); px.to_parquet(f"{V}/monthly_returns_v2.parquet", index=False)
58 122 print("monthly rows", len(px), "tickers", px.ticker.nunique(), "| caps flagged", int(flag.sum()), "| splices removed", int(px.splice.sum()), flush=True)
59 123
@@ -80,22 +144,22 @@ for r in panel[["ticker", "fiscal_year"]].drop_duplicates().itertuples(index=Fal
80 144 if splice_hold or (t, Y) in hy_key: fwd = fwd6 = fwd24 = strict = np.nan
81 145 if splice_mom: mom = np.nan
82 146 vol = win["ret"].std() * np.sqrt(12) if win["ret"].notna().sum() >= 6 else np.nan; fvol = w1["ret"].std() * np.sqrt(12) if w1["ret"].notna().sum() >= 6 else np.nan
83 − capd = d.loc[(d.date <= form) & d.mktcap.notna()]; cap = capd.mktcap.iloc[-1] if len(capd) else np.nan; cap_age = (form - capd.date.iloc[-1]).days / 30.44 if len(capd) else np.nan
84 − sh_age = capd.shares_age_m.iloc[-1] if len(capd) else np.nan
147 + capd = d.loc[(d.date <= form) & (d.date > form - pd.offsets.MonthEnd(3)) & d.mktcap.notna()]; cap = capd.mktcap.iloc[-1] if len(capd) else np.nan; cap_age = (form - capd.date.iloc[-1]).days / 30.44 if len(capd) else np.nan; cap_src = capd.mktcap_src.iloc[-1] if len(capd) else "none" # [A2b] cap observed within 3 months of formation, else missing
148 + sh_age = capd.shares_age_m.iloc[-1] if len(capd) else np.nan; bf = bool(capd.shares_backfilled.iloc[-1]) if len(capd) else False
85 149 cap1 = d.loc[(d.date <= form) & d.mktcap_v1.notna(), "mktcap_v1"]; cap1 = cap1.iloc[-1] if len(cap1) else np.nan
86 150 rows.append({"ticker": t, "fiscal_year": Y, "fwd_ret": fwd, "fwd_ret_6m": fwd6, "fwd_ret_24m": fwd24, "fwd_ret_12m_strict": strict, "n_valid_fwd": nvalid,
87 151 "spy_fwd": cum_b("SPY", form, end1), "iwv_fwd": cum_b("IWV", form, end1), "iwm_fwd": cum_b("IWM", form, end1), "mom_12m": mom, "vol_12m": vol, "fwd_vol": fvol,
88 − "mktcap": cap, "log_mktcap": np.log(cap) if pd.notna(cap) and cap > 0 else np.nan, "cap_age_m": cap_age, "shares_age_m": sh_age, "mktcap_v1": cap1,
152 + "mktcap": cap, "log_mktcap": np.log(cap) if pd.notna(cap) and cap > 0 else np.nan, "cap_age_m": cap_age, "shares_age_m": sh_age, "shares_backfilled": bf, "mktcap_src": cap_src, "mktcap_v1": cap1,
89 153 "splice_in_window": splice_hold or (t, Y) in hy_key, "first_price": d.date.min(), "months_listed": (form - d.date.min()).days / 30.44})
90 154 rp = pd.DataFrame(rows); rp["fwd_ret_mktadj"] = rp["fwd_ret"] - rp["spy_fwd"]; rp["fwd_ret_iwvadj"] = rp["fwd_ret"] - rp["iwv_fwd"]
91 −META["n_firm_years_fwd_removed_hygiene"] = int(rp.splice_in_window.sum())
155 +META["n_firm_years_fwd_removed_hygiene"] = int(rp.splice_in_window.sum()); META["n_firm_years_cap_backfilled"] = int(rp.shares_backfilled.sum())
92 156
93 157 # ---------------- key on (cik, fiscal_year) [B3] ----------------
94 158 panel["cik"] = panel["cik"].astype(str).str.zfill(10); n0 = len(panel)
95 −panel = panel.drop_duplicates(); n1 = len(panel); META["n_exact_duplicate_rows_dropped"] = n0 - n1
159 +panel = panel.drop_duplicates(); panel = panel.sort_values(["cik", "ticker", "fiscal_year", "total_chunks"], ascending=[True, True, True, False]).drop_duplicates(["cik", "ticker", "fiscal_year"], keep="first"); n1 = len(panel); META["n_duplicate_rows_dropped_same_cik_ticker_year"] = n0 - n1 # [B3] exact duplicates + same-filer duplicates (XOM: 19 rows); keeps the row with the most chunks
96 160 hist_len = px.groupby("ticker").size(); panel["_hist"] = panel.ticker.map(hist_len).fillna(0)
97 161 dual = panel.groupby("cik").ticker.nunique(); dual = dual[dual > 1].index.tolist(); META["dual_class_ciks"] = {c: sorted(panel[panel.cik == c].ticker.unique().tolist()) for c in dual}
98 −keep_t = panel[panel.cik.isin(dual)].groupby("cik").apply(lambda g: g.sort_values(["_hist", "ticker"], ascending=[False, True]).ticker.iloc[0]).to_dict()
162 +keep_t = {c: g.sort_values(["_hist", "ticker"], ascending=[False, True]).ticker.iloc[0] for c, g in panel[panel.cik.isin(dual)].groupby("cik")}
99 163 panel = panel[~panel.cik.isin(dual) | (panel.ticker == panel.cik.map(keep_t))].drop(columns="_hist"); n2 = len(panel); META["n_dual_class_rows_dropped"] = n1 - n2
100 164 assert not panel.duplicated(["cik", "fiscal_year"]).any(), "panel not unique on (cik, fiscal_year)"
101 165 META["kept_ticker_per_dual_class_cik"] = keep_t; META["n_rows_before_dedup"] = n0; META["n_rows_after_dedup"] = n2
@@ -113,7 +177,7 @@ merged = merged.sort_values(["cik", "fiscal_year"]).reset_index(drop=True); merg
113 177 est = merged[merged.fwd_ret.notna() & (merged.fiscal_year <= 2024)]
114 178 META.update({"n_firm_years": int(len(merged)), "n_with_fwd": int(merged.fwd_ret.notna().sum()), "n_estimation_sample": int(len(est)), "n_estimation_sample_v1": int(len(v1[v1.fwd_ret.notna() & (v1.fiscal_year <= 2024)])),
115 179 "n_est_without_cap": int(est.mktcap.isna().sum()), "n_est_without_cap_v1": int(v1[v1.fwd_ret.notna() & (v1.fiscal_year <= 2024)].mktcap.isna().sum()), "n_est_strict12": int(est.fwd_ret_12m_strict.notna().sum()), "n_est_lt12_valid": int((est.n_valid_fwd < 12).sum()),
116 − "size_groups": est.size_group.value_counts(dropna=False).to_dict(), "size_groups_pct42": est.size_group_pct42.value_counts(dropna=False).to_dict(), "price_vintage_unadj": "r3k_10 download (see data/processed_r3k_v2/download_v2.log)", "shares_source": "SEC companyfacts, sum of classes at latest end (r3k_12)"})
180 + "size_groups": est.size_group.value_counts(dropna=False).to_dict(), "mktcap_src_est": est.mktcap_src.value_counts(dropna=False).to_dict(), "size_groups_pct42": est.size_group_pct42.value_counts(dropna=False).to_dict(), "price_vintage_unadj": "r3k_10 download (see data/processed_r3k_v2/download_v2.log)", "shares_source": "SEC companyfacts, sum of classes at latest end (r3k_12)"})
117 181 json.dump(META, open(f"{V}/panel_meta_v2.json", "w"), indent=2, default=str)
118 182 # ---------------- diagnostics for the before/after [A: acceptance] ----------------
119 183 diag = {}
modified scripts/r3k_15_analysis_v2.py +5 −3
@@ -63,7 +63,7 @@ def spread_stats(ls, by_cohort=None):
63 63 mu, t = I.nw_t(ls); out = {"Q5_minus_Q1": mu, "t_stat": t, "t_plain": I.plain_t(ls), "ci95": I.ci95(ls), "ci95_boot": I.block_boot_ci(ls), "n_years": len(ls)}
64 64 if by_cohort is not None: out["by_cohort"] = {int(k): round(v, 4) for k, v in by_cohort.items()}
65 65 return out
66 −def sort_table(d, z, weight=None, oc="fwd_ret", stat="mean", minn=40, yrs=None, diag_label=None):
66 +def sort_table(d, z, weight=None, oc="fwd_ret", stat="mean", minn=40, yrs=None, diag_label=None, cap_w=None):
67 67 ann = {q: [] for q in range(1, 6)}; ls = []; ls_trim = []; byc = {}
68 68 for yr, g in d.groupby("fiscal_year"):
69 69 if yrs is not None and yr not in yrs: continue
@@ -76,7 +76,9 @@ def sort_table(d, z, weight=None, oc="fwd_ret", stat="mean", minn=40, yrs=None,
76 76 if weight is None:
77 77 vals[q] = s[oc].median() if stat == "median" else s[oc].mean(); st = s[s[oc] <= p99]; vals_trim[q] = st[oc].mean() if len(st) else np.nan
78 78 else:
79 − w = s[weight].clip(lower=0).fillna(0); vals[q] = np.average(s[oc], weights=w) if w.sum() > 0 else s[oc].mean(); vals_trim[q] = vals[q]
79 + w = s[weight].clip(lower=0).fillna(0)
80 + if cap_w is not None and w.sum() > 0: w = np.minimum(w, cap_w * w.sum()) # single-pass weight cap (share of the quintile's total cap)
81 + vals[q] = np.average(s[oc], weights=w) if w.sum() > 0 else s[oc].mean(); vals_trim[q] = vals[q]
80 82 if diag_label and w.sum() > 0:
81 83 ws = (w / w.sum()).sort_values(ascending=False); VWD.append({"sort": diag_label, "cohort": int(yr), "q": q, "n": len(s), "max_weight": round(float(ws.iloc[0]), 4), "max_ticker": s.loc[ws.index[0], "ticker"], "top10_weight": round(float(ws.iloc[:10].sum()), 4)})
82 84 ann[q].append(vals[q])
@@ -88,7 +90,7 @@ sorts = {}
88 90 for k in SIGL:
89 91 for sfx, oc, st in OUTS:
90 92 sorts[f"{k}_EW{sfx}"] = sort_table(df, f"z_{k}", None, oc, st)
91 − if sfx in ("", "_w"): sorts[f"{k}_VW{sfx}"] = sort_table(df, f"z_{k}", "mktcap", oc, st, diag_label=f"{k}_VW{sfx}"); sorts[f"{k}_VW_v1{sfx}"] = sort_table(df, f"z_{k}", "mktcap_v1", oc, st, diag_label=f"{k}_VW_v1{sfx}")
93 + if sfx in ("", "_w"): sorts[f"{k}_VW{sfx}"] = sort_table(df, f"z_{k}", "mktcap", oc, st, diag_label=f"{k}_VW{sfx}"); sorts[f"{k}_VW_v1{sfx}"] = sort_table(df, f"z_{k}", "mktcap_v1", oc, st, diag_label=f"{k}_VW_v1{sfx}"); sorts[f"{k}_VWcap10{sfx}"] = sort_table(df, f"z_{k}", "mktcap", oc, st, diag_label=f"{k}_VWcap10{sfx}", cap_w=0.10)
92 94 sorts[f"{k}_EW_madj"] = sort_table(df, f"z_{k}", None, "fwd_ret_mktadj"); sorts[f"{k}_EW_madj_w"] = sort_table(df, f"z_{k}", None, "fwd_madj_w")
93 95 RES["sorts"] = {k: {kk: (round(vv, 5) if isinstance(vv, float) else vv) for kk, vv in v.items()} for k, v in sorts.items()}
94 96 pd.DataFrame(sorts).T.drop(columns=["by_cohort"]).to_csv(f"{T}/t2_portfolio_sorts.csv")
modified scripts/r3k_18_survivorship.py +2 −2
@@ -28,12 +28,12 @@ for fn in sorted(glob.glob(f"{V}/iwv_holdings/IWV_holdings_asof*.json")):
28 28 s = pd.DataFrame(rows); s = s[(s.asset == "Equity") & pd.to_numeric(s.mv, errors="coerce").gt(0)].copy(); s["mv"] = pd.to_numeric(s.mv); s["asof"] = pd.Timestamp(asof); snaps[asof] = s
29 29 RES["snapshots"] = {k: int(len(v)) for k, v in snaps.items()}; print("snapshots", RES["snapshots"], flush=True)
30 30 # reference "current" sets: the latest capture (2025-08-28) by ISIN / CUSIP and the 2026 universe by ticker
31 −latest = snaps[max(snaps)]; cur_isin = set(latest.isin) - {"-", ""}; cur_cusip = set(latest.cusip) - {"-", ""}
31 +latest = snaps[max(snaps)]; cur_isin = set(latest["isin"]) - {"-", ""}; cur_cusip = set(latest["cusip"]) - {"-", ""}
32 32 uni = json.load(open(f"{R}/universe.json")); cons = uni["constituents"]; cur_tickers = set((c.get("ticker") or "") for c in cons) if isinstance(cons, list) else set(cons.keys()); panel_tickers = set(df.ticker)
33 33 # (ii) attrition by size tercile x snapshot
34 34 att = []
35 35 for asof, s in snaps.items():
36 − s = s.copy(); s["present"] = s.isin.isin(cur_isin) | s.cusip.isin(cur_cusip) | s.ticker.isin(cur_tickers); s["in_panel"] = s.ticker.isin(panel_tickers)
36 + s = s.copy(); s["present"] = s["isin"].isin(cur_isin) | s["cusip"].isin(cur_cusip) | s["ticker"].isin(cur_tickers); s["in_panel"] = s.ticker.isin(panel_tickers)
37 37 s["terc"] = pd.qcut(s.mv.rank(method="first"), 3, labels=["small", "mid", "large"])
38 38 for tc, g in s.groupby("terc", observed=True): att.append({"asof": asof, "tercile": tc, "n": int(len(g)), "absent_2026_share": round(float(1 - g.present.mean()), 4), "in_panel_share": round(float(g.in_panel.mean()), 4), "weight_share": round(float(g.weight.sum() / s.weight.sum()), 4)})
39 39 att.append({"asof": asof, "tercile": "all", "n": int(len(s)), "absent_2026_share": round(float(1 - s.present.mean()), 4), "in_panel_share": round(float(s.in_panel.mean()), 4), "weight_share": 1.0})
added scripts/r3k_19_figures_v2.py +94 −0
@@ -0,0 +1,94 @@
1 +"""r3k_19_figures_v2.py — Stage D (FIRC-0002): figures from the v2 results -> figures/r3k_v2/*.pdf + .png (same names as r3k_06 where
2 +the figure is the same object, plus new ones: fig9_eventtime redrawn with 2021 = 0 and 95% CIs [G], fig15_spec_curve [N],
3 +fig16_placebo_launches [F], fig17_attrition [D]). Run: .venv-review/bin/python -I scripts/r3k_19_figures_v2.py
4 +"""
5 +import json, numpy as np, pandas as pd, matplotlib
6 +matplotlib.use("Agg"); import matplotlib.pyplot as plt
7 +V = "data/processed_r3k_v2"; T = "results/r3k_v2"; F = "figures/r3k_v2"
8 +R = json.load(open(f"{T}/results.json")); R2 = json.load(open(f"{T}/results2.json")); RE = json.load(open(f"{T}/results_event.json")); RS = json.load(open(f"{T}/results_surv.json"))
9 +df = pd.read_parquet(f"{V}/analysis_panel_v2.parquet"); BLUE, GOLD, GREY, RED, GREEN = "#16365c", "#c9a227", "#8a8a8a", "#a63d40", "#3b7a57"
10 +def save(fig, name): fig.tight_layout(); fig.savefig(f"{F}/{name}.pdf"); fig.savefig(f"{F}/{name}.png", dpi=220); plt.close(fig); print("fig", name)
11 +# 1 quintile returns pre/post, raw and winsorized
12 +fig, axes = plt.subplots(1, 2, figsize=(9.5, 3.4), sharey=True)
13 +for ax, oc, ttl in [(axes[0], "fwd_ret", "Raw forward return"), (axes[1], "fwd_w", "Winsorized within year")]:
14 + qs = {}
15 + for per, yrs in [("Pre-ChatGPT cohorts (2010-2021)", range(2010, 2022)), ("Post-ChatGPT cohorts (2022-2024)", range(2022, 2025))]:
16 + d = df[df.fiscal_year.isin(list(yrs))].dropna(subset=["z_AI_COMPOS", oc]).copy(); d["q"] = d.groupby("fiscal_year")["z_AI_COMPOS"].transform(lambda s: pd.qcut(s.rank(method="first"), 5, labels=False) + 1); qs[per] = d.groupby(["fiscal_year", "q"])[oc].mean().groupby("q").mean() * 100
17 + x = np.arange(1, 6); w = 0.38
18 + for i, (per, s) in enumerate(qs.items()): ax.bar(x + (i - .5) * w, s.values, w, color=[BLUE, GOLD][i], label=per)
19 + ax.set_xticks(x); ax.set_xticklabels([f"Q{i}" for i in x]); ax.set_title(ttl, fontsize=9); ax.spines[["top", "right"]].set_visible(False)
20 +axes[0].set_ylabel("Mean forward 12-month return (%)"); axes[0].legend(frameon=False, fontsize=7); save(fig, "fig1_quintiles")
21 +# 2 cumulative long-short (all, large, small)
22 +fig, ax = plt.subplots(figsize=(7.2, 3.4)); ts = pd.read_parquet(f"{V}/ls_ts_v2_AI_COMPOS.parquet"); ts["date"] = pd.to_datetime(ts["date"]); ax.plot(ts.date, (1 + ts.ls).cumprod() - 1, color=BLUE, lw=2, label="All firms (Q5-Q1, EW)")
23 +for g, c, l in [("large_top1000", GOLD, "Largest 1,000 (corrected cap)"), ("small_rest", GREY, "Remaining firms")]:
24 + t2 = pd.read_parquet(f"{V}/ls_ts_v2_AI_COMPOS_{g}.parquet"); t2["date"] = pd.to_datetime(t2["date"]); ax.plot(t2.date, (1 + t2.ls).cumprod() - 1, color=c, lw=1.6, label=l)
25 +ax.axvline(pd.Timestamp("2022-11-30"), color=RED, ls="--", lw=1); ax.set_ylabel("Cumulative long-short return"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig2_cumulative_ls")
26 +# 3 cohort spreads raw vs winsorized vs median, with CI band of the pre mean
27 +cs = R2["cohorts"]["cohort_spreads_pct"]; cw = R2["cohorts_w"]["cohort_spreads_pct"]; cm = R2["cohorts_med"]["cohort_spreads_pct"]; yrs = [int(k) for k in cs]
28 +fig, ax = plt.subplots(figsize=(7.6, 3.4)); x = np.array(yrs); ax.bar(x - .27, list(cs.values()), .27, color=BLUE, label="raw"); ax.bar(x, list(cw.values()), .27, color=GOLD, label="winsorized within year"); ax.bar(x + .27, list(cm.values()), .27, color=GREY, label="median difference")
29 +ax.axhline(0, color="k", lw=.6); ax.axvline(2021.5, color=RED, ls="--", lw=1); ax.set_xlabel("Formation cohort (filing year)"); ax.set_ylabel("Q5-Q1 forward return (pp)"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig3_cohort_spreads")
30 +# 4 trend
31 +tr = df.groupby("fiscal_year")[["ai_adoption_wmean", "ai_infrastructure_wmean"]].mean(); trl = df[df.size_group == "large"].groupby("fiscal_year")["ai_adoption_wmean"].mean(); trs = df[df.size_group == "small"].groupby("fiscal_year")["ai_adoption_wmean"].mean()
32 +fig, ax = plt.subplots(figsize=(7.2, 3.2)); ax.plot(tr.index, tr.ai_adoption_wmean, color=BLUE, lw=2, marker="o", label="AI adoption (all firms)"); ax.plot(trl.index, trl.values, color=GOLD, lw=1.4, label="largest 1,000"); ax.plot(trs.index, trs.values, color=GREY, lw=1.4, label="remaining firms")
33 +ax.set_xlabel("Filing year"); ax.set_ylabel("Mean section-weighted similarity"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig4_trend")
34 +# 5 FM by year raw vs winsorized
35 +fm = pd.DataFrame(R2["fm_byyear"]); fig, ax = plt.subplots(figsize=(7.2, 3.2)); ax.errorbar(fm.year - .12, fm.coef * 100, yerr=1.96 * fm.se * 100, fmt="o", color=BLUE, capsize=2, label="raw returns"); ax.errorbar(fm.year + .12, fm.coef_w * 100, yerr=1.96 * fm.se_w * 100, fmt="s", color=GOLD, capsize=2, label="winsorized within year")
36 +ax.axhline(0, color="k", lw=.6); ax.axvline(2021.5, color=RED, ls="--", lw=1); ax.set_xlabel("Formation cohort"); ax.set_ylabel("Slope on AI composite (pp per SD)"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig5_fm_byyear")
37 +# 6 placebo factors (winsorized spreads)
38 +pl = R2["placebo"]; names = list(pl); pre = [pl[n]["pre_w"]["mean"] * 100 for n in names]; post = [pl[n]["post_w"]["mean"] * 100 for n in names]
39 +fig, ax = plt.subplots(figsize=(7.2, 3.6)); x = np.arange(len(names)); ax.bar(x - .2, pre, .4, color=BLUE, label="Pre (2010-2021), wins."); ax.bar(x + .2, post, .4, color=[GOLD if pl[n]["is_ai"] else GREY for n in names], label="Post (2022-2024), wins.")
40 +ax.set_xticks(x); ax.set_xticklabels(names, rotation=30, ha="right", fontsize=8); ax.axhline(0, color="k", lw=.6); ax.set_ylabel("Q5-Q1 spread (pp/yr)"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig6_placebo")
41 +# 7 event CAR paths with day 0 = 1 Dec 2022 and the pre-block shaded
42 +p = pd.read_parquet(f"{V}/event_car_paths_v2.parquet"); fig, ax = plt.subplots(figsize=(7.2, 3.4)); p0 = p[p.pos >= -20]
43 +for c, col in [("High", BLUE), ("Mid", GREY), ("Low", GOLD)]: ax.plot(p0.pos, (p0[c] - p0[c].iloc[0]) * 100, color=col, lw=1.6, label=f"{c} pre-event AI tercile")
44 +ax.plot(p0.pos, (p0["High_minus_Low"] - p0["High_minus_Low"].iloc[0]) * 100, color=RED, lw=2, label="High minus Low"); ax.axvspan(-20, -1, color=GREY, alpha=.12); ax.axvline(0, color="k", ls="--", lw=.8); ax.axhline(0, color="k", lw=.5)
45 +ax.set_xlabel("Trading days relative to 1 December 2022 (shaded: 1-29 November 2022)"); ax.set_ylabel("Cumulative abnormal return (%)"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig7_event_car")
46 +# 8 double sort heatmap (post, winsorized)
47 +ds = pd.DataFrame(R2["double_sort_post_w"]); cols = [c for c in ds.columns if c not in ("Hi-Lo", "N")]; M = ds[cols].values.astype(float); fig, ax = plt.subplots(figsize=(6, 3)); im = ax.imshow(M, cmap="RdYlBu_r", aspect="auto"); ax.set_xticks(range(len(cols))); ax.set_xticklabels(cols); ax.set_yticks(range(len(ds.index))); ax.set_yticklabels(ds.index); ax.set_xlabel("AI-composite tercile (within size tercile)"); ax.set_ylabel("Size tercile (corrected cap)")
48 +for i in range(M.shape[0]):
49 + for j in range(M.shape[1]): ax.text(j, i, f"{M[i, j]:.1f}", ha="center", va="center", fontsize=8)
50 +plt.colorbar(im, ax=ax, label="Mean forward return, wins. (%)"); save(fig, "fig8_double_sort")
51 +# 9 event-time: differences from 2021 with 95% CI (author exposure) + trailing exposure
52 +ev = pd.DataFrame(R2["did_eventtime"]["rows"]); et = pd.DataFrame(R2["did_eventtime_trailing"]["rows"]); fig, ax = plt.subplots(figsize=(7.2, 3.4))
53 +ax.errorbar(ev.year - .1, ev.diff_from_ref * 100, yerr=1.96 * ev.diff_se * 100, fmt="o-", color=BLUE, capsize=2, label="2019-21 exposure (author): difference from 2021"); ax.errorbar(et.year + .1, et.diff_from_ref * 100, yerr=1.96 * et.diff_se * 100, fmt="s--", color=GOLD, capsize=2, label="trailing 3-year exposure ending Y-1: difference from 2021")
54 +ax.axhline(0, color="k", lw=.6); ax.axvline(2021.5, color=RED, ls="--", lw=1); ax.axvspan(2013.5, 2018.5, color=GREY, alpha=.1); ax.text(2014, ax.get_ylim()[1] * .85, "author exposure measured after these outcomes", fontsize=7, color=GREY)
55 +ax.set_xlabel("Outcome cohort (reference 2021 = 0)"); ax.set_ylabel("Slope difference from 2021 (pp per SD)"); ax.legend(frameon=False, fontsize=7); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig9_eventtime")
56 +# 10 filing type (winsorized)
57 +N = R2["filing_type"]; tags = [("10k", "10-K"), ("10q", "10-Q"), ("8k", "8-K"), ("formw", "Equal-form")]; fig, ax = plt.subplots(figsize=(6.4, 3.2)); x = np.arange(len(tags))
58 +ax.bar(x - .2, [N[t]["pre_w"]["mean"] * 100 for t, _ in tags], .4, color=BLUE, label="Pre, wins."); ax.bar(x + .2, [N[t]["post_w"]["mean"] * 100 for t, _ in tags], .4, color=GOLD, label="Post, wins.")
59 +ax.set_xticks(x); ax.set_xticklabels([l for _, l in tags]); ax.axhline(0, color="k", lw=.6); ax.set_ylabel("Q5-Q1 spread (pp/yr)"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig10_filing_type")
60 +# 11 placebo break dates + permutation distribution
61 +pb = R2["placebo_breaks"]["winsorized"]["diff_post_minus_pre_pct_by_break_year"]; cb = R2["placebo_breaks"]["winsorized"]["permutation"]["contiguous_blocks"]; fig, axes = plt.subplots(1, 2, figsize=(9.5, 3.1)); yrs = [int(k) for k in pb]
62 +axes[0].bar(yrs, list(pb.values()), color=[RED if y == 2022 else GREY for y in yrs]); axes[0].axhline(0, color="k", lw=.6); axes[0].set_xlabel("Candidate break year (first post cohort), wins."); axes[0].set_ylabel("Post minus pre mean spread (pp)"); axes[0].spines[["top", "right"]].set_visible(False)
63 +yb = [int(k) for k in cb]; axes[1].bar(yb, [v * 100 for v in cb.values()], color=[RED if y == 2022 else GREY for y in yb]); axes[1].axhline(0, color="k", lw=.6); axes[1].set_xlabel("Contiguous three-cohort block (first cohort), wins."); axes[1].set_ylabel("Block minus rest (pp)"); axes[1].spines[["top", "right"]].set_visible(False); save(fig, "fig11_placebo_breaks")
64 +# 12 size groups pre/post, raw vs winsorized, with CIs
65 +G = R["groups"]; names = ["large_top1000", "small_rest", "sp500", "non_sp500"]; labs = ["Largest 1,000", "Remaining", "S&P 500", "Non-S&P 500"]; fig, ax = plt.subplots(figsize=(7.4, 3.3)); x = np.arange(4)
66 +for i, (sfx, c, l) in enumerate([("", BLUE, "Pre, raw"), ("_w", "#4f7bb5", "Pre, wins."), ("", GOLD, "Post, raw"), ("_w", "#e0c66a", "Post, wins.")]):
67 + per = "pre" if i < 2 else "post"; vals = [G[n_][per + sfx]["mean"] * 100 for n_ in names]; err = [[G[n_][per + sfx]["mean"] * 100 - G[n_][per + sfx]["ci95"][0] * 100 for n_ in names], [G[n_][per + sfx]["ci95"][1] * 100 - G[n_][per + sfx]["mean"] * 100 for n_ in names]]
68 + ax.bar(x + (i - 1.5) * .2, vals, .2, color=c, label=l, yerr=err, capsize=2, error_kw={"lw": .7})
69 +ax.set_xticks(x); ax.set_xticklabels(labs); ax.axhline(0, color="k", lw=.6); ax.set_ylabel("Q5-Q1 spread (pp/yr), 95% CI over cohorts"); ax.legend(frameon=False, fontsize=7, ncol=2); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig12_size_groups")
70 +# 13 cohorts both (winsorized)
71 +SM = R["samples"]; ya = [int(k) for k in SM["r3k"]["cohorts_w_pct"]]; va = list(SM["r3k"]["cohorts_w_pct"].values()); yb = [int(k) for k in SM["sp500"]["cohorts_w_pct"]]; vb = list(SM["sp500"]["cohorts_w_pct"].values())
72 +fig, ax = plt.subplots(figsize=(7.2, 3.4)); ax.bar(np.array(ya) - .2, va, .4, color=BLUE, label="Russell 3000 universe (wins.)"); ax.bar(np.array(yb) + .2, vb, .4, color=GOLD, label="S&P 500 members only (wins.)"); ax.axhline(0, color="k", lw=.6); ax.axvline(2021.5, color=RED, ls="--", lw=1); ax.set_xlabel("Formation cohort"); ax.set_ylabel("Q5-Q1 forward return (pp)"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig13_cohorts_both")
73 +# 14 cumulative LS both
74 +fig, ax = plt.subplots(figsize=(7.2, 3.4))
75 +for sname, c, l in [("r3k", BLUE, "Russell 3000 universe"), ("sp500", GOLD, "S&P 500 members only")]:
76 + t2 = pd.read_parquet(f"{V}/ls_ts_v2_AI_COMPOS_{sname}.parquet"); t2["date"] = pd.to_datetime(t2["date"]); ax.plot(t2.date, (1 + t2.ls).cumprod() - 1, color=c, lw=2, label=l)
77 +ax.axvline(pd.Timestamp("2022-11-30"), color=RED, ls="--", lw=1); ax.axhline(0, color="k", lw=.5); ax.set_ylabel("Cumulative long-short return"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig14_cumulative_both")
78 +# 15 specification curve
79 +sc = pd.read_csv(f"{T}/spec_curve.csv").sort_values("spread_pct").reset_index(drop=True); fig, axes = plt.subplots(2, 1, figsize=(8, 4.6), gridspec_kw={"height_ratios": [3, 2]}, sharex=True)
80 +col = np.where(sc.t_nw.abs() > 1.96, np.where(sc.spread_pct > 0, BLUE, RED), GREY); axes[0].scatter(sc.index, sc.spread_pct, s=4, c=col); axes[0].axhline(0, color="k", lw=.6); axes[0].set_ylabel("Q5-Q1 spread (pp/yr)"); axes[0].set_title(f"{len(sc)} specifications: {R2['spec_curve']['share_positive'] * 100:.0f}% positive, {R2['spec_curve']['share_positive_sig_nw'] * 100:.0f}% positive and |NW t| > 1.96 (blue), {R2['spec_curve']['share_negative_sig_nw'] * 100:.1f}% negative and significant (red)", fontsize=8); axes[0].spines[["top", "right"]].set_visible(False)
81 +dims = [("size_filter", ["all", "ex_top500", "large", "small"]), ("winsor", ["raw", "w199", "w595", "median"]), ("weight", ["EW", "VW"]), ("horizon", [6, 12, 24]), ("form", ["all", "10-K", "10-Q", "8-K"]), ("aggregation", ["wmean", "mean", "top5"])]; yl = []; yt = []; k = 0
82 +for dname, levels in dims:
83 + for lv in levels:
84 + m = sc[dname].astype(str) == str(lv); axes[1].scatter(sc.index[m], np.full(m.sum(), k), s=2, c=GREY); yl.append(f"{dname}={lv}"); yt.append(k); k += 1
85 +axes[1].set_yticks(yt); axes[1].set_yticklabels(yl, fontsize=5); axes[1].set_xlabel("Specifications ranked by spread"); axes[1].invert_yaxis(); save(fig, "fig15_spec_curve")
86 +# 16 placebo launches
87 +PS = RE["placebo_summary"]; yrs = sorted(PS["HL_1y_by_launch_pct"]); fig, axes = plt.subplots(1, 2, figsize=(8.6, 3.0))
88 +axes[0].bar(yrs, [PS["HL_1y_by_launch_pct"][y] for y in yrs], color=[RED if y == "2022" else GREY for y in yrs]); axes[0].axhline(0, color="k", lw=.6); axes[0].set_ylabel("1-year H-L CAR (pp), full sample"); axes[0].set_xlabel("Launch date 30 Nov of year"); axes[0].spines[["top", "right"]].set_visible(False)
89 +axes[1].bar(yrs, [PS["t_cda_by_launch"][y] for y in yrs], color=[RED if y == "2022" else GREY for y in yrs]); axes[1].axhline(0, color="k", lw=.6); axes[1].axhline(1.96, color=BLUE, ls=":", lw=.8); axes[1].axhline(-1.96, color=BLUE, ls=":", lw=.8); axes[1].set_ylabel("CDA t"); axes[1].set_xlabel("Launch date 30 Nov of year"); axes[1].spines[["top", "right"]].set_visible(False); save(fig, "fig16_placebo_launches")
90 +# 17 attrition by tercile and snapshot
91 +AP = RS["attrition_pivot_absent_share"]; dates = sorted(AP); fig, ax = plt.subplots(figsize=(7.2, 3.1)); xs = [pd.Timestamp(d) for d in dates]
92 +for tc, c in [("small", RED), ("mid", GOLD), ("large", BLUE)]: ax.plot(xs, [AP[d][tc] * 100 for d in dates], marker="o", color=c, label=f"{tc} tercile (market value)")
93 +ax.set_ylabel("Then-constituents absent in 2025-26 (%)"); ax.set_xlabel("IWV holdings snapshot date"); ax.legend(frameon=False, fontsize=8); ax.spines[["top", "right"]].set_visible(False); save(fig, "fig17_attrition")
94 +print("figures done")
added scripts/r3k_20_tables_v2.py +322 −0
@@ -0,0 +1,322 @@
1 +"""r3k_20_tables_v2.py — Stage D (FIRC-0002, package L10 + every table-producing package): LaTeX tables and number macros from the
2 +v2 results (results/r3k_v2/*.json, CSVs) -> paper/tables/ (same file names as r3k_07_tables.py so the manuscript compiles; BEFORE
3 += git HEAD 3cf7560). Every printed number is read from one JSON field (cluster O); every table carries N and a \\tabnote{} block;
4 +every legacy macro name of r3k_07 is kept (same statistic, rebuilt panel) and new macros are added for the new statistics (Stage E
5 +swaps the prose). Also writes results/r3k_v2/inline_numbers.json (macro -> value). Run: .venv-review/bin/python -I scripts/r3k_20_tables_v2.py
6 +"""
7 +import json, os, numpy as np, pandas as pd
8 +V = "data/processed_r3k_v2"; T = "results/r3k_v2"; P = "paper/tables"; TB = "results/r3k"
9 +R = json.load(open(f"{T}/results.json")); R2 = json.load(open(f"{T}/results2.json")); RE = json.load(open(f"{T}/results_event.json")); RS = json.load(open(f"{T}/results_surv.json")); META = json.load(open(f"{V}/panel_meta_v2.json")); HY = json.load(open(f"{T}/hygiene_summary.json")); MD = json.load(open(f"{T}/mktcap_diag.json"))
10 +T17 = json.load(open(f"{T}/t17_refit.json")) if os.path.exists(f"{T}/t17_refit.json") else None; N1 = json.load(open("results/n1/corpus_counts.json")) if os.path.exists("results/n1/corpus_counts.json") else None
11 +df = pd.read_parquet(f"{V}/analysis_panel_v2.parquet")
12 +def n(x, d=0): return "--" if x is None else f"{float(x):,.{d}f}".replace(",", "{,}")
13 +def w(name, body, note=None):
14 + if note: body = body + "\n\\par\\vspace{2pt}{\\footnotesize\\emph{Notes:} " + note + "}"
15 + open(f"{P}/{name}", "w").write(body); print("table", name)
16 +def pp(x, d=1): return "--" if x is None or (isinstance(x, float) and np.isnan(x)) else f"{x * 100:+.{d}f}"
17 +def tv(t): return "--" if t is None or (isinstance(t, float) and np.isnan(t)) else f"{t:.2f}"
18 +def ci(c, d=1, scale=100): return "--" if not c or c[0] is None else f"[{c[0] * scale:+.{d}f}, {c[1] * scale:+.{d}f}]"
19 +def pv(p): return "--" if p is None else f"{p:.3f}"
20 +SE_NOTE = "NW = Newey--West (Bartlett, 3 lags) $t$ of the mean of the cohort series; plain = i.i.d. $t$; CI = 95\\% $t$-interval over cohorts; $n$ = number of cohorts. Returns are the look-ahead-free 12-month forward returns (July $Y{+}1$--June $Y{+}2$); `wins.' = 1/99 winsorized within the formation year; `median' = median spread within cohort. Rebuilt panel ($N$ = " + n(R["n_obs"]) + " firm-years, " + n(R["n_firms"]) + " firms, keyed on CIK $\\times$ year, cleaned returns, corrected market capitalization)."
21 +MAC = {}
22 +# ---------------- summary (T2) with min/max and N ----------------
23 +s = pd.read_csv(f"{T}/t1_summary_stats.csv", index_col=0); lab = {"ai_adoption_wmean": "AI adoption (wmean)", "ai_infrastructure_wmean": "AI infrastructure (wmean)", "ai_disruption_risk_wmean": "AI disruption risk (wmean)", "spid_mentions_tech": "SPID technology-project mentions", "spid_ai_any": "Any AI-initiative mention (0/1)", "fwd_ret": "Forward 12-m return (raw)", "fwd_w": "Forward 12-m return (wins. within year)", "fwd_ret_mktadj": "Forward return, SPY-adjusted", "mom_12m": "Momentum (12 m)", "vol_12m": "Volatility (12 m, ann.)", "log_mktcap": "log market cap (corrected)", "total_chunks": "Chunks per firm-year"}
24 +body = "\\begin{tabular}{lrrrrrrrrr}\n\\toprule\nVariable & N & Mean & SD & Min & P1 & Median & P99 & Max \\\\\n\\midrule\n"
25 +for k, l in lab.items():
26 + r = s.loc[k]; body += f"{l} & {n(r['count'])} & {r['mean']:.4f} & {r['std']:.4f} & {r['min']:.3f} & {r['1%']:.3f} & {r['50%']:.4f} & {r['99%']:.3f} & {r['max']:.3f} \\\\\n"
27 +w("tab_summary.tex", body + "\\bottomrule\n\\end{tabular}", f"Estimation sample {n(R['n_obs'])} firm-years ({n(R['n_firms'])} firms), formation cohorts 2010--2024; {n(META['n_est_without_cap'])} firm-years without a market capitalization ({MD['share_est_without_cap_v2'] * 100:.1f}\\%; author's panel: {n(META['n_est_without_cap_v1'])}). Returns: {HY['n_flagged_monthly']} monthly returns above 300\\% and {HY['n_flagged_firm_years']} firm-year forward returns above 500\\% were checked against FinData OS daily bars; {HY['verdicts_monthly'].get('series_break', 0)} monthly and {HY['verdicts_firm_year'].get('series_break', 0)} firm-year observations were price splices and are set to missing, the rest are confirmed. Market capitalization = unadjusted FinData close $\\times$ point-in-time SEC cover-page shares (log cap range [{MD['log_cap_min_max_v2'][0]:.1f}, {MD['log_cap_min_max_v2'][1]:.1f}]).")
28 +# ---------------- sorts (T5, both panels) ----------------
29 +def sort_rows(S, k, l, pre=""):
30 + out = ""
31 + for sfx, wl in [("EW", "EW, raw"), ("EW_w", "EW, wins."), ("EW_med", "EW, median"), ("VW", "VW (corrected cap)"), ("VWcap10", "VW, weights capped 10\\%"), ("VW_v1", "VW (author's cap)")]:
32 + key = f"{k}_{sfx}"
33 + if key not in S: continue
34 + r = S[key]; out += f"{l if sfx == 'EW' else ''} & {wl} & " + " & ".join(pp(r[f'Q{q}']) for q in range(1, 6)) + f" & {pp(r['Q5_minus_Q1'])} & {tv(r['t_stat'])} & {tv(r['t_plain'])} & {ci(r['ci95'])} & {int(r['n_years'])} \\\\\n"
35 + return out
36 +SIGL = [("AI_ADOPT", "AI adoption"), ("AI_INFRA", "AI infrastructure"), ("AI_RISK", "AI disruption risk"), ("AI_COMPOS", "AI composite"), ("AI_PC1", "PC1 of three facets"), ("AI_formw", "Equal-form composite"), ("AI_10k", "10-K composite"), ("SPID_TECH", "SPID tech projects (legacy quintiles)")]
37 +hdr = "\\begin{tabular}{llrrrrrrrrrr}\n\\toprule\nSignal & Weighting / outcome & Q1 & Q2 & Q3 & Q4 & Q5 & Q5$-$Q1 & NW $t$ & plain $t$ & 95\\% CI & Years \\\\\n"
38 +body = hdr + "\\midrule\n"
39 +for k, l in SIGL: body += sort_rows(R["sorts"], k, l) + "\\addlinespace\n"
40 +w("tab_sorts.tex", body + "\\bottomrule\n\\end{tabular}", SE_NOTE + f" VW diagnostic: maximum single-firm weight in any cohort $\\times$ quintile {R['vw_diag']['max_single_firm_weight_v2'] * 100:.0f}\\% (author's capitalization: {R['vw_diag']['max_single_firm_weight_v1'] * 100:.1f}\\%); cells above 25\\%: {R['vw_diag']['n_cells_above_25pct_v2']} vs {R['vw_diag']['n_cells_above_25pct_v1']}.")
41 +body = hdr
42 +for sname, lab_ in [("r3k", "Panel A: Russell 3000 universe"), ("sp500", "Panel B: S\\&P 500 members only")]:
43 + SM = R["samples"][sname]; body += f"\\midrule\n\\multicolumn{{12}}{{l}}{{\\textit{{{lab_}}} ($N$ = {n(SM['n'])} firm-years, {n(SM['firms'])} firms)}} \\\\\n\\midrule\n"
44 + for k, l in SIGL[:5] + [SIGL[7]]: body += sort_rows(SM["sorts"], k, l)
45 +w("tab_sorts_both.tex", body + "\\bottomrule\n\\end{tabular}", SE_NOTE + " Signals re-standardized within each panel.")
46 +# ---------------- LS alphas (T6) ----------------
47 +def alpha_rows(L, keys):
48 + out = ""
49 + for k, l in keys:
50 + for per in ["full", "pre", "post"]:
51 + r = L[k][per]
52 + if "mean_ls_ann_pct" not in r: out += f"{l if per == 'full' else ''} & {per} & {r.get('n_months', 0)} & \\multicolumn{{7}}{{l}}{{too few months}} \\\\\n"; continue
53 + out += f"{l if per == 'full' else ''} & {per} & {r['n_months']} & {r['mean_ls_ann_pct']:+.2f} & {tv(r['hac_t'])} & {tv(r['plain_t'])} & {r['sharpe']:.2f} & {r['capm_alpha_ann_pct']:+.2f} & {r['ff6_alpha_ann_pct']:+.2f} & {tv(r['ff6_alpha_t'])} \\\\\n"
54 + out += "\\addlinespace\n"
55 + return out
56 +hdrA = "\\begin{tabular}{llrrrrrrrr}\n\\toprule\nSignal & Period & Months & Mean (\\%/yr) & HAC $t$ & plain $t$ & Sharpe & CAPM $\\alpha$ (\\%/yr) & FF6 $\\alpha$ (\\%/yr) & FF6 $t$ \\\\\n\\midrule\n"
57 +w("tab_alpha.tex", hdrA + alpha_rows(R["ls_alpha"], [("AI_COMPOS", "AI composite"), ("AI_ADOPT", "AI adoption"), ("AI_INFRA", "AI infrastructure"), ("AI_RISK", "AI disruption risk"), ("AI_PC1", "PC1"), ("AI_formw", "Equal-form composite"), ("SPID_TECH", "SPID tech projects")]) + "\\bottomrule\n\\end{tabular}", "Monthly equal-weighted Q5$-$Q1 portfolios formed every June from the cohort quintiles and held July--June; splice months set to missing. HAC = Newey--West (3 lags). Post = months from July 2023. Deflated Sharpe ratio of the composite (Bailey and L\\'opez de Prado 2014, trials = " + str(R2['mht']['deflated_sharpe']['n_trials']) + " long--short series tried): full " + f"{R2['mht']['deflated_sharpe']['full']['dsr_prob']:.2f}" + ", post " + f"{R2['mht']['deflated_sharpe']['post']['dsr_prob']:.2f}" + " (probability that the true Sharpe exceeds the multiple-testing hurdle).")
58 +body = "\\begin{tabular}{llrrrrrrr}\n\\toprule\nSignal & Sample & Months & Mean (\\%/yr) & Sharpe & FF6 $\\alpha$ full (\\%/yr) & ($t$) & FF6 $\\alpha$ post (\\%/yr) & ($t$) \\\\\n\\midrule\n"
59 +for k, l in SIGL[:5] + [SIGL[7]]:
60 + for sname, sl in [("r3k", "Russell 3000"), ("sp500", "S\\&P 500")]:
61 + f_ = R["samples"][sname]["ls"][k]["full"]; p_ = R["samples"][sname]["ls"][k]["post"]
62 + body += f"{l if sname == 'r3k' else ''} & {sl} & {f_['n_months']} & {f_['mean_ls_ann_pct']:+.2f} & {f_['sharpe']:.2f} & {f_['ff6_alpha_ann_pct']:+.2f} & ({tv(f_['ff6_alpha_t'])}) & {p_.get('ff6_alpha_ann_pct', float('nan')):+.2f} & ({tv(p_.get('ff6_alpha_t'))}) \\\\\n"
63 +w("tab_alpha_both.tex", body + "\\bottomrule\n\\end{tabular}", "Annually selected (June), monthly equal-weighted long--short portfolios; HAC $t$ (3 lags).")
64 +# ---------------- FE panel (T8) ----------------
65 +PF = R["panel_fe"]; vl = {"z_AI_ADOPT": "AI adoption ($z$)", "z_AI_INFRA": "AI infrastructure ($z$)", "z_AI_RISK": "AI disruption risk ($z$)", "z_AI_COMPOS": "AI composite ($z$)", "z_SPID_TECH": "SPID tech projects ($z$)", "z_size": "Size ($z$)", "z_mom": "Momentum ($z$)", "z_vol": "Volatility ($z$)"}
66 +specs = list(PF); body = "\\begin{tabular}{l" + "r" * len(specs) + "}\n\\toprule\n & " + " & ".join(f"({i + 1})" for i in range(len(specs))) + " \\\\\n\\midrule\n"
67 +for v in vl:
68 + if not any(v in PF[s_]["coefs"] for s_ in specs): continue
69 + body += vl[v] + " & " + " & ".join(f"{PF[s_]['coefs'][v]['coef'] * 100:+.2f}" if v in PF[s_]["coefs"] else "" for s_ in specs) + " \\\\\n & " + " & ".join(f"({PF[s_]['coefs'][v]['t']:.2f})" if v in PF[s_]["coefs"] else "" for s_ in specs) + " \\\\\n"
70 +body += "\\midrule\nObservations & " + " & ".join(n(PF[s_]["n"]) for s_ in specs) + " \\\\\nWithin $R^2$ & " + " & ".join(f"{PF[s_]['r2_within']:.4f}" for s_ in specs) + " \\\\\n\\bottomrule\n\\end{tabular}"
71 +w("tab_panel.tex", body, "Firm and year fixed effects; standard errors two-way clustered by firm and year; dependent variable = forward return winsorized within formation year; controls winsorized within year; (8) = composite alone.")
72 +# ---------------- FM (T7, both) ----------------
73 +FM = R["fama_macbeth"]; FB = FM["compos"]["z_AI_COMPOS"]["fixed_b_cv05"]
74 +def fm_cell(blk, v): c = blk[v]; return f"{c['mean_coef'] * 100:+.2f} & ({tv(c['t'])}) & ({tv(c['t_plain'])}) & {ci(c['ci95'], 2)}"
75 +body = "\\begin{tabular}{llrrrr}\n\\toprule\nRegressor & Returns & Slope (pp/SD) & (NW $t$) & (plain $t$) & 95\\% CI \\\\\n\\midrule\n"
76 +for spec, v, l in [("compos", "z_AI_COMPOS", "AI composite"), ("all", "z_AI_ADOPT", "AI adoption (joint)"), ("all", "z_AI_INFRA", "AI infrastructure (joint)"), ("all", "z_AI_RISK", "AI disruption risk (joint)"), ("risk", "z_AI_RISK", "AI disruption risk (alone)"), ("pc1", "z_AI_PC1", "PC1 of three facets"), ("formw", "z_AI_formw", "Equal-form composite"), ("tenk", "z_AI_10k", "10-K composite"), ("spid", "z_SPID_TECH", "SPID tech projects ($z$)"), ("spid_any_ai", "spid_ai_any", "Any AI project (0/1, pp)"), ("compos_cyber", "z_CYBER", "Cybersecurity ($z$, with composite)"), ("compos", "z_size", "Size"), ("compos", "z_mom", "Momentum"), ("compos", "z_vol", "Volatility")]:
77 + for sfx, rl in [("", "raw"), ("_w", "wins.")]:
78 + if spec + sfx in FM and v in FM[spec + sfx]: body += f"{l if sfx == '' else ''} & {rl} & {fm_cell(FM[spec + sfx], v)} \\\\\n"
79 + body += "\\addlinespace\n"
80 +w("tab_fm.tex", body + f"\\midrule\nYears & & {FM['compos']['n_years']} & \\multicolumn{{3}}{{l}}{{mean cross-section {n(FM['compos']['n_mean_per_year'])} firms}} \\\\\n\\bottomrule\n\\end{{tabular}}", f"Annual cross-sectional regressions of the forward return on the signal and size, momentum and volatility (all standardized within year; size from within-year winsorized log cap); slope = time-series mean of the {FM['compos']['n_years']} annual slopes. NW $t$ uses 3 Bartlett lags; the Kiefer--Vogelsang fixed-$b$ 5\\% critical value for this bandwidth and $T$ = 15 is {FB:.2f} (simulated), so NW $t$-statistics between 1.96 and {FB:.2f} are not significant at 5\\% under fixed-$b$ asymptotics. `raw' / `wins.' = raw / within-year 1/99-winsorized forward returns.")
81 +body = "\\begin{tabular}{llrrrrrr}\n\\toprule\nFama--MacBeth slope (pp per SD) & Returns & Russell 3000 & (NW $t$) & 95\\% CI & S\\&P 500 & (NW $t$) & 95\\% CI \\\\\n\\midrule\n"
82 +for spec, v, l in [("compos", "zz_AI_COMPOS", "AI composite"), ("all", "zz_AI_ADOPT", "AI adoption (joint)"), ("all", "zz_AI_INFRA", "AI infrastructure (joint)"), ("all", "zz_AI_RISK", "AI disruption risk (joint)"), ("pc1", "zz_AI_PC1", "PC1"), ("formw", "zz_AI_formw", "Equal-form composite"), ("spid", "zz_SPID_TECH", "SPID tech projects"), ("compos", "zz_size", "Size"), ("compos", "zz_mom", "Momentum"), ("compos", "zz_vol", "Volatility")]:
83 + for sfx, rl in [("", "raw"), ("_w", "wins.")]:
84 + a = R["samples"]["r3k"]["fm"][spec + sfx][v]; b = R["samples"]["sp500"]["fm"][spec + sfx][v]; body += f"{l if sfx == '' else ''} & {rl} & {a['mean_coef'] * 100:+.2f} & ({tv(a['t'])}) & {ci(a['ci95'], 2)} & {b['mean_coef'] * 100:+.2f} & ({tv(b['t'])}) & {ci(b['ci95'], 2)} \\\\\n"
85 +w("tab_fm_both.tex", body + "\\bottomrule\n\\end{tabular}", "Controls: size, momentum, volatility (standardized within year and panel). NW $t$ (3 lags) over 15 annual slopes; fixed-$b$ 5\\% critical value " + f"{FB:.2f}" + ".")
86 +# ---------------- break + subperiod (T9 legacy) ----------------
87 +B = R["structural_break"]; SP = R["subperiod_spread"]; body = "\\begin{tabular}{lrrrrrrrrr}\n\\toprule\nSignal & Pre slope & Post extra & $t$ (firm) & $t$ (firm$\\times$year) & wild $p$ (Webb) & Pre Q5$-$Q1 (wins.) & ($n$, NW $t$) & Post Q5$-$Q1 (wins.) & ($n$, NW $t$) \\\\\n\\midrule\n"
88 +for k, l in [("AI_ADOPT", "AI adoption"), ("AI_INFRA", "AI infrastructure"), ("AI_COMPOS", "AI composite"), ("AI_RISK", "AI disruption risk"), ("AI_PC1", "PC1"), ("AI_formw", "Equal-form"), ("SPID_TECH", "SPID tech projects")]:
89 + b = B[k]; p_ = SP[k]; body += f"{l} & {b['pre_slope'] * 100:+.2f} & {b['interaction'] * 100:+.2f} & {tv(b['interaction_t_firm'])} & {tv(b['interaction_t_twoway'])} & {pv(b.get('interaction_p_wild_webb'))} & {pp(p_['pre_w']['mean'])} & ({p_['pre_w']['n']}, {tv(p_['pre_w']['t'])}) & {pp(p_['post_w']['mean'])} & ({p_['post_w']['n']}, {tv(p_['post_w']['t'])}) \\\\\n"
90 +w("tab_break.tex", body + "\\bottomrule\n\\end{tabular}", "Pooled regression of the within-year winsorized forward return on the signal, a post-2022 cohort dummy, their interaction, size and momentum; wild-cluster bootstrap by cohort (Webb six-point weights, 9,999 draws, 15 clusters). NW $t$ on three post cohorts has no useful sampling distribution and is printed with $n$ only.")
91 +# ---------------- pre/post both (T9) ----------------
92 +body = "\\begin{tabular}{lllrrrrrrrr}\n\\toprule\nSignal & Sample & Returns & Pre Q5$-$Q1 & (95\\% CI) & Post Q5$-$Q1 & (95\\% CI, $n$) & Post cohorts 2022 / 2023 / 2024 & Pooled post ($t$ firm / two-way; wild $p$) & Post $-$ pre pooled ($t$ firm) & Post Sharpe \\\\\n\\midrule\n"
93 +for k, l in SIGL[:5] + [SIGL[7]]:
94 + for sname, sl in [("r3k", "Russell 3000"), ("sp500", "S\\&P 500")]:
95 + bl = R["samples"][sname]; ls_ = bl["ls"][k]["post"]
96 + for sfx, rl in [("", "raw"), ("_w", "wins."), ("_med", "median")]:
97 + su = bl["sub"][k]; coh = " / ".join(f"{v * 100:+.1f}" for y, v in su["post" + sfx]["by_cohort"].items())
98 + pooled = (f"{bl['pooled_post' + sfx]['coef_pct']:+.1f} ({bl['pooled_post' + sfx]['t_firm']:.2f} / {tv(bl['pooled_post' + sfx].get('t_twoway'))}; {pv(bl['pooled_post' + sfx].get('p_wild_webb'))})" if k == "AI_COMPOS" and sfx != "_med" else "")
99 + dif = f"{R['pooled']['diff' + sfx]['coef_pct']:+.1f} ({R['pooled']['diff' + sfx]['t_firm']:.2f})" if (k == "AI_COMPOS" and sname == "r3k" and sfx != "_med") else ""
100 + body += f"{l if (sname == 'r3k' and sfx == '') else ''} & {sl if sfx == '' else ''} & {rl} & {pp(su['pre' + sfx]['mean'])} & {ci(su['pre' + sfx]['ci95'])} & {pp(su['post' + sfx]['mean'])} & ({ci(su['post' + sfx]['ci95'])}, {su['post' + sfx]['n']}) & {coh} & {pooled} & {dif} & {ls_['sharpe'] if sfx == '' and 'sharpe' in ls_ else ''} \\\\\n"
101 + body += "\\addlinespace\n"
102 +w("tab_prepost_both.tex", body + "\\bottomrule\n\\end{tabular}", "Pre = cohorts 2010--2021 (12), post = 2022--2024 (3). Cohort-level 95\\% CIs are $t$-intervals over cohorts (post: 3 cohorts). Pooled post = firm-level regression of Q1/Q5 forward returns on a Q5 dummy with cohort fixed effects: firm-clustered $t$, firm $\\times$ cohort two-way $t$ and the wild-cluster bootstrap $p$ by cohort (Webb weights; only 3 clusters, so the bootstrap distribution is coarse). Post $-$ pre = the hi $\\times$ post interaction in the pooled pre+post regression (firm-clustered). Minimum detectable post $-$ pre difference at 80\\% power: " + f"{R['pooled']['diff_w']['mde_pct_firm']:.1f}" + " pp (winsorized).")
103 +# ---------------- cohorts (T10) ----------------
104 +C, Cw, Cm = R2["cohorts"], R2["cohorts_w"], R2["cohorts_med"]; PO = R["pooled"]; PB = R2["placebo_breaks"]
105 +body = "\\begin{tabular}{lrrr@{\\hspace{2em}}lr}\n\\toprule\nCohort & Q5$-$Q1 raw (pp) & wins. & median & Pooled / permutation test & Estimate \\\\\n\\midrule\n"
106 +cs = list(C["cohort_spreads_pct"].items()); pooled = [("Post cohorts, Q5 vs Q1, raw: firm $t$ / two-way $t$ / wild $p$", f"{PO['post']['coef_pct']:+.1f} ({PO['post']['t_firm']:.2f} / {tv(PO['post'].get('t_twoway'))} / {pv(PO['post'].get('p_wild_webb'))}), $N$={n(PO['post']['n'])}"), ("Post, winsorized", f"{PO['post_w']['coef_pct']:+.1f} ({PO['post_w']['t_firm']:.2f} / {tv(PO['post_w'].get('t_twoway'))} / {pv(PO['post_w'].get('p_wild_webb'))})"), ("Post 2022--2023 only (wins.)", f"{PO['post_excl_2024_w']['coef_pct']:+.1f} ({PO['post_excl_2024_w']['t_firm']:.2f})"), ("Pre cohorts (wins.)", f"{PO['pre_w']['coef_pct']:+.1f} ({PO['pre_w']['t_firm']:.2f})"), ("Difference post $-$ pre (wins.): firm / two-way / wild $p$; MDE", f"{PO['diff_w']['coef_pct']:+.1f} ({PO['diff_w']['t_firm']:.2f} / {PO['diff_w']['t_twoway']:.2f} / {pv(PO['diff_w']['p_wild_webb'])}); {PO['diff_w']['mde_pct_firm']:.1f} pp"),
107 + ("Post mean / leave-one-out (raw)", f"{C['post_mean_pct']:+.1f} / " + ", ".join(f"{v:+.1f}" for v in C["post_loo_pct"].values())), ("Post mean / leave-one-out (wins.)", f"{Cw['post_mean_pct']:+.1f} / " + ", ".join(f"{v:+.1f}" for v in Cw["post_loo_pct"].values())), ("Pre mean, 95\\% CI (raw / wins.)", f"{C['pre_mean_pct']:+.1f} {C['pre_ci95_pct']} / {Cw['pre_mean_pct']:+.1f} {Cw['pre_ci95_pct']}"),
108 + ("Permutation $p$, 2022--24 vs all 455 three-cohort blocks (raw / wins.)", f"{PB['permutation']['p_one_sided']:.2f} (rank {PB['permutation']['rank_among_subsets']}) / {PB['winsorized']['permutation']['p_one_sided']:.2f} (rank {PB['winsorized']['permutation']['rank_among_subsets']})"), ("Contiguous three-cohort blocks above 2022--24 (raw / wins.)", f"{PB['permutation']['rank_among_contiguous'] - 1} / {PB['winsorized']['permutation']['rank_among_contiguous'] - 1} of 12"), ("sup-Wald mean-shift test (Andrews 1993, 15\\% trim), raw / wins.: stat ($p$ simulated)", f"{PB['sup_wald']['sup_wald']:.2f} ({PB['sup_wald']['p_simulated']:.2f}) / {PB['winsorized']['sup_wald']['sup_wald']:.2f} ({PB['winsorized']['sup_wald']['p_simulated']:.2f})")]
109 +for i in range(max(len(cs), len(pooled))):
110 + a = f"{cs[i][0]} & {cs[i][1]:+.1f} & {Cw['cohort_spreads_pct'][cs[i][0]]:+.1f} & {Cm['cohort_spreads_pct'][cs[i][0]]:+.1f}" if i < len(cs) else "& & &"; b = f"{pooled[i][0]} & {pooled[i][1]}" if i < len(pooled) else "&"; body += f"{a} & {b} \\\\\n"
111 +w("tab_cohorts.tex", body + "\\bottomrule\n\\end{tabular}", SE_NOTE)
112 +# ---------------- groups (T11) ----------------
113 +G = R["groups"]; body = "\\begin{tabular}{llrrrrrrrrrr}\n\\toprule\nSample & Returns & Firm-years & Firms & Full Q5$-$Q1 & (NW $t$, CI) & Pre & Post & (CI) & Post LOO 2022/23/24 & Post 2022--23 pooled ($t$) & FF6 $\\alpha$ post ($t$) \\\\\n\\midrule\n"
114 +for g, l in [("large_top1000", "Largest 1,000 by cap (corrected)"), ("small_rest", "Remaining firms (corrected)"), ("large_pct42", "Largest 42\\% within cohort"), ("small_pct42", "Remaining 58\\%"), ("large_top1000_v1", "Largest 1,000 (author's cap)"), ("small_rest_v1", "Remaining (author's cap)"), ("sp500", "S\\&P 500 members"), ("non_sp500", "Non-members")]:
115 + r = G[g]
116 + for sfx, rl in [("", "raw"), ("_w", "wins."), ("_med", "median")]:
117 + loo = " / ".join(f"{v * 100:+.1f}" for v in r["post_loo" + sfx].values()); ex24 = f"{r['pooled_post_excl_2024' + sfx]['coef_pct']:+.1f} ({r['pooled_post_excl_2024' + sfx]['t_firm']:.2f})" if r.get("pooled_post_excl_2024" + sfx) else ""
118 + body += f"{l if sfx == '' else ''} & {rl} & {n(r['n']) if sfx == '' else ''} & {n(r['firms']) if sfx == '' else ''} & {pp(r['sort_EW' + sfx]['Q5_minus_Q1'])} & ({tv(r['sort_EW' + sfx]['t_stat'])}, {ci(r['sort_EW' + sfx]['ci95'])}) & {pp(r['pre' + sfx]['mean'])} & {pp(r['post' + sfx]['mean'])} & {ci(r['post' + sfx]['ci95'])} & {loo} & {ex24} & {(f'{r['ls']['post']['ff6_alpha_ann_pct']:+.2f} ({tv(r['ls']['post']['ff6_alpha_t'])})') if sfx == '' else ''} \\\\\n"
119 + body += "\\addlinespace\n"
120 +sn = R["size_neutral"]; body += f"Size-neutral quintiles (all) & raw & {n(R['n_obs'])} & {n(R['n_firms'])} & {pp(sn['full']['Q5_minus_Q1'])} & ({tv(sn['full']['t_stat'])}, {ci(sn['full']['ci95'])}) & {pp(sn['pre']['mean'])} & {pp(sn['post']['mean'])} & {ci(sn['post']['ci95'])} & & & \\\\\n & wins. & & & {pp(sn['full_w']['Q5_minus_Q1'])} & ({tv(sn['full_w']['t_stat'])}, {ci(sn['full_w']['ci95'])}) & {pp(sn['pre_w']['mean'])} & {pp(sn['post_w']['mean'])} & {ci(sn['post_w']['ci95'])} & & & \\\\\n"
121 +CT = R["contrasts"]; w("tab_groups.tex", body + "\\bottomrule\n\\end{tabular}", SE_NOTE + f" Size groups are formed at formation on the corrected capitalization ({MD['share_firm_years_changing_size_group'] * 100:.1f}\\% of firm-years change group relative to the author's variable); firm-years without a capitalization ({n(META['n_est_without_cap'])}) are in neither group. Tested contrasts (Table~\\ref{{tab:contrasts}}): hi $\\times$ post $\\times$ large = {CT['large_vs_small_w']['hi_x_post_x_group']['coef_pct']:+.1f} pp (firm $t$ {CT['large_vs_small_w']['hi_x_post_x_group']['t_firm']:.2f}, two-way {CT['large_vs_small_w']['hi_x_post_x_group']['t_twoway']:.2f}, wild $p$ {CT['large_vs_small_w']['hi_x_post_x_group']['p_wild_webb']:.2f}); hi $\\times$ post $\\times$ member = {CT['member_vs_non_w']['hi_x_post_x_group']['coef_pct']:+.1f} pp ($t$ {CT['member_vs_non_w']['hi_x_post_x_group']['t_firm']:.2f} / {CT['member_vs_non_w']['hi_x_post_x_group']['t_twoway']:.2f}).")
122 +# ---------------- contrasts (new T21) ----------------
123 +body = "\\begin{tabular}{llrrrrrr}\n\\toprule\nContrast & Returns & Coefficient (pp) & $t$ (firm) & $t$ (firm$\\times$year) & 95\\% CI (firm) & wild $p$ (Rademacher / Webb) & MDE (80\\% power) \\\\\n\\midrule\n"
124 +for key, l in [("large_vs_small_w", "hi $\\times$ post $\\times$ largest-1,000"), ("large_vs_small_raw", "hi $\\times$ post $\\times$ largest-1,000"), ("member_vs_non_w", "hi $\\times$ post $\\times$ S\\&P member"), ("member_vs_non_raw", "hi $\\times$ post $\\times$ S\\&P member")]:
125 + c = CT[key]; rl = "wins." if key.endswith("_w") else "raw"
126 + for term, tl in [("hi_x_post", "hi $\\times$ post (reference group)"), ("hi_x_group", "hi $\\times$ group (pre)"), ("hi_x_post_x_group", l)]:
127 + x = c[term]; body += f"{tl} & {rl} & {x['coef_pct']:+.2f} & {x['t_firm']:.2f} & {x['t_twoway']:.2f} & [{x['ci95_firm_pct'][0]:+.1f}, {x['ci95_firm_pct'][1]:+.1f}] & {pv(x.get('p_wild_rademacher'))} / {pv(x.get('p_wild_webb'))} & {x['mde_pct_firm']:.1f} \\\\\n"
128 + body += f"\\quad $N$ = {n(c['n'])} & & & & & & & \\\\\n\\addlinespace\n"
129 +for key, l in [("large_minus_small_postpre_boot_w", "(post $-$ pre)$_{\\text{large}}$ $-$ (post $-$ pre)$_{\\text{small}}$, cohort spreads, wins."), ("large_minus_small_postpre_boot_raw", "same, raw"), ("member_minus_non_postpre_boot_w", "(post $-$ pre)$_{\\text{S\\&P}}$ $-$ (post $-$ pre)$_{\\text{non}}$, wins.")]:
130 + c = CT[key]; body += f"{l} & & {c['diff_in_diff_pct']:+.2f} & & & cohort-bootstrap CI [{c['ci95_boot_pct'][0]:+.1f}, {c['ci95_boot_pct'][1]:+.1f}] & & \\\\\n"
131 +w("tab_contrasts.tex", body + "\\bottomrule\n\\end{tabular}", "Pooled regressions of Q1/Q5 forward returns on hi, post, group and all interactions with cohort fixed effects; wild-cluster bootstrap by cohort (15 clusters, 9,999 draws). Cohort-bootstrap CIs resample the 12 pre and 3 post cohort spreads (9,999 draws). MDE = 2.8 $\\times$ firm-clustered SE.")
132 +# ---------------- placebo (T14) ----------------
133 +PL = R2["placebo"]; MH = R2["mht"]["placebo_interactions_yearcl"]; rw = dict(zip(MH["members"], MH["p_romano_wolf"])); hb = dict(zip(MH["members"], MH["p_holm_on_wild"]))
134 +body = "\\begin{tabular}{lrrrrrrrrr}\n\\toprule\nSemantic factor & Full Q5$-$Q1 (wins.) & Pre (wins.) & Post (wins.) & (post, raw / median) & Interaction (pp/SD) & $t$ firm & $t$ firm$\\times$year & wild $p$ & RW $p$ / Holm \\\\\n\\midrule\n"
135 +for k, v in PL.items(): body += f"{k} & {pp(v['full_w']['mean'])} & {pp(v['pre_w']['mean'])} & {pp(v['post_w']['mean'])} & ({pp(v['post']['mean'])} / {pp(v['post_med']['mean'])}) & {v['interaction']['coef_pct']:+.2f} & {v['interaction']['t_firm']:.2f} & {v['interaction']['t_twoway']:.2f} & {pv(v['interaction']['p_wild_webb'])} & {rw[k]:.2f} / {hb[k]:.2f} \\\\\n"
136 +w("tab_placebo.tex", body + "\\bottomrule\n\\end{tabular}", "Eight anchor concepts scored identically. Interaction = signal $\\times$ post in the pooled regression with size and momentum; wild $p$ = Webb six-point wild-cluster bootstrap by cohort; RW = Romano--Wolf step-down adjusted $p$ over the eight interactions on the shared bootstrap draws; Holm on the wild $p$-values. Family size 8.")
137 +# ---------------- robustness (T19) ----------------
138 +rows = []; SN = R2["sector_neutral"]; RSd = R2["residual_sort"]; M7 = R2["mag7"]; AS = R2["alt_signal"]; AH = R2["alt_horizon"]; AB = R2["alt_benchmark"]; EN = R2["entrants"]; BR = R2["breaks"]; CS = RS["conditional_spreads"]; CK = R2["cluster_k"]
139 +def add(l, pre, post, pre_w=None, post_w=None): rows.append((l, pp(pre["mean"]) if pre else "--", f"{tv(pre['t']) if pre else '--'}; {ci(pre['ci95']) if pre else '--'}", pp(post["mean"]) if post else "--", f"{tv(post['t']) if post else '--'}; {ci(post['ci95']) if post else '--'}", pp(pre_w["mean"]) if pre_w else "--", pp(post_w["mean"]) if post_w else "--"))
140 +add("Baseline (EW quintiles)", SP["AI_COMPOS"]["pre"], SP["AI_COMPOS"]["post"], SP["AI_COMPOS"]["pre_w"], SP["AI_COMPOS"]["post_w"]); add("Baseline, strict 12 valid months", SP["AI_COMPOS"]["pre_strict"], SP["AI_COMPOS"]["post_strict"]); add("Baseline, median spread", SP["AI_COMPOS"]["pre_med"], SP["AI_COMPOS"]["post_med"])
141 +add("Sector-neutral composite", SN["pre"], SN["post"], SN["pre_w"], SN["post_w"]); add("Residual sort (sector$\\times$year FE + controls)", RSd["pre_w"], RSd["post_w"]); add("Excluding Magnificent-7", M7["ex_mag7_pre"], M7["ex_mag7_post"], M7["ex_mag7_pre_w"], M7["ex_mag7_post_w"]); add("Excluding 500 largest", M7["ex_top500_pre"], M7["ex_top500_post"], M7["ex_top500_pre_w"], M7["ex_top500_post_w"])
142 +for s_, l in [("mean", "Signal: unweighted mean"), ("top5pct", "Signal: top-5\\% chunks")]: add(l, AS[s_]["pre"], AS[s_]["post"], AS[s_]["pre_w"], AS[s_]["post_w"])
143 +add("Outcome: IWV-adjusted return", AB["iwv_adj"]["pre"], AB["iwv_adj"]["post"], AB["iwv_adj"]["pre_w"], AB["iwv_adj"]["post_w"]); add("Horizon 6 months", AH["6m"]["pre"], AH["6m"]["post"], AH["6m"]["pre_w"], AH["6m"]["post_w"]); add("Horizon 24 months (post = 2022--23)", AH["24m"]["pre"], AH["24m"]["post"], AH["24m"]["pre_w"], AH["24m"]["post_w"])
144 +add("PCA first component", SP["AI_PC1"]["pre"], SP["AI_PC1"]["post"], SP["AI_PC1"]["pre_w"], SP["AI_PC1"]["post_w"]); add("AI disruption risk facet", SP["AI_RISK"]["pre"], SP["AI_RISK"]["post"], SP["AI_RISK"]["pre_w"], SP["AI_RISK"]["post_w"]); add("Equal-form composite", SP["AI_formw"]["pre"], SP["AI_formw"]["post"], SP["AI_formw"]["pre_w"], SP["AI_formw"]["post_w"]); add("10-K composite", SP["AI_10k"]["pre"], SP["AI_10k"]["post"], SP["AI_10k"]["pre_w"], SP["AI_10k"]["post_w"])
145 +add("Cyber-neutral composite (quintiles within cybersecurity tercile)", CK["cyber"]["cyberneutral_sort"]["pre_w"], CK["cyber"]["cyberneutral_sort"]["post_w"]); add("Seasoned firms (listed before July 2009)", EN["seasoned"]["pre"], EN["seasoned"]["post"], EN["seasoned"]["pre_w"], EN["seasoned"]["post_w"]); add("Listed $\\geq$ 36 months", EN["listed36"]["pre"], EN["listed36"]["post"], EN["listed36"]["pre_w"], EN["listed36"]["post_w"])
146 +add("Split at 2022 (pre 2010--21 / post 2022--24)", BR["split_2022"]["pre"], BR["split_2022"]["post"], BR["split_2022"]["pre_w"], BR["split_2022"]["post_w"]); add("Split at 2021 (pre 2010--20 / post 2021--24)", BR["split_2021"]["pre"], BR["split_2021"]["post"], BR["split_2021"]["pre_w"], BR["split_2021"]["post_w"])
147 +add("Size-group $\\times$ cohort demeaned returns (wins.)", CS["size_demeaned_w"]["pre"], CS["size_demeaned_w"]["post"]); add("Below-median forward volatility (wins.)", CS["low_fwd_vol_w"]["pre"], CS["low_fwd_vol_w"]["post"]); add("Below the cohort's 99th return percentile (raw)", CS["below_p99_raw"]["pre"], CS["below_p99_raw"]["post"])
148 +body = "\\begin{tabular}{lrlrlrr}\n\\toprule\nSpecification & Pre Q5$-$Q1 & (NW $t$; 95\\% CI) & Post Q5$-$Q1 & (NW $t$; 95\\% CI) & Pre (wins.) & Post (wins.) \\\\\n\\midrule\n" + "".join(f"{a} & {b} & {c} & {d} & {e} & {f} & {g} \\\\\n" for a, b, c, d, e, f, g in rows)
149 +MR = R2["mht"]; w("tab_robust.tex", body + "\\bottomrule\n\\end{tabular}", SE_NOTE + f" The pooled firm-clustered post $-$ pre test for the two splits (wins.): 2022 split {BR['split_2022']['pooled_diff_w']['coef_pct']:+.1f} pp ($t$ {BR['split_2022']['pooled_diff_w']['t_firm']:.2f} firm / {BR['split_2022']['pooled_diff_w']['t_twoway']:.2f} two-way), 2021 split {BR['split_2021']['pooled_diff_w']['coef_pct']:+.1f} pp ($t$ {BR['split_2021']['pooled_diff_w']['t_firm']:.2f} / {BR['split_2021']['pooled_diff_w']['t_twoway']:.2f}). Romano--Wolf adjusted $p$ over the 12 robustness rows (wins.): smallest pre {min(MR['robustness_pre_w']['p_romano_wolf']):.2f}, smallest post {min(MR['robustness_post_w']['p_romano_wolf']):.2f}.")
150 +# ---------------- sector / within-sector + TECH (T13) ----------------
151 +Lc = R2["sector_controls"]; M = R2["monthly_ls"]["AI_COMPOS"]; body = "\\begin{tabular}{llrrr}\n\\toprule\nEstimator & Returns / sample & Pre & Post & Full \\\\\n\\midrule\n"
152 +for samp, sl in [("all", "all sectors"), ("ex_energy", "ex-Energy")]:
153 + for tag, rl in [("raw", "raw"), ("w", "wins.")]:
154 + for st, stl in [("nosector", "FM slope, no sector dummies (pp/SD)"), ("sector", "FM slope, sector dummies")]:
155 + cells = []
156 + for per in ["pre", "post", "full"]:
157 + x = Lc[f"fm_{samp}_{tag}_{st}_{per}"]; cells.append(f"{x['slope_pct']:+.2f} (NW {tv(x['t_nw'])}, plain {tv(x['t_plain'])})")
158 + body += f"{stl} & {rl}, {sl} & " + " & ".join(cells) + " \\\\\n"
159 + for per_lbl, key in [("Pooled, year FE (firm $t$ / two-way $t$ / wild $p$)", "year_fe"), ("Pooled, sector$\\times$year FE", "sector_year_fe")]:
160 + cells = [f"{Lc[f'pooled_{samp}_{per}'][key]['coef_pct']:+.2f} ({Lc[f'pooled_{samp}_{per}'][key]['t_firm']:.2f} / {Lc[f'pooled_{samp}_{per}'][key]['t_twoway']:.2f} / {pv(Lc[f'pooled_{samp}_{per}'][key].get('p_wild_webb'))})" for per in ["pre", "post", "full"]]; body += f"{per_lbl} & wins., {sl} & " + " & ".join(cells) + " \\\\\n"
161 + body += "\\addlinespace\n"
162 +body += f"Sector-neutral quintile sort, Q5$-$Q1 (pp) & raw & {pp(SN['pre']['mean'])} ({tv(SN['pre']['t'])}) & {pp(SN['post']['mean'])} ({tv(SN['post']['t'])}) & {pp(SN['full']['mean'])} ({tv(SN['full']['t'])}) \\\\\n & wins. & {pp(SN['pre_w']['mean'])} ({tv(SN['pre_w']['t'])}) & {pp(SN['post_w']['mean'])} ({tv(SN['post_w']['t'])}) & {pp(SN['full_w']['mean'])} ({tv(SN['full_w']['t'])}) \\\\\n & median & & & {pp(SN['full_med']['mean'])} ({tv(SN['full_med']['t'])}) \\\\\n & wins., ex-Energy & & & {pp(SN['full_ex_energy_w']['mean'])} ({tv(SN['full_ex_energy_w']['t'])}) \\\\\n"
163 +body += f"Residual sort (returns orthogonalized to sector$\\times$year FE + controls) & wins. & {pp(RSd['pre_w']['mean'])} ({tv(RSd['pre_w']['t'])}) & {pp(RSd['post_w']['mean'])} ({tv(RSd['post_w']['t'])}) & {pp(RSd['full_w']['mean'])} ({tv(RSd['full_w']['t'])}) \\\\\n\\midrule\n"
164 +for lab_, key in [("Monthly LS mean (\\%/yr)", "mean_ann_pct"), ("Sharpe", "sharpe"), ("FF6 $\\alpha$ (\\%/yr)", "ff6_alpha_pct"), ("FF6 $t$", "ff6_t"), ("FF6+TECH $\\alpha$ (\\%/yr)", "ff6_tech_alpha_pct"), ("FF6+TECH $t$", "ff6_tech_t"), ("$\\beta_{TECH}$", "tech_beta")]:
165 + body += f"{lab_} & monthly & " + " & ".join(f"{M[p_].get(key, float('nan')):.2f}" for p_ in ["pre", "post", "full"]) + " \\\\\n"
166 +w("tab_sector_tech.tex", body + "\\bottomrule\n\\end{tabular}", f"Three within-sector estimators side by side. NW $t$ on 15 annual slopes (3 lags) vs the plain $t$; the fixed-$b$ 5\\% critical value for the NW $t$ is {FB:.2f}. Pooled regressions: firm-clustered $t$, two-way firm $\\times$ cohort $t$, Webb wild-cluster bootstrap $p$ by cohort. $N$ (pooled, full, all sectors) = {n(Lc['pooled_all_full']['n'])}.")
167 +# ---------------- filing type (T18) ----------------
168 +N = R2["filing_type"]; body = "\\begin{tabular}{lrrrrrrr}\n\\toprule\nSignal built from & Firm-years covered & Corr. w/ baseline & Full Q5$-$Q1 (wins.) & Pre (wins.) & Post (wins.) & ($n$, NW $t$) & FM slope wins. (NW $t$ / plain $t$) \\\\\n\\midrule\n"
169 +for t, l in [("10k", "Form 10-K only"), ("10q", "Form 10-Q only"), ("8k", "Form 8-K only"), ("formw", "Equal-form composite (mean of three $z$)")]:
170 + v = N[t]; fmv = N["fm_forms_w"]["single"].get(f"z_AI_{t}") if t != "formw" else None; fms = f"{fmv['slope_pct']:+.2f} ({tv(fmv['t'])} / {tv(fmv['t_plain'])})" if fmv else (f"{FM['formw_w']['z_AI_formw']['mean_coef'] * 100:+.2f} ({tv(FM['formw_w']['z_AI_formw']['t'])} / {tv(FM['formw_w']['z_AI_formw']['t_plain'])})")
171 + body += f"{l} & {v['share_firm_years'] * 100:.1f}\\% & {v['corr_with_baseline']:.2f} & {pp(v['full_w']['mean'])} & {pp(v['pre_w']['mean'])} & {pp(v['post_w']['mean'])} & ({v['post_w']['n']}, {tv(v['post_w']['t'])}) & {fms} \\\\\n"
172 +hr = N["fm_forms_w"]["horse_race"]; wd = N["fm_forms_w"]["wald_equality"]; hr0 = N["fm_forms_raw"]["horse_race"]; wd0 = N["fm_forms_raw"]["wald_equality"]
173 +body += "\\midrule\nHorse race (FM, all three, wins.): & \\multicolumn{7}{l}{" + "; ".join(f"{l}: {hr[f'z_AI_{t}']['slope_pct']:+.2f} ({hr[f'z_AI_{t}']['t']:.2f})" for t, l in [("10k", "10-K"), ("10q", "10-Q"), ("8k", "8-K")]) + f"; HAC Wald test of equal slopes $\\chi^2_2$ = {wd['stat']:.2f}, $p$ = {wd['p']:.2f}}} \\\\\n"
174 +body += "Horse race (raw): & \\multicolumn{7}{l}{" + "; ".join(f"{l}: {hr0[f'z_AI_{t}']['slope_pct']:+.2f} ({hr0[f'z_AI_{t}']['t']:.2f})" for t, l in [("10k", "10-K"), ("10q", "10-Q"), ("8k", "8-K")]) + f"; Wald $\\chi^2_2$ = {wd0['stat']:.2f}, $p$ = {wd0['p']:.2f}}} \\\\\n"
175 +c22 = N["cohort2022"]; body += f"2022 cohort text filed after 30 Nov 2022: & \\multicolumn{{7}}{{l}}{{{c22['share_filings_post_launch'] * 100:.1f}\\% of filings, {c22['share_chunks_post_launch'] * 100:.1f}\\% of chunks; 2022 spread on pre-launch filings only: {c22['spread_prelaunch_text_only_w_pct']:+.1f} pp (wins.) vs {c22['spread_full_text_same_firms_w_pct']:+.1f} on the full-year text, same {n(c22['n_firms_with_prelaunch_filings'])} firms; rank correlation of the two signals {c22['rank_corr_prelaunch_vs_full']:.2f}}} \\\\\n"
176 +w("tab_filing_type.tex", body + "\\bottomrule\n\\end{tabular}", "Form-specific composites = $\\frac12[z(\\text{adoption}) + z(\\text{infrastructure})]$ from the filings of that form only; the equal-form composite averages the three form $z$-scores. Romano--Wolf $p$ over the three form slopes: " + ", ".join(f"{m}: {p:.2f}" for m, p in zip(R2["mht"]["forms_fm_w"]["members"], R2["mht"]["forms_fm_w"]["p_romano_wolf"])) + f". Equation (3) aggregation reproduced from filing-level scores for {c22['aggregation_reproduced_share_1e6'] * 100:.2f}\\% of firm-years.")
177 +# ---------------- event study (T15, T16) ----------------
178 +EB = RE["by_benchmark"]; base = EB["mm_2022"]; HL = [("pre_m20_m1", "Pre-launch block [$-$20, $-$1] (1--29 Nov 2022)"), ("car_m1_p1", "[$-$1, $+$1]"), ("car_0_5", "[0, $+$5]"), ("car_0_20", "[0, $+$20]"), ("car_60d", "60 trading days"), ("car_6m", "125 days (6 months)"), ("car_1y", "250 days (1 year)"), ("car_18m", "395 days (18 months)")]
179 +body = "\\begin{tabular}{lrrrrrrr}\n\\toprule\nHorizon (day 0 = 1 Dec 2022) & Days & High & Low & High$-$Low & CDA $t$ & event-SD $t$ & BMP $t$ (H$-$L) \\\\\n\\midrule\n"
180 +for h, l in HL:
181 + r = base["full"]["horizons"].get(h)
182 + if r: body += f"{l} & {r['days']} & {r['High_pct']:+.2f} & {r['Low_pct']:+.2f} & {r['HL_CAR_pct']:+.2f} & {r['t_cda']:.2f} & {tv(r['t_event_sd'])} & {tv(r.get('t_bmp_HL'))} \\\\\n"
183 +body += "\\midrule\n\\multicolumn{8}{l}{\\textit{Sub-samples, 1-year H$-$L CAR (CDA $t$; BMP $t$)}} \\\\\n"
184 +for s_, l in [("large", "Largest 1,000 (corrected cap)"), ("small", "Remaining firms"), ("large_within", "Largest 1,000, terciles within group"), ("small_within", "Remaining, terciles within group"), ("large_v1", "Largest 1,000 (author's cap)"), ("small_v1", "Remaining (author's cap)"), ("sp500", "S\\&P 500 members"), ("non_sp500", "Non-members")]:
185 + r = base.get(s_)
186 + if r and "car_1y" in r["horizons"]: x = r["horizons"]["car_1y"]; body += f"{l} ($n$ = {r['n_high']}+{r['n_low']}) & {x['days']} & {x['High_pct']:+.2f} & {x['Low_pct']:+.2f} & {x['HL_CAR_pct']:+.2f} & {x['t_cda']:.2f} & {tv(x['t_event_sd'])} & {tv(x.get('t_bmp_HL'))} \\\\\n"
187 +WR = RE["welch_reference"]; w("tab_event.tex", body + "\\bottomrule\n\\end{tabular}", f"IWV market model estimated 3 Jan--31 Oct 2022; {n(RE['n_firms'])} firms with 2021-cohort exposure; terciles on rank. CDA $t$ = H$-$L daily-portfolio CAR divided by the estimation-window SD of the H$-$L portfolio $\\times \\sqrt{{\\text{{days}}}}$ (dependence-robust); event-SD $t$ uses the event-window SD; BMP = Boehmer--Musumeci--Poulsen standardized-CAR difference (independence-based; the Kolari--Pynn\\\"onen factor {base['full']['kp_factor']:.3f} for $\\bar\\rho$ = {base['full']['rho_bar']:.4f} would deflate it further, but the common component nets out of the H$-$L portfolio, which is why the portfolio CDA and the KP-deflated firm-level $t$ differ). Welch $t$ (independence-based, author's statistic) for reference: full {WR['full']['t_welch']:.2f}, largest 1,000 {WR['large']['t_welch']:.2f}, remaining {WR['small']['t_welch']:.2f}, S\\&P {WR['sp500']['t_welch']:.2f}, non-S\\&P {WR['non_sp500']['t_welch']:.2f}. Tercile characteristics: " + "; ".join(f"{k}: $\\beta$ {v['beta']:.2f}, log cap {v['log_mktcap']:.2f}, share large {v['share_large'] * 100:.0f}\\%" for k, v in RE["tercile_characteristics"].items()) + ".")
188 +body = "\\begin{tabular}{llrrrrrr}\n\\toprule\nBenchmark & Statistic & Full & Largest 1,000 & Remaining & S\\&P 500 & Non-S\\&P & Within-size large / small \\\\\n\\midrule\n"
189 +for b_, l in [("mm_2022", "(a) IWV market model, est. Jan--Oct 2022"), ("beta_only", "(b) Beta-only, AR $= r - \\hat\\beta m$"), ("market_adjusted", "(c) Market-adjusted, $r - m$"), ("mm_est2021", "(d) Market model estimated over 2021"), ("size_mom_matched", "(f) Size $\\times$ momentum matched")]:
190 + for stat, sl in [("HL_CAR_pct", "1-y H$-$L (pp)"), ("t_cda", "CDA $t$")]:
191 + cells = []
192 + for s_ in ["full", "large", "small", "sp500", "non_sp500"]:
193 + r = EB[b_].get(s_); cells.append(f"{r['horizons']['car_1y'][stat]:+.2f}" if (r and 'car_1y' in r['horizons'] and stat == 'HL_CAR_pct') else (f"{r['horizons']['car_1y'][stat]:.2f}" if r and 'car_1y' in r['horizons'] else "--"))
194 + wi = " / ".join(f"{EB[b_][x]['horizons']['car_1y'][stat]:.2f}" if EB[b_].get(x) else "--" for x in ["large_within", "small_within"]); body += f"{l if stat == 'HL_CAR_pct' else ''} & {sl} & " + " & ".join(cells) + f" & {wi} \\\\\n"
195 + body += "\\addlinespace\n"
196 +FF = RE["ff6_calendar_time"]; body += "(e) FF5+MOM calendar-time, 250 days & $\\alpha \\times$ days (pp) & " + " & ".join(f"{FF[s_]['car_1y']['ff6_alpha_cum_pct']:+.2f}" if s_ in FF and 'car_1y' in FF[s_] else "--" for s_ in ["full", "large", "small", "sp500", "non_sp500"]) + " & \\\\\n & HAC $t$ & " + " & ".join(f"{FF[s_]['car_1y']['ff6_t']:.2f}" if s_ in FF and 'car_1y' in FF[s_] else "--" for s_ in ["full", "large", "small", "sp500", "non_sp500"]) + " & \\\\\n"
197 +PS = RE["placebo_summary"]; body += "\\midrule\n\\multicolumn{8}{l}{Placebo launches (30 Nov of each year, same design): 1-year H$-$L (pp) / CDA $t$: " + "; ".join(f"{y}: {PS['HL_1y_by_launch_pct'][y]:+.1f} / {PS['t_cda_by_launch'][y]:.2f}" for y in sorted(PS["HL_1y_by_launch_pct"])) + f". 2022 ranks {PS['rank_2022_by_HL']} of {PS['n_launches']} by H$-$L and {PS['rank_2022_by_t']} by $t$.}} \\\\\n"
198 +ME = RE["mht_event"]; w("tab_event_cda.tex", body + "\\bottomrule\n\\end{tabular}", f"Every cell is dependence-robust (H$-$L daily portfolio). Holm-adjusted $p$ over the {ME['family_size']} sub-sample $\\times$ horizon CDA statistics of benchmark (a): smallest {min(ME['p_holm']):.3f}; BH smallest {min(ME['p_bh']):.3f}. Size $\\times$ momentum matching uses 5 $\\times$ 5 cells of the 2021-cohort panel (B/M not available).")
199 +# ---------------- DiD + SPID (T17) ----------------
200 +Dd = R2["did"]; Pp = R2["spid"]; body = "\\begin{tabular}{lrrrr}\n\\toprule\nSpecification & Coefficient (pp) & $t$ (firm) & $t$ (firm$\\times$year) & 95\\% CI (firm) \\\\\n\\midrule\n"
201 +body += f"DiD: treated (above-median 2019--21 exposure) $\\times$ post, firm + year FE & {Dd['binary']['coef'] * 100:+.2f} & & {Dd['binary']['t']:.2f} & \\\\\nDiD: continuous exposure $\\times$ post, firm + year FE & {Dd['continuous']['coef'] * 100:+.2f} & & {Dd['continuous']['t']:.2f} & \\\\\n\\midrule\n"
202 +for k, l in [("z_SPID_TECH", "SPID technology projects ($z$)"), ("spid_ai_any", "Any AI-initiative project (0/1)"), ("spid_tech_any", "Any technology project (0/1)")]:
203 + for v, vl_ in [(k, l), (f"{k}:post", l + " $\\times$ post"), ("z_AI_COMPOS", "AI composite ($z$)"), ("z_AI_COMPOS:post", "AI composite $\\times$ post"), ("post", "Post-launch cohorts")]:
204 + c = Pp[k]["terms"].get(v)
205 + if c: body += f"{vl_} & {c['coef_pct']:+.2f} & {c['t_firm']:.2f} & {c['t_twoway']:.2f} & [{c['ci95_firm_pct'][0]:+.2f}, {c['ci95_firm_pct'][1]:+.2f}] \\\\\n"
206 + body += f"\\quad $N$ = {n(Pp[k]['n'])}; design rank {Pp[k]['rank']}, condition number {Pp[k]['cond']:.1f} \\\\\n\\addlinespace\n"
207 +w("tab_did_spid.tex", body + "\\bottomrule\n\\end{tabular}", "Explicit-terms formula (signal, post, signal $\\times$ post, composite, composite $\\times$ post, size, momentum), rebuilt panel, dependent variable winsorized within year. The committed exhibit (HEAD 3cf7560) printed $t$ = 0.00 throughout from a degenerate fit; the refit of the committed panel (L1) gave SPID $\\times$ post $-1.57$ pp ($t$ $-1.72$ firm / $-0.88$ two-way).")
208 +# ---------------- SPID binary contrasts (new) ----------------
209 +SB = Pp["binary_contrasts"]; body = "\\begin{tabular}{llrrrrr}\n\\toprule\nContrast & Period & Estimate (pp) & $t$ & 95\\% CI & MDE (80\\%) & $n$ \\\\\n\\midrule\n"
210 +for flag, l in [("spid_ai_any", "Any AI project vs none"), ("spid_tech_any", "Any technology project vs none")]:
211 + for per in ["full", "pre", "post"]:
212 + c = SB[flag][per]["cohort_w"]; f_ = SB[flag][per]["firm_w"]
213 + body += f"{l + ', cohort-level (wins.)' if per == 'full' else ''} & {per} & {c['mean_pct'] if c['mean_pct'] is not None else '--'} & {tv(c['t_plain'])} & {c['ci95_pct']} & {c['mde_pct']} & {c['n_cohorts']} cohorts \\\\\n"
214 + body += f"{l + ', firm-level (wins., cohort FE, size, momentum)' if per == 'full' else ''} & {per} & {f_['coef_pct']:+.2f} & {f_['t_firm']:.2f} & [{f_['ci95_pct'][0]:+.1f}, {f_['ci95_pct'][1]:+.1f}] & {f_['mde_pct']:.1f} & {n(f_['n_treated'])} treated / {n(f_['n'])} \\\\\n"
215 + body += "\\addlinespace\n"
216 +tc = Pp["tercile_nonzero_w"]; body += f"T3 $-$ T1 among firms with $>$ 0 technology projects (wins.) & full & {tc['mean_pct']:+.2f} & {tv(tc['t_plain'])} & {tc['ci95_pct']} & & {tc['n_cohorts']} cohorts \\\\\n"
217 +w("tab_spid_binary.tex", body + "\\bottomrule\n\\end{tabular}", f"Any-AI prevalence {R['spid']['share_firm_years_any_ai'] * 100:.1f}\\% of firm-years ({R['spid']['share_post_any_ai'] * 100:.1f}\\% post); cohorts enter the cohort-level contrast only with $\\geq$ 15 treated firms. The legacy quintile sort on a variable with median and P75 equal to zero (MDE {Pp.get('legacy_quintile_full_mde_pct', R['spid'].get('legacy_quintile_full_mde_pct'))} pp) is retained only in Table~\\ref{{tab:sorts}} for comparison. Extractor: gpt-4o-mini (FIRC-0001 pipeline snapshot), run 2026-10-27/28; human precision frame of 200 mentions delivered (results/r3k\\_v2/validation), labels pending.")
218 +# ---------------- reversal + cyber (new, cluster K) ----------------
219 +RV = CK["reversal"]; body = "\\begin{tabular}{lrrrr}\n\\toprule\nTest & Estimate (pp) & NW $t$ / plain $t$ & 95\\% CI & $n$ cohorts \\\\\n\\midrule\n"
220 +for key, l in [("month_1_12_w", "High-language firms WITH vs WITHOUT disclosed tech projects: months 1--12 (wins.)"), ("month_13_24_w", "same, months 13--24 (wins.)"), ("month_1_24_raw", "same, months 1--24 (raw)")]:
221 + c = RV["high_language_with_vs_without_projects"][key]; body += f"{l} & {c['mean_pct'] if c['mean_pct'] is not None else '--'} & {tv(c['t'])} / {tv(c['t_plain'])} & {c['ci95_pct']} & {c['n_cohorts']} \\\\\n"
222 +for key, l in [("w", "Composite Q5$-$Q1, months 13--24 (wins., cohorts $\\leq$ 2023)"), ("raw", "same, raw"), ("pre_w", "months 13--24, pre cohorts (wins.)"), ("post_w", "months 13--24, post cohorts 2022--23 (wins.)")]:
223 + c = RV["Q5_minus_Q1_month_13_24"][key]; body += f"{l} & {pp(c['mean'])} & {tv(c['t'])} / {tv(c['t_plain'])} & {ci(c['ci95'])} & {c['n']} \\\\\n"
224 +c = RV["Q5_minus_Q1_month_1_24"]["w"]; body += f"Composite Q5$-$Q1, months 1--24 (wins.) & {pp(c['mean'])} & {tv(c['t'])} / {tv(c['t_plain'])} & {ci(c['ci95'])} & {c['n']} \\\\\n\\midrule\n"
225 +CY = CK["cyber"]; body += f"corr(cybersecurity $z$, AI composite $z$) & {CY['corr_cyber_composite']:.2f} & & & \\\\\nCyber-neutral composite Q5$-$Q1, full (wins.) & {pp(CY['cyberneutral_sort']['full_w']['mean'])} & {tv(CY['cyberneutral_sort']['full_w']['t'])} / {tv(CY['cyberneutral_sort']['full_w']['t_plain'])} & {ci(CY['cyberneutral_sort']['full_w']['ci95'])} & {CY['cyberneutral_sort']['full_w']['n']} \\\\\nCybersecurity Q5$-$Q1, full / pre / post (wins.) & {pp(CY['cyber_sort']['full_w']['mean'])} / {pp(CY['cyber_sort']['pre_w']['mean'])} / {pp(CY['cyber_sort']['post_w']['mean'])} & {tv(CY['cyber_sort']['full_w']['t'])} & {ci(CY['cyber_sort']['full_w']['ci95'])} & {CY['cyber_sort']['full_w']['n']} \\\\\n"
226 +fc = CY["fm_compos_with_cyber_w"]; body += f"FM, composite with cybersecurity control (wins.): composite / cyber & {fc['z_AI_COMPOS']['mean_coef'] * 100:+.2f} / {fc['z_CYBER']['mean_coef'] * 100:+.2f} & {tv(fc['z_AI_COMPOS']['t'])} / {tv(fc['z_CYBER']['t'])} & & {fc['n_years']} \\\\\n"
227 +w("tab_reversal.tex", body + "\\bottomrule\n\\end{tabular}", f"High-language = top quintile of the composite within cohort; `with projects' = SPID technology-project mentions $>$ 0 ({n(RV['high_language_with_vs_without_projects']['n_high_language_with_projects'])} of {n(RV['high_language_with_vs_without_projects']['n_high_language'])} high-language firm-years). Month 13--24 return $= (1 + r_{{24}}) / (1 + r_{{12}}) - 1$.")
228 +# ---------------- MHT (new T20) ----------------
229 +body = "\\begin{tabular}{llrrrrr}\n\\toprule\nFamily (size) & Member & $t$ & $p$ & Holm & BH & Romano--Wolf \\\\\n\\midrule\n"
230 +for fam, fl in [("sorts_full_w", "Table 5 full-sample spreads, wins. (10)"), ("sorts_post_w", "Table 9 post spreads, wins. (10)"), ("forms_fm_w", "Form FM slopes, wins. (3)"), ("robustness_pre_w", "Table 19 pre rows, wins. (12)"), ("robustness_post_w", "Table 19 post rows, wins. (12)")]:
231 + f_ = MR[fam]
232 + for i, m_ in enumerate(f_["members"]): body += f"{fl if i == 0 else ''} & {m_.replace('_', ' ')} & {f_['t'][i]:.2f} & {f_['p'][i]:.3f} & {f_['p_holm'][i]:.3f} & {f_['p_bh'][i]:.3f} & {f_['p_romano_wolf'][i]:.3f} \\\\\n"
233 + body += "\\addlinespace\n"
234 +f_ = MR["placebo_interactions_yearcl"]
235 +for i, m_ in enumerate(f_["members"]): body += f"{'Table 14 interactions, cohort-clustered (8)' if i == 0 else ''} & {m_} & {f_['t_year'][i]:.2f} & {f_['p_wild_webb'][i]:.3f} (wild) & {f_['p_holm_on_wild'][i]:.3f} & {f_['p_bh_on_wild'][i]:.3f} & {f_['p_romano_wolf'][i]:.3f} \\\\\n"
236 +body += "\\addlinespace\n"; f_ = MR["break_dates_w"]
237 +for i, m_ in enumerate(f_["members"]): body += f"{'Break dates, post - pre (10)' if i == 0 else ''} & {m_} ({f_['diff_pct'][i]:+.1f} pp) & {f_['t_boot_se'][i]:.2f} & {f_['p_boot'][i]:.3f} & & & {f_['p_romano_wolf'][i]:.3f} \\\\\n"
238 +DS = MR["deflated_sharpe"]; body += f"\\midrule\n\\multicolumn{{7}}{{l}}{{Deflated Sharpe ratio of the composite long--short (trials = {DS['n_trials']} series): full {DS['full']['dsr_prob']:.2f} (SR hurdle {DS['full']['sr0_ann']:.2f}), pre {DS['pre']['dsr_prob']:.2f}, post {DS['post']['dsr_prob']:.2f}. HLZ $|t| > 3$ met by {sum(MR['hlz_summary'].values())} statistics across the cohort families.}} \\\\\n"
239 +w("tab_mht.tex", body + "\\bottomrule\n\\end{tabular}", "Cohort families: shared cohort-resampling bootstrap (9,999 draws) with recentred $t$-statistics; Romano--Wolf = step-down max-$t$. Interaction family: shared Webb wild-cluster draws by cohort. Event family in Table~\\ref{tab:event_cda}. Specification curve: " + f"{R2['spec_curve']['n_specs']} specifications, {R2['spec_curve']['share_positive'] * 100:.0f}\\% positive, {R2['spec_curve']['share_positive_sig_nw'] * 100:.0f}\\% positive and NW-significant ({R2['spec_curve']['share_positive_sig_plain'] * 100:.0f}\\% with the plain $t$), {R2['spec_curve']['share_negative_sig_nw'] * 100:.1f}\\% negative and significant; median spread {R2['spec_curve']['median_spread_pct']:+.1f} pp, IQR [{R2['spec_curve']['iqr_spread_pct'][0]:+.1f}, {R2['spec_curve']['iqr_spread_pct'][1]:+.1f}].")
240 +# ---------------- event-time (new, cluster G) ----------------
241 +body = "\\begin{tabular}{lrrrrrr}\n\\toprule\nOutcome cohort & Author coding: absolute slope (pp/SD) & (SE) & Difference from 2021 (pp/SD) & (SE) & Trailing exposure ($Y{-}3$..$Y{-}1$): difference from 2021 & (SE) \\\\\n\\midrule\n"
242 +ET = {r["year"]: r for r in R2["did_eventtime"]["rows"]}; ETt = {r["year"]: r for r in R2["did_eventtime_trailing"]["rows"]}
243 +for y in sorted(ET): r = ET[y]; t_ = ETt.get(y, {}); body += f"{y} & {r['abs_slope'] * 100:+.1f} & ({r['abs_se'] * 100:.1f}) & {r['diff_from_ref'] * 100:+.1f} & ({r['diff_se'] * 100:.1f}) & {t_.get('diff_from_ref', 0) * 100:+.1f} & ({t_.get('diff_se', 0) * 100:.1f}) \\\\\n"
244 +w("tab_eventtime.tex", body + "\\bottomrule\n\\end{tabular}", f"Firm-clustered SEs. Author's coding (no main effect) estimates one absolute slope per cohort; with the main effect restored (2021 slope {R2['did_eventtime']['base_slope_ref_year'] * 100:+.1f} pp/SD, $t$ {R2['did_eventtime']['base_t']:.2f}) the coefficients are differences from 2021. With the 2019--21 exposure, rows before 2019 interact future exposure with past outcomes (look-back, not pre-trends); the trailing-exposure column uses the mean composite over the three years before each outcome year. $N$ = {n(R2['did_eventtime']['n'])}.")
245 +# ---------------- attrition / survivorship (new, cluster D) ----------------
246 +AP = RS["attrition_pivot_absent_share"]; AE = RS["annualised_exit_rate"]; body = "\\begin{tabular}{lrrrrrr}\n\\toprule\nIWV holdings as of & Small tercile & Mid & Large & All & Annualised exit, small / large & Then-members in panel \\\\\n\\midrule\n"
247 +att = pd.DataFrame(RS["attrition"]); allrow = att[att.tercile == "all"].set_index("asof")
248 +for a in sorted(AP): body += f"{a[:4]}-{a[4:6]}-{a[6:]} & {AP[a]['small'] * 100:.1f}\\% & {AP[a]['mid'] * 100:.1f}\\% & {AP[a]['large'] * 100:.1f}\\% & {allrow.loc[a, 'absent_2026_share'] * 100:.1f}\\% & {(AE[a]['small'] * 100 if a in AE else float('nan')):.1f}\\% / {(AE[a]['large'] * 100 if a in AE else float('nan')):.1f}\\% & {allrow.loc[a, 'in_panel_share'] * 100:.1f}\\% \\\\\n"
249 +TM = RS["then_members_present"]; body += "\\midrule\n\\multicolumn{7}{l}{\\textit{Then-members-present cohort spreads (wins.), all panel firms vs firms that were IWV members at formation}} \\\\\n"
250 +for y, v in TM.items(): body += f"Cohort {y} (snapshot {v['snapshot'][:4]}-{v['snapshot'][4:6]}) & \\multicolumn{{6}}{{l}}{{{v['share_member_then'] * 100:.0f}\\% members then; spread all {pp(v['spread_all_w'])} vs members {pp(v['spread_members_w'])} pp; mean return members {pp(v['mean_fwd_members'])} vs non-members {pp(v['mean_fwd_nonmembers']) if v['mean_fwd_nonmembers'] is not None else '--'}}} \\\\\n"
251 +EI = RS["ew_vs_index"]; BD = RS["bounding"]; body += f"\\midrule\n\\multicolumn{{7}}{{l}}{{EW panel return minus IWV total return, mean over cohorts: all {EI['mean_gap_all_minus_iwv_pp']:+.1f} pp (wins. {EI['mean_gap_all_w_minus_iwv_pp']:+.1f}; median {EI['mean_gap_median_minus_iwv_pp']:+.1f}); largest 1,000 {EI['mean_gap_large_minus_iwv_pp']:+.1f}; remaining vs IWM {EI['mean_gap_small_minus_iwm_pp']:+.1f}; S\\&P members {EI['mean_gap_sp500_minus_iwv_pp']:+.1f}; pre {EI['mean_gap_pre_pp']:+.1f} / post {EI['mean_gap_post_pp']:+.1f}.}} \\\\\n"
252 +body += "\\multicolumn{7}{l}{Bounding: Q1 $-$ Q5 exit-rate difference needed to generate the whole observed spread (" + f"{BD['observed_full_spread_w'] * 100:+.1f} pp, wins.) for exit returns of $-$30 / $-$50 / $-$70 / $-$100\\%: " + " / ".join(f"{v * 100:.1f} pp" for v in BD["required_exit_rate_difference_e1_minus_e5"].values()) + f"; measured annualised exit rate, small tercile {max(v for v in BD['measured_annualised_exit_small_tercile'].values() if v is not None) * 100:.1f}\\% at most; forward-volatility gradient Q5 $-$ Q1 {BD['fwd_vol_gradient_Q5_minus_Q1'] * 100:+.1f} pp.}} \\\\\n"
253 +w("tab_attrition.tex", body + "\\bottomrule\n\\end{tabular}", "Point-in-time IWV constituents from the dated iShares holdings files captured by the Internet Archive (17 captures; no capture exists for June 2010--2018). Absent = ISIN/CUSIP not in the 28 Aug 2025 holdings and ticker not in the 2026 universe. Size terciles by market value within the snapshot. Delisted firms have no AI scores, so attrition by AI quintile cannot be computed on this corpus (author-side package X1).")
254 +# ---------------- double sort, corr, top firms, coverage, corpus ----------------
255 +body = "\\begin{tabular}{lllrrrrr}\n\\toprule\nPeriod & Returns & Size tercile & Low AI & Mid & High AI & High$-$Low & $N$ \\\\\n"
256 +for per, key in [("Pre-ChatGPT (2010--2021)", "double_sort_pre"), ("Post-ChatGPT (2022--2024)", "double_sort_post")]:
257 + for sfx, rl in [("", "raw"), ("_w", "wins.")]:
258 + dsx = pd.DataFrame(R2[key + sfx]); body += "\\midrule\n"
259 + for sz in ["Small", "Mid", "Large"]:
260 + if sz in dsx.index: body += f"{per if (sz == 'Small' and sfx == '') else ''} & {rl if sz == 'Small' else ''} & {sz} & {dsx.loc[sz, 'Lo']:.1f} & {dsx.loc[sz, 'Mid']:.1f} & {dsx.loc[sz, 'Hi']:.1f} & {dsx.loc[sz, 'Hi-Lo']:+.1f} & {n(dsx.loc[sz, 'N'])} \\\\\n"
261 +w("tab_doublesort.tex", body + "\\bottomrule\n\\end{tabular}", "Size terciles on the corrected capitalization within cohort (rank-based), AI terciles within size tercile; mean forward return in \\%.")
262 +c = pd.read_csv(f"{T}/t1b_correlations.csv", index_col=0); lab2 = {"z_AI_ADOPT": "AI adoption", "z_AI_INFRA": "AI infrastructure", "z_AI_RISK": "AI disruption risk", "z_AI_PC1": "PC1", "z_SPID_TECH": "SPID tech projects", "z_CYBER": "Cybersecurity", "z_size": "Size", "z_mom": "Momentum", "fwd_ret": "Forward return"}
263 +cols_ = list(c.columns); body = "\\begin{tabular}{l" + "r" * len(cols_) + "}\n\\toprule\n & " + " & ".join(lab2[x] for x in cols_) + " \\\\\n\\midrule\n"
264 +for i_, rname in enumerate(c.index): body += lab2[rname] + " & " + " & ".join(f"{c.loc[rname, cc]:.2f}" if j_ <= i_ else "" for j_, cc in enumerate(cols_)) + " \\\\\n"
265 +w("tab_corr.tex", body + "\\bottomrule\n\\end{tabular}", f"Pearson correlations over the {n(R['n_obs'])} firm-years of the estimation sample (signals standardized within year).")
266 +tf = df.groupby(["ticker", "company_name", "sector"]).agg(z=("z_AI_COMPOS", "mean"), yrs=("fiscal_year", "count")).reset_index(); tf = tf[tf.yrs >= 5].nlargest(12, "z")
267 +body = "\\begin{tabular}{lllrr}\n\\toprule\nTicker & Company & Sector & Mean composite ($z$) & Years \\\\\n\\midrule\n"
268 +for r_ in tf.itertuples(): body += f"{r_.ticker} & {str(r_.company_name).title().replace('&', '\\&')[:34]} & {r_.sector} & {r_.z:.2f} & {r_.yrs} \\\\\n"
269 +w("tab_topfirms.tex", body + "\\bottomrule\n\\end{tabular}", "Firms with at least five cohorts; tickers as listed.")
270 +cv = R2["coverage"]; body = "\\begin{tabular}{lrrrrrrr}\n\\toprule\nCohort & Firms & Largest 1,000 & Remaining & No cap & Largest 42\\% & Median filings & Median chunks \\\\\n\\midrule\n"
271 +for y in cv["firms_per_cohort"]: body += f"{y} & {n(cv['firms_per_cohort'][y])} & {n(cv['n_large'][y])} & {n(cv['n_small'][y])} & {n(cv['n_nocap'][y])} & {n(cv['n_large_pct42'][y])} & {cv['median_filings'][y]:.0f} & {cv['median_chunks'][y]:.0f} \\\\\n"
272 +w("tab_coverage.tex", body + "\\bottomrule\n\\end{tabular}", "Estimation sample by formation cohort; size groups at formation on the corrected capitalization.")
273 +fl = pd.read_parquet("data/raw_russell3000/filings"); body = "\\begin{tabular}{lrrrr}\n\\toprule\nForm & Filings 2010--2026 & Issuers & Parsed & Firm-years with a scored filing (estimation sample) \\\\\n\\midrule\n"
274 +for fm_ in ["10-K", "10-Q", "8-K"]:
275 + sub = fl[fl.form_type == fm_]; tag = {"10-K": "n_10k", "10-Q": "n_10q", "8-K": "n_8k"}[fm_]; body += f"{fm_} & {n(len(sub))} & {n(sub.cik.nunique())} & {n((sub.parse_status == 'complete').sum())} & {df[tag].notna().mean() * 100:.1f}\\% \\\\\n"
276 +body += f"\\midrule\nTotal & {n(len(fl))} & {n(fl.cik.nunique())} & {n((fl.parse_status == 'complete').sum())} & \\\\\n"
277 +w("tab_corpus.tex", body + "\\bottomrule\n\\end{tabular}", "Filings index of the EDGAR Research Database (data/raw\\_russell3000/filings)." + (f" Corpus counts exported by the node audit (results/n1): {N1}" if N1 else ""))
278 +# ---------------- hygiene table (new, cluster B) ----------------
279 +hl = pd.read_csv(f"{T}/hygiene_log.csv"); body = "\\begin{tabular}{llrrrl}\n\\toprule\nTicker & Window & Yahoo return & FinData return & FinData first bar & Verdict \\\\\n\\midrule\n"
280 +for r_ in hl[hl.verdict != "confirmed"].sort_values(["kind", "ticker"]).itertuples(): body += f"{r_.ticker} & {r_.date} & {r_.yahoo_ret * 100:+.0f}\\% & {('--' if pd.isna(r_.findata_ret) else f'{r_.findata_ret * 100:+.0f}\\%')} & {r_.findata_first_bar if isinstance(r_.findata_first_bar, str) else '--'} & {r_.verdict.replace('_', ' ')} \\\\\n"
281 +w("tab_hygiene.tex", body + "\\bottomrule\n\\end{tabular}", f"Second-source verification of every monthly return above 300\\% ({HY['n_flagged_monthly']}) and every firm-year forward return above 500\\% ({HY['n_flagged_firm_years']}) against FinData OS daily bars (split- and dividend-adjusted; {HY['second_source']}). Confirmed observations ({HY['verdicts_monthly'].get('confirmed', 0)} monthly, {HY['verdicts_firm_year'].get('confirmed', 0)} firm-year) are kept; the series breaks listed here are set to missing together with every forward return whose window contains them ({META['n_firm_years_fwd_removed_hygiene']} firm-years). Rule: {HY['rule']}.")
282 +# ---------------- macros ----------------
283 +S = R["sorts"]; sp = SP["AI_COMPOS"]; la = R["ls_alpha"]["AI_COMPOS"]; E1 = base["full"]["horizons"]["car_1y"]; Cc = R2["cohorts"]; SMr = R["samples"]["r3k"]; SMs = R["samples"]["sp500"]; Lc_ = R2["sector_controls"]; Pp_ = R2["spid"]; PLm = R2["placebo"]; Nf = R2["filing_type"]; Gm = R["groups"]
284 +def f1(x): return f"{x:+.1f}"
285 +MAC.update({"nObs": n(R["n_obs"]), "nFirms": n(R["n_firms"]), "nFirmYearsPanel": n(META["n_firm_years"]), "yrs": R["yrs"], "nLarge": n(R["n_large"]), "nSPfy": n(R["n_sp500_firm_years"]),
286 + "spreadFull": pp(S["AI_COMPOS_EW"]["Q5_minus_Q1"]), "spreadFullT": tv(S["AI_COMPOS_EW"]["t_stat"]), "spreadFullVW": pp(S["AI_COMPOS_VW"]["Q5_minus_Q1"]), "spreadFullVWT": tv(S["AI_COMPOS_VW"]["t_stat"]),
287 + "spreadFullW": pp(S["AI_COMPOS_EW_w"]["Q5_minus_Q1"]), "spreadFullWT": tv(S["AI_COMPOS_EW_w"]["t_stat"]), "spreadFullWTplain": tv(S["AI_COMPOS_EW_w"]["t_plain"]), "spreadFullWCI": ci(S["AI_COMPOS_EW_w"]["ci95"]), "spreadFullCI": ci(S["AI_COMPOS_EW"]["ci95"]), "spreadFullTplain": tv(S["AI_COMPOS_EW"]["t_plain"]), "spreadFullMed": pp(S["AI_COMPOS_EW_med"]["Q5_minus_Q1"]), "spreadFullMedT": tv(S["AI_COMPOS_EW_med"]["t_stat"]), "spreadFullVWcap": pp(S["AI_COMPOS_VWcap10"]["Q5_minus_Q1"]), "spreadFullVWcapT": tv(S["AI_COMPOS_VWcap10"]["t_stat"]), "spreadFullStrict": pp(S["AI_COMPOS_EW_strict"]["Q5_minus_Q1"]), "topOnePctContrib": pp(S["AI_COMPOS_EW"]["top1pct_contribution"]),
288 + "spreadPre": pp(sp["pre"]["mean"]), "spreadPreT": tv(sp["pre"]["t"]), "spreadPost": pp(sp["post"]["mean"]), "spreadPostT": tv(sp["post"]["t"]), "spreadPreW": pp(sp["pre_w"]["mean"]), "spreadPreWT": tv(sp["pre_w"]["t"]), "spreadPostW": pp(sp["post_w"]["mean"]), "spreadPostWT": tv(sp["post_w"]["t"]), "spreadPreCI": ci(sp["pre"]["ci95"]), "spreadPostCI": ci(sp["post"]["ci95"]), "spreadPreWCI": ci(sp["pre_w"]["ci95"]), "spreadPostWCI": ci(sp["post_w"]["ci95"]), "spreadPreMed": pp(sp["pre_med"]["mean"]), "spreadPostMed": pp(sp["post_med"]["mean"]),
289 + "ffsixFull": f1(la["full"]["ff6_alpha_ann_pct"]), "ffsixFullT": tv(la["full"]["ff6_alpha_t"]), "ffsixPost": f1(la["post"]["ff6_alpha_ann_pct"]), "ffsixPostT": tv(la["post"]["ff6_alpha_t"]), "sharpePre": f"{la['pre']['sharpe']:.2f}", "sharpePost": f"{la['post']['sharpe']:.2f}", "meanLSPre": f1(la["pre"]["mean_ls_ann_pct"]), "meanLSPost": f1(la["post"]["mean_ls_ann_pct"]), "dsrFull": f"{DS['full']['dsr_prob']:.2f}", "dsrPost": f"{DS['post']['dsr_prob']:.2f}", "dsrTrials": str(DS["n_trials"]),
290 + "cohortsPost": ", ".join(f"{y}: {v:+.1f}" for y, v in Cc["cohort_spreads_pct"].items() if int(y) >= 2022), "cohortsPostW": ", ".join(f"{y}: {v:+.1f}" for y, v in Cw["cohort_spreads_pct"].items() if int(y) >= 2022), "postLOO": ", ".join(f"{v:+.1f}" for v in Cc["post_loo_pct"].values()), "postLOOW": ", ".join(f"{v:+.1f}" for v in Cw["post_loo_pct"].values()), "preMean": f1(Cc["pre_mean_pct"]), "postMean": f1(Cc["post_mean_pct"]), "preMeanW": f1(Cw["pre_mean_pct"]), "postMeanW": f1(Cw["post_mean_pct"]), "preMeanCI": str(Cc["pre_ci95_pct"]).replace("[", "[").replace("]", "]"), "preMin": f1(Cc["pre_min_pct"]),
291 + "pooledPost": f1(PO["post"]["coef_pct"]), "pooledPostT": f"{PO['post']['t_firm']:.2f}", "pooledPostTtw": tv(PO["post"].get("t_twoway")), "pooledPostPwild": pv(PO["post"].get("p_wild_webb")), "pooledPostW": f1(PO["post_w"]["coef_pct"]), "pooledPostWT": f"{PO['post_w']['t_firm']:.2f}", "pooledPostWTtw": tv(PO["post_w"].get("t_twoway")), "pooledPostWPwild": pv(PO["post_w"].get("p_wild_webb")), "pooledPostWCI": f"[{PO['post_w']['ci95_firm_pct'][0]:+.1f}, {PO['post_w']['ci95_firm_pct'][1]:+.1f}]", "pooledDiff": f1(PO["diff"]["coef_pct"]), "pooledDiffT": f"{PO['diff']['t_firm']:.2f}", "pooledDiffW": f1(PO["diff_w"]["coef_pct"]), "pooledDiffWT": f"{PO['diff_w']['t_firm']:.2f}", "pooledDiffWTtw": f"{PO['diff_w']['t_twoway']:.2f}", "pooledDiffWMDE": f"{PO['diff_w']['mde_pct_firm']:.1f}", "pooledPostExTwentyFourW": f1(PO["post_excl_2024_w"]["coef_pct"]), "pooledPostExTwentyFourWT": f"{PO['post_excl_2024_w']['t_firm']:.2f}",
292 + "eventHL": f1(E1["HL_CAR_pct"]), "eventT": f"{WR['full']['t_welch']:.2f}", "eventCDA": f"{E1['t_cda']:.2f}", "eventBMP": tv(E1.get("t_bmp_HL")), "eventN": n(RE["n_firms"]), "eventHLsixty": f1(base["full"]["horizons"]["car_60d"]["HL_CAR_pct"]), "eventCDAsixty": f"{base['full']['horizons']['car_60d']['t_cda']:.2f}", "eventHLpre": f1(base["full"]["horizons"]["pre_m20_m1"]["HL_CAR_pct"]), "eventHLtwenty": f1(base["full"]["horizons"]["car_0_20"]["HL_CAR_pct"]), "eventCDAtwenty": f"{base['full']['horizons']['car_0_20']['t_cda']:.2f}", "eventHLthree": f1(base["full"]["horizons"]["car_m1_p1"]["HL_CAR_pct"]), "evHLeighteen": f1(base["full"]["horizons"]["car_18m"]["HL_CAR_pct"]), "evCDAeighteen": f"{base['full']['horizons']['car_18m']['t_cda']:.2f}", "evCDAsixty": f"{base['full']['horizons']['car_60d']['t_cda']:.2f}", "evRho": f"{base['full']['rho_bar']:.3f}", "evKP": f"{base['full']['kp_factor']:.3f}",
293 + "largePre": pp(Gm["large_top1000"]["pre"]["mean"]), "largePost": pp(Gm["large_top1000"]["post"]["mean"]), "largePostT": tv(Gm["large_top1000"]["post"]["t"]), "smallPre": pp(Gm["small_rest"]["pre"]["mean"]), "smallPost": pp(Gm["small_rest"]["post"]["mean"]), "smallPostT": tv(Gm["small_rest"]["post"]["t"]), "largePreW": pp(Gm["large_top1000"]["pre_w"]["mean"]), "largePostW": pp(Gm["large_top1000"]["post_w"]["mean"]), "smallPreW": pp(Gm["small_rest"]["pre_w"]["mean"]), "smallPostW": pp(Gm["small_rest"]["post_w"]["mean"]), "largePostLOOW": ", ".join(f"{v * 100:+.1f}" for v in Gm["large_top1000"]["post_loo_w"].values()), "largePostExTwentyFourW": f1(Gm["large_top1000"]["pooled_post_excl_2024_w"]["coef_pct"]), "largePostExTwentyFourWT": f"{Gm['large_top1000']['pooled_post_excl_2024_w']['t_firm']:.2f}", "largePostCI": ci(Gm["large_top1000"]["post_w"]["ci95"]), "smallPostCI": ci(Gm["small_rest"]["post_w"]["ci95"]), "largePreCI": ci(Gm["large_top1000"]["pre_w"]["ci95"]), "smallPreCI": ci(Gm["small_rest"]["pre_w"]["ci95"]),
294 + "spPre": pp(Gm["sp500"]["pre"]["mean"]), "spPost": pp(Gm["sp500"]["post"]["mean"]), "nonspPre": pp(Gm["non_sp500"]["pre"]["mean"]), "nonspPost": pp(Gm["non_sp500"]["post"]["mean"]), "nonspPostT": tv(Gm["non_sp500"]["post"]["t"]),
295 + "contrastLarge": f1(CT["large_vs_small_w"]["hi_x_post_x_group"]["coef_pct"]), "contrastLargeT": f"{CT['large_vs_small_w']['hi_x_post_x_group']['t_firm']:.2f}", "contrastLargeTtw": f"{CT['large_vs_small_w']['hi_x_post_x_group']['t_twoway']:.2f}", "contrastLargePwild": pv(CT["large_vs_small_w"]["hi_x_post_x_group"]["p_wild_webb"]), "contrastLargeMDE": f"{CT['large_vs_small_w']['hi_x_post_x_group']['mde_pct_firm']:.1f}", "contrastLargeCI": f"[{CT['large_vs_small_w']['hi_x_post_x_group']['ci95_firm_pct'][0]:+.1f}, {CT['large_vs_small_w']['hi_x_post_x_group']['ci95_firm_pct'][1]:+.1f}]", "contrastMember": f1(CT["member_vs_non_w"]["hi_x_post_x_group"]["coef_pct"]), "contrastMemberT": f"{CT['member_vs_non_w']['hi_x_post_x_group']['t_firm']:.2f}", "contrastMemberTtw": f"{CT['member_vs_non_w']['hi_x_post_x_group']['t_twoway']:.2f}", "contrastMemberPwild": pv(CT["member_vs_non_w"]["hi_x_post_x_group"]["p_wild_webb"]), "contrastMemberMDE": f"{CT['member_vs_non_w']['hi_x_post_x_group']['mde_pct_firm']:.1f}",
296 + "sectNeuPost": pp(SN["post"]["mean"]), "sectNeuPostT": tv(SN["post"]["t"]), "sectNeuFull": pp(SN["full"]["mean"]), "sectNeuFullT": tv(SN["full"]["t"]), "sectNeuFullW": pp(SN["full_w"]["mean"]), "sectNeuFullWT": tv(SN["full_w"]["t"]), "residSortFullW": pp(RSd["full_w"]["mean"]), "residSortFullWT": tv(RSd["full_w"]["t"]), "techBetaPost": f"{M['post'].get('tech_beta', float('nan')):.2f}", "ffsixTechPost": f1(M["post"].get("ff6_tech_alpha_pct", float("nan"))), "ffsixTechPostT": f"{M['post'].get('ff6_tech_t', float('nan')):.2f}",
297 + "fmPostSector": f"{Lc_['fm_all_raw_sector_post']['slope_pct']:+.2f}", "fmPostSectorT": tv(Lc_["fm_all_raw_sector_post"]["t_nw"]), "fmPost": f"{Lc_['fm_all_raw_nosector_post']['slope_pct']:+.2f}", "fmPostT": tv(Lc_["fm_all_raw_nosector_post"]["t_nw"]), "syPost": f"{Lc_['pooled_all_post']['sector_year_fe']['coef_pct']:+.2f}", "syPostT": f"{Lc_['pooled_all_post']['sector_year_fe']['t_firm']:.2f}", "syPostTtw": f"{Lc_['pooled_all_post']['sector_year_fe']['t_twoway']:.2f}",
298 + "fmFullSector": f"{Lc_['fm_all_raw_sector_full']['slope_pct']:+.2f}", "fmFullSectorT": tv(Lc_["fm_all_raw_sector_full"]["t_nw"]), "fmFullSectorTplain": tv(Lc_["fm_all_raw_sector_full"]["t_plain"]), "fmFullSectorW": f"{Lc_['fm_all_w_sector_full']['slope_pct']:+.2f}", "fmFullSectorWT": tv(Lc_["fm_all_w_sector_full"]["t_nw"]), "fmFullSectorWTplain": tv(Lc_["fm_all_w_sector_full"]["t_plain"]), "fmFullSectorExEnergyW": f"{Lc_['fm_ex_energy_w_sector_full']['slope_pct']:+.2f}", "fmFullSectorExEnergyWT": tv(Lc_["fm_ex_energy_w_sector_full"]["t_nw"]), "fixedBcv": f"{FB:.2f}", "fmPre": f"{Lc_['fm_all_raw_nosector_pre']['slope_pct']:+.2f}", "fmPreT": tv(Lc_["fm_all_raw_nosector_pre"]["t_nw"]), "fmFull": f"{Lc_['fm_all_raw_nosector_full']['slope_pct']:+.2f}", "fmFullT": tv(Lc_["fm_all_raw_nosector_full"]["t_nw"]), "syFull": f"{Lc_['pooled_all_full']['sector_year_fe']['coef_pct']:+.2f}", "syFullT": f"{Lc_['pooled_all_full']['sector_year_fe']['t_firm']:.2f}", "syFullTtw": f"{Lc_['pooled_all_full']['sector_year_fe']['t_twoway']:.2f}", "syFullPwild": pv(Lc_["pooled_all_full"]["sector_year_fe"].get("p_wild_webb")),
299 + "placeboRank": str(PB["rank_of_2022"]), "placeboN": str(PB["n_candidates"]), "placeboRankW": str(PB["winsorized"]["rank_of_2022"]), "permP": f"{PB['permutation']['p_one_sided']:.2f}", "permPW": f"{PB['winsorized']['permutation']['p_one_sided']:.2f}", "permRank": str(PB["permutation"]["rank_among_subsets"]), "supWald": f"{PB['sup_wald']['sup_wald']:.2f}", "supWaldP": f"{PB['sup_wald']['p_simulated']:.2f}", "supWaldW": f"{PB['winsorized']['sup_wald']['sup_wald']:.2f}", "supWaldWP": f"{PB['winsorized']['sup_wald']['p_simulated']:.2f}", "placeboBest": max(PB["diff_post_minus_pre_pct_by_break_year"], key=PB["diff_post_minus_pre_pct_by_break_year"].get), "placeboDiffTwentyTwo": f1(PB["diff_post_minus_pre_pct_by_break_year"]["2022"]),
300 + "didCont": f"{Dd['continuous']['coef'] * 100:+.2f}", "didContT": f"{Dd['continuous']['t']:.2f}", "didBin": f"{Dd['binary']['coef'] * 100:+.2f}", "didBinT": f"{Dd['binary']['t']:.2f}",
301 + "evtTwentyTwo": f1(ET[2022]["abs_slope"] * 100), "evtTwentyFour": f1(ET[2024]["abs_slope"] * 100), "evtEighteen": f1(ET[2018]["abs_slope"] * 100), "evtDiffTwentyTwo": f1(ET[2022]["diff_from_ref"] * 100), "evtDiffTwentyThree": f1(ET[2023]["diff_from_ref"] * 100), "evtDiffTwentyFour": f1(ET[2024]["diff_from_ref"] * 100), "evtDiffTwentyTwoSE": f"{ET[2022]['diff_se'] * 100:.1f}", "evtBase": f1(R2["did_eventtime"]["base_slope_ref_year"] * 100), "evtTrailTwentyTwo": f1(ETt[2022]["diff_from_ref"] * 100), "evtTrailTwentyFour": f1(ETt[2024]["diff_from_ref"] * 100),
302 + "tenkPost": pp(Nf["10k"]["post"]["mean"]), "tenkPostT": tv(Nf["10k"]["post"]["t"]), "tenkPre": pp(Nf["10k"]["pre"]["mean"]), "tenqPost": pp(Nf["10q"]["post"]["mean"]), "eightkPost": pp(Nf["8k"]["post"]["mean"]), "corrTenK": f"{Nf['10k']['corr_with_baseline']:.2f}", "tenqPre": pp(Nf["10q"]["pre"]["mean"]), "eightkPre": pp(Nf["8k"]["pre"]["mean"]), "corrEightK": f"{Nf['8k']['corr_with_baseline']:.2f}", "hrTenK": f"{hr0['z_AI_10k']['slope_pct']:+.2f}", "hrTenKT": f"{hr0['z_AI_10k']['t']:.2f}", "hrEightK": f"{hr0['z_AI_8k']['slope_pct']:+.2f}", "hrEightKT": f"{hr0['z_AI_8k']['t']:.2f}", "hrTenQ": f"{hr0['z_AI_10q']['slope_pct']:+.2f}", "hrTenQT": f"{hr0['z_AI_10q']['t']:.2f}", "formWaldP": f"{wd['p']:.2f}", "formWaldPraw": f"{wd0['p']:.2f}", "formwFull": pp(Nf["formw"]["full_w"]["mean"]), "formwFullT": tv(Nf["formw"]["full_w"]["t"]), "cohortTwentyTwoPostShare": f"{c22['share_chunks_post_launch'] * 100:.1f}", "cohortTwentyTwoPreOnly": f1(c22["spread_prelaunch_text_only_w_pct"]), "cohortTwentyTwoFullSame": f1(c22["spread_full_text_same_firms_w_pct"]),
303 + "spidPre": pp(SP["SPID_TECH"]["pre"]["mean"]), "spidPost": pp(SP["SPID_TECH"]["post"]["mean"]), "spidPostT": tv(SP["SPID_TECH"]["post"]["t"]), "spidAnyShare": f"{R['spid']['share_firm_years_any_ai'] * 100:.1f}", "spidAnyPostShare": f"{R['spid']['share_post_any_ai'] * 100:.1f}", "spidAnyPre": f"{SB['spid_ai_any']['pre']['cohort_w']['mean_pct']:+.1f}" if SB["spid_ai_any"]["pre"]["cohort_w"]["mean_pct"] is not None else "--", "spidAnyPost": f"{SB['spid_ai_any']['post']['cohort_w']['mean_pct']:+.1f}", "spidAnyPostT": tv(SB["spid_ai_any"]["post"]["cohort_w"]["t"]), "spidAnyPostCI": str(SB["spid_ai_any"]["post"]["cohort_w"]["ci95_pct"]), "spidAnyFirmFull": f"{SB['spid_ai_any']['full']['firm_w']['coef_pct']:+.1f}", "spidAnyFirmFullT": f"{SB['spid_ai_any']['full']['firm_w']['t_firm']:.2f}", "spidAnyFirmFullMDE": f"{SB['spid_ai_any']['full']['firm_w']['mde_pct']:.1f}", "spidAnyFirmFullCI": f"[{SB['spid_ai_any']['full']['firm_w']['ci95_pct'][0]:+.1f}, {SB['spid_ai_any']['full']['firm_w']['ci95_pct'][1]:+.1f}]", "spidCohortMDE": f"{SB['spid_ai_any']['full']['cohort_w']['mde_pct']:.1f}" if SB["spid_ai_any"]["full"]["cohort_w"]["mde_pct"] else "--", "spidLegacyMDE": f"{R['spid']['legacy_quintile_full_mde_pct']:.1f}", "spidXpost": f"{Pp_['z_SPID_TECH']['terms']['z_SPID_TECH:post']['coef_pct']:+.2f}", "spidXpostT": f"{Pp_['z_SPID_TECH']['terms']['z_SPID_TECH:post']['t_firm']:.2f}", "spidXpostTtw": f"{Pp_['z_SPID_TECH']['terms']['z_SPID_TECH:post']['t_twoway']:.2f}", "spidTechFirmFull": f"{SB['spid_tech_any']['full']['firm_w']['coef_pct']:+.1f}", "spidTechFirmFullT": f"{SB['spid_tech_any']['full']['firm_w']['t_firm']:.2f}",
304 + "esgPost": pp(PLm["ESG commitment"]["post"]["mean"]), "climPost": pp(PLm["Climate risk"]["post"]["mean"]), "cyberPost": pp(PLm["Cybersecurity"]["post"]["mean"]), "autoPost": pp(PLm["Automation/labor"]["post"]["mean"]), "cyberPostW": pp(PLm["Cybersecurity"]["post_w"]["mean"]), "cyberIntT": f"{PLm['Cybersecurity']['interaction']['t_firm']:.2f}", "cyberIntTtw": f"{PLm['Cybersecurity']['interaction']['t_twoway']:.2f}", "cyberIntRW": f"{rw['Cybersecurity']:.2f}", "riskIntTtw": f"{PLm['AI disruption risk']['interaction']['t_twoway']:.2f}", "esgIntTtw": f"{PLm['ESG commitment']['interaction']['t_twoway']:.2f}",
305 + "riskPost": pp(PLm["AI disruption risk"]["post"]["mean"]), "riskPostT": tv(PLm["AI disruption risk"]["post"]["t"]), "riskPre": pp(PLm["AI disruption risk"]["pre"]["mean"]), "adoptPost": pp(PLm["AI adoption"]["post"]["mean"]), "infraPost": pp(PLm["AI infrastructure"]["post"]["mean"]), "macroPost": pp(PLm["Macro risk"]["post"]["mean"]), "riskFFsixPost": f1(R["ls_alpha"]["AI_RISK"]["post"]["ff6_alpha_ann_pct"]), "riskFFsixPostT": tv(R["ls_alpha"]["AI_RISK"]["post"]["ff6_alpha_t"]), "pcaPostW": pp(SP["AI_PC1"]["post_w"]["mean"]), "pcaFullW": pp(SP["AI_PC1"]["full_w"]["mean"]) if "full_w" in SP["AI_PC1"] else pp(S["AI_PC1_EW_w"]["Q5_minus_Q1"]), "pcaFullWT": tv(S["AI_PC1_EW_w"]["t_stat"]), "pcaEVR": f"{R['pca']['evr'][0] * 100:.1f}", "pcaPost": pp(SP["AI_PC1"]["post"]["mean"]),
306 + "nEntrantFirms": n(EN["n_firms"] - EN["n_seasoned"]), "nSeasoned": n(EN["n_seasoned"]), "rankAuto": f"{cv['rank_autocorr_lag1']:.2f}", "rankAutoFive": f"{cv['rank_autocorr_lag5']:.2f}", "entrantPrem": f1(EN["entrant_minus_seasoned_fwd_pct"]), "seasonedPost": pp(EN["seasoned"]["post"]["mean"]), "seasonedPre": pp(EN["seasoned"]["pre"]["mean"]), "seasonedFullW": pp(EN["seasoned"]["full_w"]["mean"]), "seasonedFullWT": tv(EN["seasoned"]["full_w"]["t"]), "listedPre": pp(EN["listed36"]["pre"]["mean"]), "listedPost": pp(EN["listed36"]["post"]["mean"]), "magSevenPost": pp(M7["ex_mag7_post"]["mean"]), "exTopFiveHundredPost": pp(M7["ex_top500_post"]["mean"]), "exTopFiveHundredPre": pp(M7["ex_top500_pre"]["mean"]), "sixmPost": pp(AH["6m"]["post"]["mean"]), "iwvPost": pp(AB["iwv_adj"]["post"]["mean"]), "iwvPre": pp(AB["iwv_adj"]["pre"]["mean"]), "fvolPost": f1(R2["fwd_vol"]["post"]["mean"] * 100), "fvolPostT": tv(R2["fwd_vol"]["post"]["t"]), "fvolPre": f1(R2["fwd_vol"]["pre"]["mean"] * 100),
307 + "breakInfraT": tv(B["AI_INFRA"]["interaction_t_firm"]), "breakComposT": tv(B["AI_COMPOS"]["interaction_t_firm"]), "breakComposTtw": tv(B["AI_COMPOS"]["interaction_t_twoway"]), "breakComposPwild": pv(B["AI_COMPOS"].get("interaction_p_wild_webb")), "breakCompos": f"{B['AI_COMPOS']['interaction'] * 100:+.2f}", "vwQone": pp(S["AI_COMPOS_VW"]["Q1"]), "vwQfive": pp(S["AI_COMPOS_VW"]["Q5"]), "ewQone": pp(S["AI_COMPOS_EW"]["Q1"]), "ewQfive": pp(S["AI_COMPOS_EW"]["Q5"]), "vwMaxWeight": f"{R['vw_diag']['max_single_firm_weight_v2'] * 100:.0f}", "vwMaxWeightVone": f"{R['vw_diag']['max_single_firm_weight_v1'] * 100:.1f}",
308 + "feCompos": f"{PF['M7_all_spid']['coefs']['z_AI_COMPOS']['coef'] * 100:+.2f}", "feComposT": f"{PF['M7_all_spid']['coefs']['z_AI_COMPOS']['t']:.2f}", "feAdopt": f"{PF['M2_adopt_ctrl']['coefs']['z_AI_ADOPT']['coef'] * 100:+.2f}", "feAdoptT": f"{PF['M2_adopt_ctrl']['coefs']['z_AI_ADOPT']['t']:.2f}", "feInfra": f"{PF['M3_infra_ctrl']['coefs']['z_AI_INFRA']['coef'] * 100:+.2f}", "feInfraT": f"{PF['M3_infra_ctrl']['coefs']['z_AI_INFRA']['t']:.2f}", "feSize": f"{PF['M2_adopt_ctrl']['coefs']['z_size']['coef'] * 100:+.2f}", "feSizeT": f"{PF['M2_adopt_ctrl']['coefs']['z_size']['t']:.2f}",
309 + "nLargeFirms": n(Gm["large_top1000"]["firms"]), "nSmallFirms": n(Gm["small_rest"]["firms"]), "nSmall": n(Gm["small_rest"]["n"]), "largeFFsixPost": f1(Gm["large_top1000"]["ls"]["post"]["ff6_alpha_ann_pct"]), "largeFFsixPostT": tv(Gm["large_top1000"]["ls"]["post"]["ff6_alpha_t"]), "largeSharpePost": f"{Gm['large_top1000']['ls']['post']['sharpe']:.2f}", "smallSharpePost": f"{Gm['small_rest']['ls']['post']['sharpe']:.2f}", "smallSharpePre": f"{Gm['small_rest']['ls']['pre']['sharpe']:.2f}", "largeSharpePre": f"{Gm['large_top1000']['ls']['pre']['sharpe']:.2f}", "smallFull": pp(Gm["small_rest"]["sort_EW"]["Q5_minus_Q1"]), "smallFullT": tv(Gm["small_rest"]["sort_EW"]["t_stat"]), "largeFull": pp(Gm["large_top1000"]["sort_EW"]["Q5_minus_Q1"]), "largeFullT": tv(Gm["large_top1000"]["sort_EW"]["t_stat"]), "smallFullW": pp(Gm["small_rest"]["sort_EW_w"]["Q5_minus_Q1"]), "smallFullWT": tv(Gm["small_rest"]["sort_EW_w"]["t_stat"]), "largeFullW": pp(Gm["large_top1000"]["sort_EW_w"]["Q5_minus_Q1"]), "largeFullWT": tv(Gm["large_top1000"]["sort_EW_w"]["t_stat"]), "sizeGroupChange": f"{MD['share_firm_years_changing_size_group'] * 100:.1f}", "nNoCap": n(META["n_est_without_cap"]), "nNoCapVone": n(META["n_est_without_cap_v1"]),
310 + "sizeNeuFull": pp(sn["full"]["Q5_minus_Q1"]), "sizeNeuFullT": tv(sn["full"]["t_stat"]), "sizeNeuPost": pp(sn["post"]["mean"]), "sizeNeuPre": pp(sn["pre"]["mean"]), "sizeNeuFullW": pp(sn["full_w"]["Q5_minus_Q1"]), "sizeNeuFullWT": tv(sn["full_w"]["t_stat"]), "largeCohorts": ", ".join(f"{y}: {v * 100:+.1f}" for y, v in Gm["large_top1000"]["post"]["by_cohort"].items()), "smallCohorts": ", ".join(f"{y}: {v * 100:+.1f}" for y, v in Gm["small_rest"]["post"]["by_cohort"].items()), "largeCohortsW": ", ".join(f"{y}: {v * 100:+.1f}" for y, v in Gm["large_top1000"]["post_w"]["by_cohort"].items()), "smallCohortsW": ", ".join(f"{y}: {v * 100:+.1f}" for y, v in Gm["small_rest"]["post_w"]["by_cohort"].items()),
311 + "fmPostSectorFull": f"{Lc_['fm_all_raw_sector_full']['slope_pct']:+.2f}", "preCohortsAbove": str(Cc["n_pre_above_2022"]), "preCohortsPositive": f"{Cc['pre_cohorts_positive_share'] * 100:.0f}", "preCohortsPositiveW": f"{Cw['pre_cohorts_positive_share'] * 100:.0f}", "nonSPfirms": n(Gm["non_sp500"]["firms"]),
312 + "spN": n(SMs["n"]), "spFirms": n(SMs["firms"]), "spSpreadFull": pp(SMs["sorts"]["AI_COMPOS_EW"]["Q5_minus_Q1"]), "spSpreadFullT": tv(SMs["sorts"]["AI_COMPOS_EW"]["t_stat"]), "spSpreadFullW": pp(SMs["sorts"]["AI_COMPOS_EW_w"]["Q5_minus_Q1"]), "spSpreadFullWT": tv(SMs["sorts"]["AI_COMPOS_EW_w"]["t_stat"]), "spSpreadPre": pp(SMs["sub"]["AI_COMPOS"]["pre"]["mean"]), "spSpreadPost": pp(SMs["sub"]["AI_COMPOS"]["post"]["mean"]), "spSpreadPostT": tv(SMs["sub"]["AI_COMPOS"]["post"]["t"]), "spSpreadPreW": pp(SMs["sub"]["AI_COMPOS"]["pre_w"]["mean"]), "spSpreadPostW": pp(SMs["sub"]["AI_COMPOS"]["post_w"]["mean"]), "spCohorts": ", ".join(f"{y}: {v:+.1f}" for y, v in SMs["cohorts_pct"].items() if int(y) >= 2022), "spCohortsW": ", ".join(f"{y}: {v:+.1f}" for y, v in SMs["cohorts_w_pct"].items() if int(y) >= 2022), "spSharpePost": f"{SMs['ls']['AI_COMPOS']['post']['sharpe']:.2f}", "spSharpePre": f"{SMs['ls']['AI_COMPOS']['pre']['sharpe']:.2f}", "spFFsixPost": f1(SMs["ls"]["AI_COMPOS"]["post"]["ff6_alpha_ann_pct"]), "spFFsixPostT": tv(SMs["ls"]["AI_COMPOS"]["post"]["ff6_alpha_t"]), "spPooledPost": f1(SMs["pooled_post"]["coef_pct"]), "spPooledPostT": f"{SMs['pooled_post']['t_firm']:.2f}", "spPooledPostW": f1(SMs["pooled_post_w"]["coef_pct"]), "spPooledPostWT": f"{SMs['pooled_post_w']['t_firm']:.2f}", "spPooledPostWTtw": tv(SMs["pooled_post_w"].get("t_twoway")), "spFMcompos": f"{SMs['fm']['compos']['zz_AI_COMPOS']['mean_coef'] * 100:+.2f}", "spFMcomposT": tv(SMs["fm"]["compos"]["zz_AI_COMPOS"]["t"]), "rkFMcompos": f"{SMr['fm']['compos']['zz_AI_COMPOS']['mean_coef'] * 100:+.2f}", "rkFMcomposT": tv(SMr["fm"]["compos"]["zz_AI_COMPOS"]["t"]), "spFMcomposW": f"{SMs['fm']['compos_w']['zz_AI_COMPOS']['mean_coef'] * 100:+.2f}", "spFMcomposWT": tv(SMs["fm"]["compos_w"]["zz_AI_COMPOS"]["t"]),
313 + "fmCompos": f"{FM['compos']['z_AI_COMPOS']['mean_coef'] * 100:+.2f}", "fmComposT": tv(FM["compos"]["z_AI_COMPOS"]["t"]), "fmComposTplain": tv(FM["compos"]["z_AI_COMPOS"]["t_plain"]), "fmComposW": f"{FM['compos_w']['z_AI_COMPOS']['mean_coef'] * 100:+.2f}", "fmComposWT": tv(FM["compos_w"]["z_AI_COMPOS"]["t"]), "fmComposWTplain": tv(FM["compos_w"]["z_AI_COMPOS"]["t_plain"]), "fmComposWCI": ci(FM["compos_w"]["z_AI_COMPOS"]["ci95"], 2), "fmComposCI": ci(FM["compos"]["z_AI_COMPOS"]["ci95"], 2),
314 + "nFilingsCorpus": n(len(fl)), "nIssuersCorpus": n(fl.cik.nunique()), "corrAdoptInfra": f"{c.loc['z_AI_ADOPT', 'z_AI_INFRA']:.2f}", "corrAdoptRisk": f"{c.loc['z_AI_ADOPT', 'z_AI_RISK']:.2f}", "corrSpid": f"{c.loc['z_AI_ADOPT', 'z_SPID_TECH']:.2f}", "corrCyber": f"{CY['corr_cyber_composite']:.2f}", "cyberNeuFullW": pp(CY["cyberneutral_sort"]["full_w"]["mean"]), "cyberNeuFullWT": tv(CY["cyberneutral_sort"]["full_w"]["t"]),
315 + "dsSmallPost": f1(pd.DataFrame(R2["double_sort_post"]).loc["Small", "Hi-Lo"]), "dsLargePost": f1(pd.DataFrame(R2["double_sort_post"]).loc["Large", "Hi-Lo"]), "dsSmallPre": f1(pd.DataFrame(R2["double_sort_pre"]).loc["Small", "Hi-Lo"]), "dsLargePre": f1(pd.DataFrame(R2["double_sort_pre"]).loc["Large", "Hi-Lo"]), "dsSmallPostW": f1(pd.DataFrame(R2["double_sort_post_w"]).loc["Small", "Hi-Lo"]), "dsLargePostW": f1(pd.DataFrame(R2["double_sort_post_w"]).loc["Large", "Hi-Lo"]),
316 + "evLarge": f1(base["large"]["horizons"]["car_1y"]["HL_CAR_pct"]), "evLargeT": f"{WR['large']['t_welch']:.2f}", "evLargeCDA": f"{base['large']['horizons']['car_1y']['t_cda']:.2f}", "evSmall": f1(base["small"]["horizons"]["car_1y"]["HL_CAR_pct"]), "evSmallT": f"{WR['small']['t_welch']:.2f}", "evSmallCDA": f"{base['small']['horizons']['car_1y']['t_cda']:.2f}", "evSP": f1(base["sp500"]["horizons"]["car_1y"]["HL_CAR_pct"]), "evSPT": f"{WR['sp500']['t_welch']:.2f}", "evSPCDA": f"{base['sp500']['horizons']['car_1y']['t_cda']:.2f}", "evNonSP": f1(base["non_sp500"]["horizons"]["car_1y"]["HL_CAR_pct"]), "evNonSPT": f"{WR['non_sp500']['t_welch']:.2f}", "evNonSPCDA": f"{base['non_sp500']['horizons']['car_1y']['t_cda']:.2f}",
317 + "evLargeBetaCDA": f"{EB['beta_only']['large']['horizons']['car_1y']['t_cda']:.2f}", "evLargeMadjCDA": f"{EB['market_adjusted']['large']['horizons']['car_1y']['t_cda']:.2f}", "evLargeEstTwentyOneCDA": f"{EB['mm_est2021']['large']['horizons']['car_1y']['t_cda']:.2f}", "evLargeSizeMomCDA": f"{EB['size_mom_matched']['large']['horizons']['car_1y']['t_cda']:.2f}" if EB["size_mom_matched"].get("large") else "--", "evLargeFFsixT": f"{FF['large']['car_1y']['ff6_t']:.2f}" if "large" in FF and "car_1y" in FF["large"] else "--", "evLargeFFsixAlpha": f1(FF["large"]["car_1y"]["ff6_alpha_cum_pct"]) if "large" in FF and "car_1y" in FF["large"] else "--", "evFullFFsixT": f"{FF['full']['car_1y']['ff6_t']:.2f}" if "car_1y" in FF.get("full", {}) else "--", "evLargeWithinCDA": f"{base['large_within']['horizons']['car_1y']['t_cda']:.2f}" if base.get("large_within") else "--", "evLargeEighteenCDA": f"{base['large']['horizons']['car_18m']['t_cda']:.2f}", "evSmallEighteenCDA": f"{base['small']['horizons']['car_18m']['t_cda']:.2f}", "evPlaceboRank": str(PS["rank_2022_by_HL"]), "evPlaceboRankT": str(PS["rank_2022_by_t"]), "evPlaceboN": str(PS["n_launches"]),
318 + "evXsec": f1(RE["xsec_1y"]["ai_pre_pct"]), "evXsecT": f"{RE['xsec_1y']['t']:.2f}", "evXsecSect": f1(RE["xsec_1y_sector"]["ai_pre_pct"]), "evXsecSectT": f"{RE['xsec_1y_sector']['t']:.2f}",
319 + "attritionSmallMax": f"{max(v['small'] for v in AP.values()) * 100:.0f}", "attritionLargeMax": f"{max(v['large'] for v in AP.values()) * 100:.0f}", "ewMinusIWV": f1(EI["mean_gap_all_minus_iwv_pp"]), "ewMinusIWVW": f1(EI["mean_gap_all_w_minus_iwv_pp"]), "ewMinusIWVmed": f1(EI["mean_gap_median_minus_iwv_pp"]), "smallMinusIWM": f1(EI["mean_gap_small_minus_iwm_pp"]), "largeMinusIWV": f1(EI["mean_gap_large_minus_iwv_pp"]), "sizeDemeanedFullW": pp(CS["size_demeaned_w"]["full"]["mean"]), "sizeDemeanedFullWT": tv(CS["size_demeaned_w"]["full"]["t"]), "lowVolFullW": pp(CS["low_fwd_vol_w"]["full"]["mean"]), "lowVolFullWT": tv(CS["low_fwd_vol_w"]["full"]["t"]), "belowPninetynineFull": pp(CS["below_p99_raw"]["full"]["mean"]), "belowPninetynineFullT": tv(CS["below_p99_raw"]["full"]["t"]), "fvolGradFull": pp(CS["fwd_vol_gradient"]["full"]["mean"]), "nIWVsnapshots": str(len(RS["snapshots"])),
320 + "specN": str(R2["spec_curve"]["n_specs"]), "specPos": f"{R2['spec_curve']['share_positive'] * 100:.0f}", "specPosSig": f"{R2['spec_curve']['share_positive_sig_nw'] * 100:.0f}", "specPosSigPlain": f"{R2['spec_curve']['share_positive_sig_plain'] * 100:.0f}", "specMedian": f1(R2["spec_curve"]["median_spread_pct"]), "rwComposFull": f"{dict(zip(MR['sorts_full_w']['members'], MR['sorts_full_w']['p_romano_wolf']))['AI_COMPOS_EW']:.2f}", "hlzCount": str(sum(MR["hlz_summary"].values())),
321 + "hygieneMonthlyFlagged": str(HY["n_flagged_monthly"]), "hygieneMonthlyBreaks": str(HY["verdicts_monthly"].get("series_break", 0)), "hygieneFYFlagged": str(HY["n_flagged_firm_years"]), "hygieneFYBreaks": str(HY["verdicts_firm_year"].get("series_break", 0)), "hygieneFYRemoved": str(META["n_firm_years_fwd_removed_hygiene"]), "nDedupDropped": str(META["n_rows_before_dedup"] - META["n_rows_after_dedup"]), "nStrictTwelve": n(R["n_strict12"]), "nLtTwelve": str(META["n_est_lt12_valid"]), "aaplCapTwentyEleven": f"{MD['AAPL_cohort2010_cap_v2_usd'] / 1e9:.0f}", "aaplRankTwentyEleven": f"{MD['AAPL_cohort2010_rank_v2']:.0f}", "logCapMin": f"{MD['log_cap_min_max_v2'][0]:.1f}", "logCapMax": f"{MD['log_cap_min_max_v2'][1]:.1f}"})
322 +open(f"{P}/macros.tex", "w").write("".join(f"\\newcommand{{\\m{k}}}{{{v}}}\n" for k, v in MAC.items())); json.dump(MAC, open(f"{T}/inline_numbers.json", "w"), indent=1); print("macros", len(MAC))
added scripts/run_all.sh +25 −0
@@ -0,0 +1,25 @@
1 +#!/bin/bash
2 +# run_all.sh — FIRC-0002 v2.1 (Stage D review) single entry point. Rebuilds everything from the staged author inputs
3 +# (data/raw_russell3000/, data/processed_r3k/ — the 2026-10-28 Yahoo/SEC vintage, sha256 in reproducibility/FIRC-0002/environment/inputs.sha256)
4 +# into data/processed_r3k_v2/, results/r3k_v2/, figures/r3k_v2/ and paper/tables/, then compiles the manuscript.
5 +# Network steps (10 = yfinance second price pass + Ken French daily factors, 12 = SEC companyfacts, 13/13b = FinData OS read-only SQL,
6 +# 21 = doi.org/Crossref) are skipped when their outputs exist (set FORCE_DOWNLOAD=1 to redo them; prices are a new vintage then).
7 +# Usage: bash scripts/run_all.sh [--no-latex] Environment: uv venv .venv-review --python 3.12; uv pip install -r reproducibility/FIRC-0002/environment/requirements.lock
8 +set -euo pipefail; cd "$(dirname "$0")/.."; PY=${PY:-.venv-review/bin/python}; export OMP_NUM_THREADS=${OMP_NUM_THREADS:-4} OPENBLAS_NUM_THREADS=4 MKL_NUM_THREADS=4 PYTHONHASHSEED=0
9 +LOG=results/r3k_v2/run_all.log; mkdir -p results/r3k_v2 data/processed_r3k_v2 figures/r3k_v2; : > "$LOG"
10 +step() { local name=$1; shift; local t0=$(date +%s); echo "== $name start $(date -Iseconds)" | tee -a "$LOG"; nice -n 10 "$PY" -I "$@" >> "$LOG" 2>&1; echo "== $name done in $(( $(date +%s) - t0 )) s" | tee -a "$LOG"; }
11 +V=data/processed_r3k_v2; F=${FORCE_DOWNLOAD:-0}
12 +[ "$F" = 1 ] || [ ! -f $V/ff_daily.parquet ] && step r3k_10_download_v2 scripts/r3k_10_download_v2.py || echo "skip r3k_10 (outputs present)"
13 +[ "$F" = 1 ] || [ ! -f $V/sec_shares_raw.parquet ] && step r3k_12_sec_shares_v2 scripts/r3k_12_sec_shares_v2.py || step r3k_12_reaggregate scripts/r3k_12_sec_shares_v2.py --reaggregate
14 +[ "$F" = 1 ] || [ ! -f results/r3k_v2/hygiene_log.csv ] && step r3k_13_hygiene_findata scripts/r3k_13_hygiene_findata.py || echo "skip r3k_13 (hygiene_log.csv present)"
15 +[ "$F" = 1 ] || [ ! -f $V/prices_monthly_findata_raw.parquet ] && step r3k_13b_findata_raw_closes scripts/r3k_13b_findata_raw_closes.py || echo "skip r3k_13b (present)"
16 +step r3k_14_build_panel_v2 scripts/r3k_14_build_panel_v2.py
17 +step r3k_15_analysis_v2 scripts/r3k_15_analysis_v2.py
18 +step r3k_16_extended_v2 scripts/r3k_16_extended_v2.py
19 +step r3k_17_event_study_v2 scripts/r3k_17_event_study_v2.py
20 +step r3k_18_survivorship scripts/r3k_18_survivorship.py
21 +step r3k_19_figures_v2 scripts/r3k_19_figures_v2.py
22 +step r3k_20_tables_v2 scripts/r3k_20_tables_v2.py
23 +[ -f results/r3k_v2/t17_refit.json ] || step r3k_11_t17_refit scripts/r3k_11_t17_refit.py
24 +if [ "${1:-}" != "--no-latex" ]; then t0=$(date +%s); (cd paper && latexmk -pdf -interaction=nonstopmode -halt-on-error main.tex > ../results/r3k_v2/latexmk.log 2>&1) && echo "== latexmk done in $(( $(date +%s) - t0 )) s" | tee -a "$LOG" || echo "latexmk FAILED (see results/r3k_v2/latexmk.log)" | tee -a "$LOG"; fi
25 +echo "ALL DONE $(date -Iseconds)" | tee -a "$LOG"
added tests/test_regenerate.py +57 −0
@@ -0,0 +1,57 @@
1 +"""tests/test_regenerate.py — FIRC-0002 v2.1 reproducibility test (Stage D, package L9).
2 +Re-runs the deterministic analysis chain (r3k_14 -> 15 -> 16 -> 17 -> 18) from the staged inputs into a scratch directory and
3 +compares every leaf of results.json, results2.json, results_event.json, results_surv.json with the committed results/r3k_v2/
4 +versions at relative tolerance 1e-6. All bootstraps, permutations and simulated critical values use fixed seeds
5 +(r3k_inference.RNG_SEED = 20261105), so no block is allow-listed; runtime differs only. Also checks: panel keyed uniquely on
6 +(cik, fiscal_year); no forward return beyond the hygiene bounds without a 'confirmed' verdict; formation date strictly before the
7 +first holding month (chronology); the five staged input directories match reproducibility/FIRC-0002/environment/inputs.sha256.
8 +Run from the worktree root: .venv-review/bin/python -I tests/test_regenerate.py [--quick] (--quick: schema/leakage checks only)
9 +"""
10 +import os, sys, json, math, shutil, subprocess, hashlib, tempfile, pandas as pd
11 +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))); os.chdir(ROOT); PY = sys.executable; TOL = 1e-6
12 +def walk(a, b, path, out):
13 + if isinstance(a, dict) and isinstance(b, dict):
14 + for k in sorted(set(a) | set(b)):
15 + if k not in a or k not in b: out.append((path + "/" + k, "MISSING")); continue
16 + walk(a[k], b[k], path + "/" + k, out)
17 + elif isinstance(a, list) and isinstance(b, list):
18 + if len(a) != len(b): out.append((path, "LEN")); return
19 + for i, (x, y) in enumerate(zip(a, b)): walk(x, y, f"{path}[{i}]", out)
20 + else:
21 + try:
22 + fa, fb = float(a), float(b)
23 + if math.isnan(fa) and math.isnan(fb): return
24 + if abs(fa - fb) > TOL * max(1.0, abs(fa), abs(fb)): out.append((path, f"{a} != {b}"))
25 + except (TypeError, ValueError):
26 + if a != b and not path.endswith("runtime_s") and "seconds" not in path and "when" not in path: out.append((path, f"{a} != {b}"))
27 +fails = []
28 +# 1. schema / leakage / chronology checks on the committed-vintage derived panel
29 +m = pd.read_parquet("data/processed_r3k_v2/merged_panel_v2.parquet"); e = m[m.fwd_ret.notna() & (m.fiscal_year <= 2024)]
30 +if m.duplicated(["cik", "fiscal_year"]).any(): fails.append("panel not unique on (cik, fiscal_year)")
31 +hy = pd.read_csv("results/r3k_v2/hygiene_log.csv"); conf = set(zip(hy[(hy.kind == "firm_year_fwd") & (hy.verdict == "confirmed")].ticker, hy[(hy.kind == "firm_year_fwd") & (hy.verdict == "confirmed")].fiscal_year.astype(int)))
32 +big = e[e.fwd_ret > 5.0]; unconf = [(t, y) for t, y in zip(big.ticker, big.fiscal_year) if (t, y) not in conf]
33 +if unconf: fails.append(f"{len(unconf)} forward returns > 500% without a confirmed second-source verdict: {unconf[:5]}")
34 +px = pd.read_parquet("data/processed_r3k_v2/monthly_returns_v2.parquet"); first_hold = px.groupby("ticker").date.min()
35 +if (e.first_price > pd.to_datetime(e.fiscal_year + 1, format="%Y") + pd.offsets.MonthEnd(6)).any(): fails.append("chronology: a firm's first price is after its formation date")
36 +if (e.months_listed < 0).any(): fails.append("chronology: negative months listed")
37 +# 2. input manifest
38 +man = {l.split()[1]: l.split()[0] for l in open("../../reproducibility/FIRC-0002/environment/inputs.sha256") if l.strip()} if os.path.exists("../../reproducibility/FIRC-0002/environment/inputs.sha256") else {}
39 +for rel, h in list(man.items())[:400]:
40 + p = os.path.join("data", rel)
41 + if os.path.exists(p) and hashlib.sha256(open(p, "rb").read()).hexdigest() != h: fails.append(f"input changed: {rel}")
42 +print("schema/leakage/manifest checks:", "OK" if not fails else fails, flush=True)
43 +if "--quick" in sys.argv: sys.exit(1 if fails else 0)
44 +# 3. regenerate the analysis chain in a scratch copy and diff the JSON outputs
45 +S = tempfile.mkdtemp(prefix="firc0002_test_"); os.makedirs(f"{S}/results/r3k_v2"); os.makedirs(f"{S}/results/n1", exist_ok=True); shutil.copytree("scripts", f"{S}/scripts"); os.makedirs(f"{S}/data"); os.symlink(os.path.abspath("data/raw_russell3000"), f"{S}/data/raw_russell3000"); os.symlink(os.path.abspath("data/processed_r3k"), f"{S}/data/processed_r3k")
46 +shutil.copytree("data/processed_r3k_v2", f"{S}/data/processed_r3k_v2", ignore=shutil.ignore_patterns("analysis_panel_v2.parquet", "merged_panel_v2.parquet", "monthly_returns_v2.parquet", "ls_ts_v2_*", "event_car*"))
47 +for f in ["hygiene_log.csv", "hygiene_summary.json"]: shutil.copy(f"results/r3k_v2/{f}", f"{S}/results/r3k_v2/{f}")
48 +env = dict(os.environ, OMP_NUM_THREADS="4", PYTHONHASHSEED="0")
49 +for s in ["r3k_14_build_panel_v2", "r3k_15_analysis_v2", "r3k_16_extended_v2", "r3k_17_event_study_v2", "r3k_18_survivorship"]:
50 + r = subprocess.run([PY, "-I", f"scripts/{s}.py"], cwd=S, env=env, capture_output=True, text=True)
51 + if r.returncode != 0: fails.append(f"{s} failed: {r.stderr[-500:]}"); break
52 + print("regenerated", s, flush=True)
53 +for f in ["results.json", "results2.json", "results_event.json", "results_surv.json"]:
54 + if not os.path.exists(f"{S}/results/r3k_v2/{f}"): continue
55 + out = []; walk(json.load(open(f"results/r3k_v2/{f}")), json.load(open(f"{S}/results/r3k_v2/{f}")), "", out); out = [o for o in out if "runtime" not in o[0] and "seconds" not in o[0] and "/when" not in o[0]]
56 + print(f, "differing leaves:", len(out), out[:5]); fails += [f"{f}: {len(out)} leaves differ"] if out else []
57 +shutil.rmtree(S, ignore_errors=True); print("RESULT:", "PASS" if not fails else "FAIL"); [print(" -", x) for x in fails]; sys.exit(1 if fails else 0)
58
Menu

Theme