SPB Git

spb/anomaly-atlas Public License

Systematic discovery & rigorous validation of statistical anomalies in open HF market data (hfmarketdata.io) — pre-registered, artifact-null-driven, fully reproducible. Live atlas: www.anomaly-atlas.io

quant-researchmarket-microstructureanomaliesreproducibilityduckdbapple-silicon
Python 61.4% JavaScript 28.7% CSS 8.6% Shell 0.7% Makefile 0.5%

fix: expE lint (semicolons) + duplicated Next-experiment block

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Simon-Pierre Boucher committed 4 h ago (Aug 12, 2026) parent ac72935

Showing 9 changed files with +698 and −32

modified data_manifest/index.jsonl +1 −0
@@ -651,3 +651,4 @@
651 651 {"author": "Simon-Pierre Boucher", "cache_key": "e0663a26b52935dd0f8051c9aadb4ec0", "data_source": "hfmarketdata.io", "endpoint": "/v1/bars/futures/ES", "fetched_utc": "2026-08-12T06:46:03Z", "first": "2015-07-31 17:14:00", "last": "2015-09-22 07:44:00", "params": {"adjustment": "contin_UNadj", "end": "2016-01-01", "limit": 50000, "order": "asc", "start": "2015-07-31 17:14:00", "timeframe": "1min"}, "rows": 50000, "sha256": "db08bb2f91a67cbb534224e6636eeb392baec609d3f98ac7fdbb656c7182ac52"}
652 652 {"author": "Simon-Pierre Boucher", "cache_key": "445e288bf8143ffb040e377f273cd8cd", "data_source": "hfmarketdata.io", "endpoint": "/v1/bars/futures/ES", "fetched_utc": "2026-08-12T06:46:05Z", "first": "2015-09-22 07:44:00", "last": "2015-11-12 01:05:00", "params": {"adjustment": "contin_UNadj", "end": "2016-01-01", "limit": 50000, "order": "asc", "start": "2015-09-22 07:44:00", "timeframe": "1min"}, "rows": 50000, "sha256": "60b1af5b39fcff997d033db731e833290a1ec47bdbc613859191067c8f548016"}
653 653 {"author": "Simon-Pierre Boucher", "cache_key": "feba5806d5ceea82ac362ed3f089b499", "data_source": "hfmarketdata.io", "endpoint": "/v1/bars/futures/ES", "fetched_utc": "2026-08-12T06:46:06Z", "first": "2015-11-12 01:05:00", "last": "2015-12-31 16:59:00", "params": {"adjustment": "contin_UNadj", "end": "2016-01-01", "limit": 50000, "order": "asc", "start": "2015-11-12 01:05:00", "timeframe": "1min"}, "rows": 46407, "sha256": "e4174e785a531002bd4ea75c32b00544a709b2875062f9329631b1818b60baff"}
654 +{"author": "Simon-Pierre Boucher", "cache_key": "5d727ed992485b958eff75dcc9626c9d", "data_source": "hfmarketdata.io", "endpoint": "/v1/bars/etf/SPY", "fetched_utc": "2026-08-12T06:51:16Z", "first": "2000-01-03", "last": "2015-12-31", "params": {"adjustment": "adj_splitdiv", "end": "2016-01-01", "limit": 50000, "order": "asc", "start": "2000-01-01", "timeframe": "1day"}, "rows": 4025, "sha256": "a903002c5b520d0a643c7703edd5d9196fc90d3fa99869b2612a6b31140072e4"}
modified experiments/micro/expE_calendar_scan/README.md +1 −1
@@ -12,4 +12,4 @@ status: draft
12 12
13 13 Calendar scan: intraday, day-of-week, turn-of-month with pre-counted hypothesis budget
14 14
15 −Status: scaffolded 2026-08-12, not yet run.
15 +Status: **completed 2026-08-12** — zero of 8 pre-declared calendar tests survive the permuted-calendar null; turn-of-month decayed inside train; controls held. H20 intraday artifact profile measured.
modified experiments/micro/expE_calendar_scan/analysis.md +48 −2
@@ -5,9 +5,55 @@ author: Simon-Pierre Boucher
5 5 contact: contact@spboucher.ai
6 6 data_source: hfmarketdata.io
7 7 created: 2026-08-12
8 −status: draft
8 +modified: 2026-08-12
9 +status: reviewed
9 10 ---
10 11
11 12 # Analysis — expE_calendar_scan
12 13
13 −*To be written after results exist. Must include the seven-field block and the evidence standard of CLAUDE.md §10 (never report an in-sample number as a finding).*
14 +Run: `results/expE_calendar_scan/20260812T065222Z/results.json` (protocol and
15 +8-test budget pre-specified; single declared instrument SPY; permuted-calendar
16 +null n=2000; manifest embedded). Level 0 throughout.
17 +
18 +## 1. The calendar is empty (and that is the finding)
19 +
20 +| class | observed (bp) | perm 95% band | p family-wise |
21 +|---|---|---|---|
22 +| mon | −1.4 | [−7.9, +7.9] | 1.00 |
23 +| tue | +4.0 | [−7.9, +8.0] | 0.99 |
24 +| fri | −3.8 | [−8.0, +8.2] | 0.99 |
25 +| **turn_of_month** | **+4.7** | [−8.2, +8.3] | **0.97** |
26 +| pre_holiday | +5.7 | [−20.8, +21.0] | 0.93 |
27 +| post_holiday | +2.9 | [−21.2, +20.7] | 1.00 |
28 +
29 +Every dot inside its band; zero FDR rejections; the family-wise max-stat
30 +(mini reality check) never gets close. **Turn-of-month — the last calendar
31 +survivor as of Marquering et al. (2006) — fails the permuted-calendar bar
32 +in train (2000–2016) and decays inside it** (+7.9 bp in 2000–2007 → +1.6 bp
33 +in 2008–2015). This is exactly the Sullivan–Timmermann–White prediction,
34 +reproduced on open data with an honest budget.
35 +
36 +## 2. Controls behaved — the negative is trustworthy
37 +
38 +* **Monday (negative control)**: −1.4 bp, p 0.73. Had the pipeline "found"
39 + the canonical corpse, the pipeline would have been declared broken (the
40 + pre-registered criterion). It did not.
41 +* **H20 U-shape (positive control)**: strongly present. Volatility 6.6 bp
42 + (09:30) → 2.4 (midday) → 2.9 (15:30); EDGE spread **declines monotonically
43 + 2.8 → 1.2 bp** — the spread profile is an L, not a U (the close's
44 + liquidity is cheap; the open's is expensive). Both numbers now live in the
45 + taxonomy: any first-30-minutes return claim fights 2–3× the midday
46 + artifact level, and open-heavy strategies pay double spread.
47 +
48 +## 3. Honest caveats
49 +
50 +Holiday classes have n=144 → bands of ±21 bp: low power, reported as such
51 +(absence of evidence here is weak evidence of absence). Single instrument by
52 +design — cross-sectional calendar claims were deliberately out of budget.
53 +
54 +## 4. Hand-off
55 +
56 +Nothing from expE enters the expF candidate pool. Candidate 02 (H13
57 +turn-of-month) keeps its pre-registered validation protocol but now carries
58 +a strong dead-on-arrival prior — the honest expectation is a Level-1-fails
59 +negative finding for the atlas. expF (correction battery) is next.
modified experiments/micro/expE_calendar_scan/benchmark.py +192 −11
@@ -1,7 +1,7 @@
1 1 # =============================================================================
2 2 # Project : anomaly-atlas
3 3 # File : experiments/micro/expE_calendar_scan/benchmark.py
4 −# Purpose : Benchmark runner: Calendar scan: intraday, day-of-week, turn-of-month with pre-cou…
4 +# Purpose : Calendar scan with pre-counted budget + permuted-calendar null
5 5 # Author : Simon-Pierre Boucher
6 6 # Contact : contact@spboucher.ai
7 7 # Data src : hfmarketdata.io (sole data source)
@@ -10,25 +10,206 @@
10 10 # Platform : macOS / Apple Silicon (arm64)
11 11 # License : All rights reserved (research code)
12 12 # =============================================================================
13 +"""Experiment E — calendar scan (protocol + 8-test budget pre-specified in
14 +hypothesis.md). Level 0 throughout. Also measures the H20 intraday artifact
15 +profile (taxonomy input, not a hypothesis test)."""
13 16
14 −"""Benchmark entry point for expE_calendar_scan.
15 −
16 −Must embed the hardware manifest in all result output
17 −(see benchmarks/hardware_manifest.py) and write results to
18 −results/expE_calendar_scan/<timestamp>/. Uses hfmarketdata.io data ONLY, exclusively
19 −through src/anomaly_atlas/data/hf_client.py.
20 −"""
17 +from __future__ import annotations
21 18
19 +import json
22 20 import sys
21 +from datetime import UTC, date, datetime
23 22 from pathlib import Path
24 23
25 −sys.path.insert(0, str(Path(__file__).resolve().parents[3] / "benchmarks"))
24 +import numpy as np
25 +
26 +REPO_ROOT = Path(__file__).resolve().parents[3]
27 +sys.path.insert(0, str(REPO_ROOT / "benchmarks"))
28 +sys.path.insert(0, str(REPO_ROOT / "src"))
29 +
26 30 from hardware_manifest import collect_manifest # noqa: E402
27 31
32 +from anomaly_atlas.data.hf_client import HFMarketDataClient # noqa: E402
33 +from anomaly_atlas.data.universe import LIQUID_ETF, LIQUID_STOCK, TRAIN # noqa: E402
34 +from anomaly_atlas.stats.multiple_testing import benjamini_hochberg # noqa: E402
35 +from anomaly_atlas.validation.artifacts import edge_spread # noqa: E402
36 +
37 +N_PERM, PERM_SEED = 2000, 42
38 +UMICRO_WINDOW = ("2014-01-01", "2016-01-01") # H20 profile, cached from expC/D
39 +HALF_HOURS = [f"{h:02d}:{m:02d}" for h, m in
40 + [(9, 30), (10, 0), (10, 30), (11, 0), (11, 30), (12, 0), (12, 30),
41 + (13, 0), (13, 30), (14, 0), (14, 30), (15, 0), (15, 30)]]
42 +
43 +
44 +def weekday(d: str) -> int:
45 + return date(int(d[:4]), int(d[5:7]), int(d[8:10])).weekday()
46 +
47 +
48 +def class_masks(dates: list[str]) -> dict[str, np.ndarray]:
49 + """The 8 pre-declared calendar classes as boolean masks over days."""
50 + n = len(dates)
51 + wd = np.array([weekday(d) for d in dates])
52 + masks = {name: wd == i for i, name in enumerate(["mon", "tue", "wed", "thu", "fri"])}
53 +
54 + month = np.array([d[:7] for d in dates])
55 + tom = np.zeros(n, dtype=bool)
56 + for i in range(n):
57 + if i + 1 < n and month[i + 1] != month[i]:
58 + tom[i] = True # last trading day of month (-1)
59 + if i > 0 and month[i - 1] != month[i]:
60 + for j in range(i, min(i + 3, n)): # first 3 trading days (+1..+3)
61 + if month[j] == month[i]:
62 + tom[j] = True
63 + masks["turn_of_month"] = tom
64 +
65 + # holidays: a non-weekend gap in the trading calendar
66 + pre = np.zeros(n, dtype=bool)
67 + post = np.zeros(n, dtype=bool)
68 + for i in range(n - 1):
69 + d0 = date(int(dates[i][:4]), int(dates[i][5:7]), int(dates[i][8:10]))
70 + d1 = date(int(dates[i + 1][:4]), int(dates[i + 1][5:7]), int(dates[i + 1][8:10]))
71 + gap_weekdays = np.busday_count(d0.isoformat(), d1.isoformat()) - 1
72 + if gap_weekdays >= 1:
73 + pre[i] = True
74 + post[i + 1] = True
75 + masks["pre_holiday"] = pre
76 + masks["post_holiday"] = post
77 + return masks
78 +
79 +
80 +def stats_for(returns: np.ndarray, masks: dict[str, np.ndarray]) -> dict[str, float]:
81 + mu = returns.mean()
82 + return {name: float(returns[m].mean() - mu) if m.sum() >= 20 else float("nan")
83 + for name, m in masks.items()}
84 +
85 +
86 +def permutation_test(returns: np.ndarray, years: np.ndarray,
87 + masks: dict[str, np.ndarray]) -> tuple[dict, dict]:
88 + """Within-year permutation: marginal p per class + family-wise max-stat p."""
89 + rng = np.random.default_rng(PERM_SEED)
90 + observed = stats_for(returns, masks)
91 + names = list(masks)
92 + exceed = dict.fromkeys(names, 0)
93 + fw_exceed = dict.fromkeys(names, 0)
94 + perm_dist: dict[str, list[float]] = {k: [] for k in names}
95 + year_idx = [np.where(years == y)[0] for y in np.unique(years)]
96 + for _ in range(N_PERM):
97 + perm = returns.copy()
98 + for idx in year_idx:
99 + perm[idx] = perm[idx][rng.permutation(len(idx))]
100 + s = stats_for(perm, masks)
101 + max_abs = max(abs(v) for v in s.values() if np.isfinite(v))
102 + for name in names:
103 + perm_dist[name].append(s[name])
104 + if np.isfinite(s[name]) and abs(s[name]) >= abs(observed[name]):
105 + exceed[name] += 1
106 + if np.isfinite(observed[name]) and max_abs >= abs(observed[name]):
107 + fw_exceed[name] += 1
108 + marg = {k: (exceed[k] + 1) / (N_PERM + 1) for k in names}
109 + fw = {k: (fw_exceed[k] + 1) / (N_PERM + 1) for k in names}
110 + band = {k: [round(float(np.percentile(perm_dist[k], q)) * 1e4, 3) for q in (2.5, 97.5)]
111 + for k in names}
112 + return {"observed_bp": {k: round(v * 1e4, 3) for k, v in observed.items()},
113 + "perm_band95_bp": band,
114 + "p_marginal": {k: round(v, 4) for k, v in marg.items()},
115 + "p_familywise": {k: round(v, 4) for k, v in fw.items()}}, observed
116 +
117 +
118 +def h20_profile(client: HFMarketDataClient) -> dict:
119 + """Intraday half-hour profile of |return|, EDGE spread, staleness."""
120 + tickers = [("stock", t) for t in LIQUID_STOCK] + [("etf", t) for t in LIQUID_ETF]
121 + buckets = {hh: {"absret": [], "bars": [], "minutes": 0, "present": 0} for hh in HALF_HOURS}
122 + s, e = UMICRO_WINDOW
123 + for asset, ticker in tickers:
124 + bars = client.get_bars(asset, ticker, "1min", "adj_split", s, e)
125 + by_day: dict[str, list[dict]] = {}
126 + for b in bars:
127 + t = b["datetime"][11:16]
128 + if "09:30" <= t < "16:00":
129 + by_day.setdefault(b["datetime"][:10], []).append(b)
130 + n_days = len(by_day)
131 + for hh_i, hh in enumerate(HALF_HOURS):
132 + hi = HALF_HOURS[hh_i + 1] if hh_i + 1 < len(HALF_HOURS) else "16:00"
133 + sel = [b for bs in by_day.values() for b in bs if hh <= b["datetime"][11:16] < hi]
134 + closes = np.array([b["close"] for b in sel])
135 + if len(closes) > 100:
136 + r = np.abs(np.diff(np.log(closes)))
137 + buckets[hh]["absret"].append(float(np.median(r)))
138 + buckets[hh]["bars"].append(sel)
139 + buckets[hh]["minutes"] += 30 * n_days
140 + buckets[hh]["present"] += len(sel)
141 + profile = []
142 + for hh in HALF_HOURS:
143 + b = buckets[hh]
144 + spreads = []
145 + for sel in b["bars"]:
146 + o = np.array([x["open"] for x in sel])
147 + h = np.array([x["high"] for x in sel])
148 + lo = np.array([x["low"] for x in sel])
149 + c = np.array([x["close"] for x in sel])
150 + sp = edge_spread(o, h, lo, c)
151 + if np.isfinite(sp):
152 + spreads.append(sp)
153 + profile.append({
154 + "bucket": hh,
155 + "median_abs_1min_ret_bp": round(float(np.median(b["absret"])) * 1e4, 3)
156 + if b["absret"] else None,
157 + "median_edge_spread_bp": round(float(np.median(spreads)) * 1e4, 3)
158 + if spreads else None,
159 + "staleness": round(1 - b["present"] / b["minutes"], 4) if b["minutes"] else None,
160 + })
161 + return {"window": UMICRO_WINDOW, "universe": LIQUID_STOCK + LIQUID_ETF,
162 + "profile": profile}
163 +
28 164
29 165 def main() -> None:
30 − collect_manifest() # embedded in results once implemented
31 − raise NotImplementedError("experiment not yet implemented")
166 + run_utc = datetime.now(UTC)
167 + client = HFMarketDataClient()
168 +
169 + bars = client.get_bars("etf", "SPY", "1day", "adj_splitdiv", TRAIN[0], TRAIN[1])
170 + dates = [b["datetime"][:10] for b in bars]
171 + closes = np.array([b["close"] for b in bars])
172 + returns = np.diff(np.log(closes))
173 + dates = dates[1:] # return dates
174 + years = np.array([d[:4] for d in dates])
175 + masks = class_masks(dates)
176 +
177 + table, observed = permutation_test(returns, years, masks)
178 + fdr = benjamini_hochberg(np.array([table["p_marginal"][k] for k in masks]), alpha=0.05)
179 + table["fdr_marginal"] = {k: bool(r) for k, r in zip(masks, fdr, strict=True)}
180 +
181 + # sub-period stability, descriptive
182 + subs = {}
183 + for name, (lo, hi) in {"2000-2007": ("2000", "2007"), "2008-2015": ("2008", "2015")}.items():
184 + sel = (years >= lo) & (years <= hi)
185 + subs[name] = {k: round(v * 1e4, 3)
186 + for k, v in stats_for(returns[sel],
187 + {k: m[sel] for k, m in masks.items()}).items()}
188 +
189 + results = {
190 + "experiment": "expE_calendar_scan",
191 + "run_utc": run_utc.isoformat(),
192 + "author": "Simon-Pierre Boucher",
193 + "contact": "contact@spboucher.ai",
194 + "data_source": "hfmarketdata.io",
195 + "confidence_level": 0,
196 + "protocol": {"instrument": "SPY 1day adj_splitdiv", "train": TRAIN,
197 + "budget_tests": 8, "n_perm": N_PERM, "perm_seed": PERM_SEED,
198 + "class_counts": {k: int(m.sum()) for k, m in masks.items()},
199 + "n_days": int(len(returns))},
200 + "calendar_tests": table,
201 + "subperiods_bp": subs,
202 + "h20_intraday_profile": h20_profile(client),
203 + "client_stats": vars(client.stats) | {"refreshes": list(client.stats.refreshes)},
204 + "manifest": collect_manifest(),
205 + }
206 + out_dir = REPO_ROOT / "results" / "expE_calendar_scan" / run_utc.strftime("%Y%m%dT%H%M%SZ")
207 + out_dir.mkdir(parents=True)
208 + (out_dir / "results.json").write_text(json.dumps(results, indent=2) + "\n")
209 + print(f"wrote {out_dir.relative_to(REPO_ROOT)}/results.json")
210 + print(json.dumps({k: results["calendar_tests"][k] for k in
211 + ("observed_bp", "p_marginal", "p_familywise", "fdr_marginal")}, indent=1))
212 + print("H20 profile:", json.dumps(results["h20_intraday_profile"]["profile"], indent=1)[:800])
32 213
33 214
34 215 if __name__ == "__main__":
modified experiments/micro/expE_calendar_scan/hypothesis.md +58 −13
@@ -5,34 +5,79 @@ author: Simon-Pierre Boucher
5 5 contact: contact@spboucher.ai
6 6 data_source: hfmarketdata.io
7 7 created: 2026-08-12
8 −status: draft
8 +modified: 2026-08-12
9 +status: final
9 10 ---
10 11
11 12 # Hypothesis — expE_calendar_scan
12 13
14 +*Pre-specified 2026-08-12 before the scan ran. The calendar space is where
15 +p-hacking is easiest (STW 2001), so the hypothesis budget below is a hard
16 +cap: any test not listed here is not run.*
17 +
13 18 ```text
14 19 Hypothesis
15 − <what we believe and why — pre-specified BEFORE looking at results>
20 + Tests H13/H14/H18 (scan half) + measures H20. Priors from the verified
21 + literature: the Monday effect is dead (French 1980 -> decayed); holiday
22 + effects are dead; turn-of-month was the last survivor as of 2006
23 + (Marquering et al.) and its TRAIN-period status is the open question.
24 + H20 (intraday U-shape of vol and spread) is expected to be strongly
25 + present — it is a POSITIVE CONTROL and a cost-model input, not an anomaly.
16 26
17 27 Falsification criterion
18 − <the concrete measurable outcome that would prove this wrong>
28 + H14 (negative control): the pipeline is BROKEN if any weekday passes the
29 + family-wise permutation test — investigate the pipeline, not the market.
30 + H13: "turn-of-month survived in train" is falsified if the ToM window
31 + mean fails the family-wise permutation test. H20: the U-shape control
32 + FAILS the pipeline if flat (spread/vol profile must peak at open/close).
19 33
20 34 Artifact null(s)
21 − <the fake-signal baseline(s) this must beat: bounce / staleness /
22 − non-synchronous timestamps / permuted calendar / random walk>
35 + Permuted-calendar null (MANDATORY per charter §8.5): within-YEAR
36 + permutation of daily returns (preserves annual regimes, destroys calendar
37 + alignment), n = 2000, seed = 42. Family-wise: the max-|statistic| across
38 + all 8 tests is recomputed on every permutation (mini reality check) —
39 + each observed statistic gets both a marginal and a family-wise p.
40 + T4 guard: daily bars only (one close convention throughout).
23 41
24 −Method
25 − <exact procedure, universe, split (train/validation/holdout), seeds,
26 − number of hypotheses tested, correction applied>
42 +Method (budget pre-counted: 8 tests)
43 + Instrument: SPY 1day, adj_splitdiv, TRAIN 2000-2016 (single pre-declared
44 + instrument — no cross-sectional expansion). Statistics, each = mean
45 + daily return in the class minus overall mean:
46 + T1..T5 day-of-week (Mon..Fri) [H14, 5 tests]
47 + T6 turn-of-month window: last trading day through
48 + +3 first trading days of the next month [H13, 1 test]
49 + T7 pre-holiday day (day before a non-weekend gap) [H18]
50 + T8 post-holiday day [H18]
51 + Corrections: BH-FDR over the 8 marginal permutation p-values AND the
52 + family-wise max-stat p. Sub-period stability (2000-2007 / 2008-2015)
53 + reported descriptively for anything that passes.
54 + H20 measurement (no hypothesis test): per half-hour bucket on the liquid
55 + 12, 1min, 2014-2015 (cached): median |1min return|, EDGE spread per
56 + bucket, staleness per bucket -> taxonomy T1/T2 intraday profile.
27 57
28 58 Result
29 − <filled after the run: effect size, bootstrap CIs, corrected p-values,
30 − OOS status, cost-adjusted effect, credits used>
59 + Run 20260812T065222Z (1 new request — everything else cache-served;
60 + 4 024 trading days). NOTHING SURVIVES: all 8 observed effects sit inside
61 + their permuted-calendar 95% bands (family-wise p >= 0.93 everywhere;
62 + best marginal p = 0.24, turn-of-month). Monday: -1.4 bp, p_marg 0.73 —
63 + the negative control held. Turn-of-month: +4.7 bp overall, and DECAYING
64 + inside train (+7.9 bp 2000-2007 -> +1.6 bp 2008-2015). Holiday effects:
65 + wide bands (n=144), nothing. H20 control: PASSED — volatility U-shape
66 + (6.6 bp open / 2.4 midday / 2.9 close); spread declines monotonically
67 + (2.8 -> 1.2 bp), i.e. an L-shape, not a U — recorded as measured.
31 68
32 69 Interpretation
33 − <what the numbers mean, WITH confidence level (0-3); alternative
34 − explanations considered — artifact first>
70 + (Level 0.) The cleanest possible calendar answer on this data: the
71 + pre-declared 8-test budget produces zero candidates. The 2006-era "last
72 + survivor" (turn-of-month) does not clear the permuted-calendar bar in
73 + train and decays within it — candidate 02 (H13) now carries a strong
74 + dead-on-arrival prior into its validation-split protocol. Both pipeline
75 + controls behaved (Monday stayed dead; U-shape present), so this negative
76 + is a *finding*, not a power failure — though holiday-class power is low
77 + (wide bands) and is reported as such. H20 numbers feed taxonomy T1/T2:
78 + first-30-minutes claims face 2-3x the midday artifact level.
35 79
36 80 Next experiment
37 − <the most informative follow-up given this result>
81 + expF: correction battery (White RC/SPA + DSR with the 22-hypothesis
82 + budget) over everything C/D/E surfaced.
38 83 ```
modified research/LOG.md +25 −0
@@ -174,3 +174,28 @@ mid-stale, dilutes ultra-stale).
174 174
175 175 **Decision.** To expF: 2014-15 FDR residuals + ES->SPY + decay curve.
176 176 Next: expE calendar scan (pre-counted budget), then expF.
177 +
178 +## 2026-08-12 07:55 ET — expE complete: the calendar is empty; controls held
179 +
180 +**Question.** Do any of the 8 pre-declared calendar classes (H13/H14/H18)
181 +beat the permuted-calendar null on SPY train?
182 +
183 +**Experiment.** expE — budget pre-counted at 8 tests, single declared
184 +instrument, within-year permutation null (n=2000, seed 42) with family-wise
185 +max-stat, plus the H20 intraday profile measurement. 1 new API request
186 +(rest cache-served).
187 +
188 +**Result.** ZERO survivors (family-wise p >= 0.93 everywhere). Monday
189 +negative control held (-1.4bp, p .73). Turn-of-month — the 2006-era last
190 +survivor — fails (p_marg .24) and decays inside train (+7.9bp -> +1.6bp).
191 +Holiday power is low (n=144, bands +/-21bp) — reported as such. H20:
192 +volatility U-shape confirmed (6.6/2.4/2.9 bp); spread declines 2.8 -> 1.2bp
193 +(L-shape, not U) — taxonomy updated with the measured profile.
194 +
195 +**Interpretation.** STW (2001) reproduced on open data with an honest
196 +budget. Candidate 02 (turn-of-month) now carries a dead-on-arrival prior
197 +into its validation protocol — the expected atlas outcome is a first-class
198 +negative finding.
199 +
200 +**Decision.** Nothing from expE enters the expF pool. Next: expF correction
201 +battery over the C/D triage output, with the declared 22-hypothesis budget.
modified research/artifact_taxonomy.md +11 −4
@@ -113,7 +113,14 @@ pre-specified universe). Detectors validated on synthetic ground truth first
113 113
114 114 ---
115 115
116 −*Open items: intraday-seasonality of spread/staleness (U-shape) as a
117 −confounder for Q3 calendar scans (to be measured in expE); continuous-futures
118 −splice choice (3 variants exposed by the API) as a testable artifact for
119 −futures-based hypotheses.*
116 +**T1/T2 intraday profile (measured, expE/H20 — liquid 12, 1min, 2014-2015).**
117 +Volatility is U-shaped: median |1min return| 6.6 bp at 09:30 → 2.4 bp midday
118 +→ 2.9 bp at 15:30. The EDGE spread declines monotonically 2.8 → 1.2 bp (an
119 +L, not a U). Consequence: intraday return claims concentrated at the open
120 +face 2–3× the midday artifact level and double the closing spread — expE/expG
121 +must bucket-match their nulls and costs by time of day.
122 +
123 +*Open items: continuous-futures splice choice (3 variants exposed by the
124 +API) as a testable artifact for futures-based hypotheses — expD found the
125 +ES→SPY cross-serial effect splice-INVARIANT, so the residual exposure is
126 +futures-only level/trend work.*
added results/expE_calendar_scan/20260812T065222Z/results.json +279 −0
@@ -0,0 +1,279 @@
1 +{
2 + "experiment": "expE_calendar_scan",
3 + "run_utc": "2026-08-12T06:52:22.054344+00:00",
4 + "author": "Simon-Pierre Boucher",
5 + "contact": "contact@spboucher.ai",
6 + "data_source": "hfmarketdata.io",
7 + "confidence_level": 0,
8 + "protocol": {
9 + "instrument": "SPY 1day adj_splitdiv",
10 + "train": [
11 + "2000-01-01",
12 + "2016-01-01"
13 + ],
14 + "budget_tests": 8,
15 + "n_perm": 2000,
16 + "perm_seed": 42,
17 + "class_counts": {
18 + "mon": 757,
19 + "tue": 824,
20 + "wed": 827,
21 + "thu": 810,
22 + "fri": 806,
23 + "turn_of_month": 764,
24 + "pre_holiday": 144,
25 + "post_holiday": 144
26 + },
27 + "n_days": 4024
28 + },
29 + "calendar_tests": {
30 + "observed_bp": {
31 + "mon": -1.4,
32 + "tue": 3.965,
33 + "wed": -0.468,
34 + "thu": 1.501,
35 + "fri": -3.767,
36 + "turn_of_month": 4.734,
37 + "pre_holiday": 5.723,
38 + "post_holiday": 2.855
39 + },
40 + "perm_band95_bp": {
41 + "mon": [
42 + -7.85,
43 + 7.922
44 + ],
45 + "tue": [
46 + -7.904,
47 + 7.977
48 + ],
49 + "wed": [
50 + -7.886,
51 + 7.317
52 + ],
53 + "thu": [
54 + -7.368,
55 + 7.578
56 + ],
57 + "fri": [
58 + -7.986,
59 + 8.177
60 + ],
61 + "turn_of_month": [
62 + -8.208,
63 + 8.293
64 + ],
65 + "pre_holiday": [
66 + -20.797,
67 + 20.984
68 + ],
69 + "post_holiday": [
70 + -21.199,
71 + 20.675
72 + ]
73 + },
74 + "p_marginal": {
75 + "mon": 0.7286,
76 + "tue": 0.2974,
77 + "wed": 0.9075,
78 + "thu": 0.7031,
79 + "fri": 0.3368,
80 + "turn_of_month": 0.2444,
81 + "pre_holiday": 0.5752,
82 + "post_holiday": 0.7766
83 + },
84 + "p_familywise": {
85 + "mon": 1.0,
86 + "tue": 0.99,
87 + "wed": 1.0,
88 + "thu": 1.0,
89 + "fri": 0.992,
90 + "turn_of_month": 0.9735,
91 + "pre_holiday": 0.9275,
92 + "post_holiday": 0.9995
93 + },
94 + "fdr_marginal": {
95 + "mon": false,
96 + "tue": false,
97 + "wed": false,
98 + "thu": false,
99 + "fri": false,
100 + "turn_of_month": false,
101 + "pre_holiday": false,
102 + "post_holiday": false
103 + }
104 + },
105 + "subperiods_bp": {
106 + "2000-2007": {
107 + "mon": 1.895,
108 + "tue": -0.849,
109 + "wed": 2.258,
110 + "thu": 0.976,
111 + "fri": -4.177,
112 + "turn_of_month": 7.881,
113 + "pre_holiday": 1.424,
114 + "post_holiday": -1.701
115 + },
116 + "2008-2015": {
117 + "mon": -4.674,
118 + "tue": 8.727,
119 + "wed": -3.178,
120 + "thu": 2.03,
121 + "fri": -3.342,
122 + "turn_of_month": 1.602,
123 + "pre_holiday": 10.173,
124 + "post_holiday": 7.413
125 + }
126 + },
127 + "h20_intraday_profile": {
128 + "window": [
129 + "2014-01-01",
130 + "2016-01-01"
131 + ],
132 + "universe": [
133 + "AAPL",
134 + "MSFT",
135 + "NVDA",
136 + "AMZN",
137 + "GOOGL",
138 + "META",
139 + "TSLA",
140 + "JPM",
141 + "XOM",
142 + "UNH",
143 + "SPY",
144 + "QQQ"
145 + ],
146 + "profile": [
147 + {
148 + "bucket": "09:30",
149 + "median_abs_1min_ret_bp": 6.586,
150 + "median_edge_spread_bp": 2.777,
151 + "staleness": 0.0005
152 + },
153 + {
154 + "bucket": "10:00",
155 + "median_abs_1min_ret_bp": 4.37,
156 + "median_edge_spread_bp": 2.008,
157 + "staleness": 0.0004
158 + },
159 + {
160 + "bucket": "10:30",
161 + "median_abs_1min_ret_bp": 3.541,
162 + "median_edge_spread_bp": 1.348,
163 + "staleness": 0.0006
164 + },
165 + {
166 + "bucket": "11:00",
167 + "median_abs_1min_ret_bp": 3.162,
168 + "median_edge_spread_bp": 1.858,
169 + "staleness": 0.001
170 + },
171 + {
172 + "bucket": "11:30",
173 + "median_abs_1min_ret_bp": 2.761,
174 + "median_edge_spread_bp": 1.289,
175 + "staleness": 0.0019
176 + },
177 + {
178 + "bucket": "12:00",
179 + "median_abs_1min_ret_bp": 2.496,
180 + "median_edge_spread_bp": 1.401,
181 + "staleness": 0.0031
182 + },
183 + {
184 + "bucket": "12:30",
185 + "median_abs_1min_ret_bp": 2.425,
186 + "median_edge_spread_bp": 2.015,
187 + "staleness": 0.0037
188 + },
189 + {
190 + "bucket": "13:00",
191 + "median_abs_1min_ret_bp": 2.387,
192 + "median_edge_spread_bp": 1.556,
193 + "staleness": 0.0092
194 + },
195 + {
196 + "bucket": "13:30",
197 + "median_abs_1min_ret_bp": 2.362,
198 + "median_edge_spread_bp": 1.663,
199 + "staleness": 0.011
200 + },
201 + {
202 + "bucket": "14:00",
203 + "median_abs_1min_ret_bp": 2.503,
204 + "median_edge_spread_bp": 1.631,
205 + "staleness": 0.0112
206 + },
207 + {
208 + "bucket": "14:30",
209 + "median_abs_1min_ret_bp": 2.444,
210 + "median_edge_spread_bp": 1.714,
211 + "staleness": 0.0108
212 + },
213 + {
214 + "bucket": "15:00",
215 + "median_abs_1min_ret_bp": 2.602,
216 + "median_edge_spread_bp": 1.311,
217 + "staleness": 0.01
218 + },
219 + {
220 + "bucket": "15:30",
221 + "median_abs_1min_ret_bp": 2.88,
222 + "median_edge_spread_bp": 1.212,
223 + "staleness": 0.009
224 + }
225 + ]
226 + },
227 + "client_stats": {
228 + "network_requests": 0,
229 + "cache_hits": 68,
230 + "rows_fetched": 0,
231 + "seconds_waiting": 0.0,
232 + "errors_retried": 0,
233 + "refreshes": []
234 + },
235 + "manifest": {
236 + "author": "Simon-Pierre Boucher",
237 + "contact": "contact@spboucher.ai",
238 + "project": "anomaly-atlas",
239 + "data_source": "hfmarketdata.io",
240 + "collected_utc": "2026-08-12T06:52:26.380407+00:00",
241 + "chip": {
242 + "brand": "Apple M5 Max",
243 + "arch": "arm64",
244 + "cores_total": 18,
245 + "cores_performance": 6,
246 + "cores_efficiency": 12,
247 + "gpu_cores": 40
248 + },
249 + "memory": {
250 + "unified_bytes": 51539607552,
251 + "unified_gb": 48.0,
252 + "pagesize": 16384
253 + },
254 + "ssd": {
255 + "model": "APPLE SSD AP2048Z",
256 + "size": "2 TB",
257 + "smart_status": "Verified"
258 + },
259 + "os": {
260 + "product": "macOS",
261 + "version": "27.0",
262 + "build": "26A5388g",
263 + "kernel": "27.0.0"
264 + },
265 + "software": {
266 + "python": "3.14.4",
267 + "numpy": "2.5.2",
268 + "pandas": "3.0.5",
269 + "polars": "1.43.2",
270 + "duckdb": "1.5.5",
271 + "statsmodels": "0.14.6",
272 + "arch": "8.0.0"
273 + },
274 + "git": {
275 + "commit": "ac729354bfe6c9f528b2a942e6fe399730c806b8",
276 + "dirty_tree": true
277 + }
278 + }
279 +}
modified web/lib/charts.js +83 −1
@@ -200,10 +200,92 @@ function expDFigure(C) {
200 200 return out.join("");
201 201 }
202 202
203 +// ------------------------------------------------------------------- expE
204 +function expEFigure(C) {
205 + const res = latestResults(C, "expE_calendar_scan");
206 + if (!res) return "";
207 + const t = res.data.calendar_tests || {};
208 + const obs = t.observed_bp || {}, band = t.perm_band95_bp || {};
209 + const names = Object.keys(obs);
210 + if (!names.length) return "";
211 + let out = "";
212 +
213 + // Panel 1 — observed effect vs permuted-calendar 95% band (interval + dot)
214 + {
215 + const W = 760, ML = 130, MR = 20, MT = 30, RH = 30, MB = 44;
216 + const H = MT + names.length * RH + MB;
217 + const vals = names.flatMap((n) => [obs[n], ...(band[n] || [])]);
218 + const xmin = Math.min(...vals) - 2, xmax = Math.max(...vals) + 2;
219 + const iw = W - ML - MR;
220 + const xp = (v) => ML + ((v - xmin) / (xmax - xmin)) * iw;
221 + let g = "";
222 + for (const tick of [-20, -10, 0, 10, 20].filter((v) => v > xmin && v < xmax)) {
223 + g += `<line x1="${xp(tick)}" y1="${MT - 6}" x2="${xp(tick)}" y2="${MT + names.length * RH}" stroke="${tick === 0 ? "#c9c4b6" : GRID}" stroke-width="1"/>
224 +<text x="${xp(tick)}" y="${MT + names.length * RH + 16}" text-anchor="middle" ${AXIS_TXT}>${tick}</text>`;
225 + }
226 + g += `<text x="${ML + iw / 2}" y="${H - 5}" text-anchor="middle" ${AXIS_TXT}>mean daily SPY return in class minus overall mean (bp) · gray bar = permuted-calendar 95% band</text>`;
227 + names.forEach((n, i) => {
228 + const y = MT + i * RH + RH / 2;
229 + const [lo, hi] = band[n] || [0, 0];
230 + g += `<text x="${ML - 10}" y="${y + 4}" text-anchor="end" font-size="11.5" fill="${INK}">${esc(n.replace(/_/g, " "))}</text>
231 +<rect x="${xp(lo)}" y="${y - 5}" width="${Math.max(1, xp(hi) - xp(lo))}" height="10" rx="4" fill="${GRID}"/>
232 +<circle cx="${xp(obs[n]).toFixed(1)}" cy="${y}" r="5" fill="${BLUE}" stroke="${SURFACE}" stroke-width="2"><title>${esc(n)}: observed ${obs[n]} bp — marginal p ${t.p_marginal?.[n]}, family-wise p ${t.p_familywise?.[n]}</title></circle>`;
233 + });
234 + out += fig(
235 + `<svg viewBox="0 0 ${W} ${H}" role="img" aria-label="expE calendar effects vs permuted-calendar null">${g}</svg>`,
236 + `expE — all 8 pre-declared calendar tests on SPY (train 2000–2016): every observed effect (dot) sits ` +
237 + `inside its permuted-calendar 95% band (bar). Nothing survives; the last-survivor turn-of-month included. ` +
238 + `Run ${esc(res.run)}, regenerated from results.json.`
239 + );
240 + }
241 +
242 + // Panel 2 — H20 intraday profile: |return| and EDGE spread by half-hour
243 + const prof = (res.data.h20_intraday_profile || {}).profile || [];
244 + if (prof.length) {
245 + const W = 760, ML = 52, MR = 16, MT = 34, MB = 46, H = 260;
246 + const iw = W - ML - MR, ih = H - MT - MB;
247 + const ys = prof.flatMap((p) => [p.median_abs_1min_ret_bp, p.median_edge_spread_bp]).filter(Number.isFinite);
248 + const ymax = Math.max(...ys) * 1.12;
249 + const xp = (i) => ML + (i / (prof.length - 1)) * iw;
250 + const yp = (v) => MT + ih - (v / ymax) * ih;
251 + let g = "";
252 + for (const tick of [0, 2, 4, 6].filter((v) => v <= ymax)) {
253 + g += `<line x1="${ML}" y1="${yp(tick)}" x2="${ML + iw}" y2="${yp(tick)}" stroke="${GRID}" stroke-width="1"/>
254 +<text x="${ML - 6}" y="${yp(tick) + 4}" text-anchor="end" ${AXIS_TXT}>${tick}</text>`;
255 + }
256 + prof.forEach((p, i) => {
257 + if (i % 2 === 0) g += `<text x="${xp(i)}" y="${MT + ih + 16}" text-anchor="middle" ${AXIS_TXT}>${esc(p.bucket)}</text>`;
258 + });
259 + const series = [
260 + ["median_abs_1min_ret_bp", BLUE, "median |1min return|"],
261 + ["median_edge_spread_bp", ORANGE, "median EDGE spread"],
262 + ];
263 + for (const [key, color, label] of series) {
264 + const pts = prof.map((p, i) => [xp(i), yp(p[key])]).filter((q) => Number.isFinite(q[1]));
265 + g += `<path d="${pts.map((q, i) => `${i ? "L" : "M"}${q[0].toFixed(1)},${q[1].toFixed(1)}`).join(" ")}" fill="none" stroke="${color}" stroke-width="2" stroke-linejoin="round"/>`;
266 + prof.forEach((p, i) => {
267 + if (Number.isFinite(p[key])) g += `<circle cx="${xp(i).toFixed(1)}" cy="${yp(p[key]).toFixed(1)}" r="4" fill="${color}" stroke="${SURFACE}" stroke-width="2"><title>${esc(p.bucket)} — ${esc(label)}: ${p[key]} bp</title></circle>`;
268 + });
269 + }
270 + g += `<g font-size="11" fill="${INK}">
271 +<circle cx="${ML}" cy="${MT - 16}" r="4" fill="${BLUE}" stroke="${SURFACE}" stroke-width="2"/><text x="${ML + 9}" y="${MT - 12}">median |1min return| (bp)</text>
272 +<circle cx="${ML + 210}" cy="${MT - 16}" r="4" fill="${ORANGE}" stroke="${SURFACE}" stroke-width="2"/><text x="${ML + 219}" y="${MT - 12}">median EDGE spread (bp)</text></g>
273 +<text x="${ML + iw / 2}" y="${H - 5}" text-anchor="middle" ${AXIS_TXT}>half-hour bucket (RTH) · liquid 12, 1min, 2014–2015</text>`;
274 + out += fig(
275 + `<svg viewBox="0 0 ${W} ${H}" role="img" aria-label="expE intraday profile of volatility and spread">${g}</svg>`,
276 + `expE / H20 — the intraday artifact profile (taxonomy input): volatility is U-shaped (6.6 bp at the open, ` +
277 + `2.4 midday, 2.9 at the close); the spread declines monotonically (2.8 → 1.2 bp). Any "first-30-minutes" return ` +
278 + `claim faces 2–3× the midday artifact level. Run ${esc(res.run)}.`
279 + );
280 + }
281 + return out;
282 +}
283 +
203 284 const BUILDERS = {
204 285 expB_artifact_baselines: expBFigure,
205 286 expC_reversion_scan: expCFigure,
206 287 expD_leadlag_scan: expDFigure,
288 + expE_calendar_scan: expEFigure,
207 289 };
208 290
209 291 /** Figures for an experiment page ("" when none apply). */
@@ -218,7 +300,7 @@ function figuresFor(experiment, C) {
218 300
219 301 /** The most recent experiment figure, for the home page. */
220 302 function homeFigure(C) {
221 − for (const exp of ["expD_leadlag_scan", "expC_reversion_scan", "expB_artifact_baselines"]) {
303 + for (const exp of ["expE_calendar_scan", "expD_leadlag_scan", "expC_reversion_scan", "expB_artifact_baselines"]) {
222 304 const html = figuresFor(exp, C);
223 305 if (html) return { experiment: exp, html: html.split("</figure>")[0] + "</figure>" };
224 306 }
225 307