SPB Git

spb/anomaly-atlas Public License

Systematic discovery & rigorous validation of statistical anomalies in open HF market data (hfmarketdata.io) — pre-registered, artifact-null-driven, fully reproducible. Live atlas: www.anomaly-atlas.io

Python 61.4% JavaScript 28.7% CSS 8.6% Shell 0.7% Makefile 0.5%
29.7 KB

# CLAUDE.md

Project codename: anomaly-atlas — Systematic Discovery & Rigorous Validation of Statistical Anomalies in Open High-Frequency Market Data Data source (sole permitted source): HF Market Data — https://www.hfmarketdata.io (API) Public site (living atlas of findings): provisional — decided later (e.g. anomalyatlas.io) Principal investigator / Author: Simon-Pierre Boucher — contact@spboucher.ai Primary platform: Apple Silicon Mac (macOS 14+) Document status: Living research charter. Claude must treat this as the authoritative project specification.


# 0. Administrative conventions (MANDATORY — read before anything else)

# 0.1 Author header requirement

Every single source file created in this project — without exception — must begin with a standardized author header.

Applies to: Python, C++, shell, SQL, Makefiles, CMake, TypeScript/JS (site), and any config that supports comments.

# Python / shell / SQL / YAML / TOML header

python
# =============================================================================
#  Project   : anomaly-atlas
#  File      : <relative/path/to/file.py>
#  Purpose   : <one-line description of what this file does>
#  Author    : Simon-Pierre Boucher
#  Contact   : contact@spboucher.ai
#  Data src  : hfmarketdata.io (sole data source)
#  Created   : <YYYY-MM-DD>
#  Modified  : <YYYY-MM-DD>
#  Platform  : macOS / Apple Silicon (arm64)
#  License   : All rights reserved (research code)
# =============================================================================

# C++ / TS / JS header

cpp
// ============================================================================
//  Project   : anomaly-atlas
//  File      : <relative/path/to/file>
//  Purpose   : <one-line description>
//  Author    : Simon-Pierre Boucher
//  Contact   : contact@spboucher.ai
//  Data src  : hfmarketdata.io (sole data source)
//  Created   : <YYYY-MM-DD>
//  Modified  : <YYYY-MM-DD>
//  Platform  : macOS / Apple Silicon (arm64)
//  License   : All rights reserved (research code)
// ============================================================================

# Markdown research documents (front matter)

markdown
---
project: anomaly-atlas
document: <name>
author: Simon-Pierre Boucher
contact: contact@spboucher.ai
data_source: hfmarketdata.io
created: <YYYY-MM-DD>
status: draft | reviewed | final
---

Rules:

  1. Header is the first content of the file (after a shebang if present).
  2. Modified updated on every substantial change.
  3. Write tools/check_headers.py early; it fails if any tracked source file lacks a conforming header, and runs before every commit.
  4. Every result file (JSON/CSV/parquet) embeds "author": "Simon-Pierre Boucher", "contact": "contact@spboucher.ai", "data_source": "hfmarketdata.io" in its metadata.
  5. Every published finding on the site carries a visible attribution footer.

# 0.2 macOS-first design constraint

Everything must run on a Mac. Specifically:

  • Target machine: Apple Silicon (M1–M4), 16–64 GB unified memory, internal NVMe SSD.
  • Stack: Python (pandas/polars, numpy, statsmodels, arch, scipy) + DuckDB for out-of-core columnar queries over the local data cache; Accelerate/NEON via numpy. No Linux-only assumptions.
  • Data volumes (years of 1-minute bars across many symbols + full options chains) will exceed RAM: use DuckDB + parquet on the internal NVMe, columnar and memory-mapped, benchmarked on macOS/APFS. Never assume the dataset fits in memory.
  • Instrumentation uses macOS-native sources where relevant (mach task_info, vm_stat, fs_usage).
  • Site is a static site that builds and previews locally on the Mac.
  • A hardware/software manifest (benchmarks/hardware_manifest.py) is recorded into every result.

# 0.3 Data-source discipline (CRITICAL)

  • hfmarketdata.io is the SOLE data source. No other market data may enter any experiment or finding. Every result must be reproducible from hfmarketdata.io alone.
  • Build a single, well-tested API client (src/anomaly_atlas/data/hf_client.py) that ALL code uses. No ad-hoc requests scattered around.
  • The client must: respect rate limits (read them from response headers, back off on 429, never hammer), cache every response to a local parquet/DuckDB store keyed by (endpoint, params, as-of date), and never silently refetch — the local cache is the reproducibility anchor.
  • Record, for every symbol/field used: the exact endpoint, parameters, granularity actually returned, and the fetch timestamp, into a committed data-manifest index (the raw data lives outside git; the index is committed).
  • Confirm the real capabilities empirically before designing anything (Phase 0.5): what symbols, what history depth, what the finest true granularity is (advertised 1-minute to daily; verify), whether timestamps are exchange time or vendor time, how corporate actions/splits/dividends are handled, and what the rate/credit limits are. Write findings to research/data_source_profile.md.

# 0.4 Repository discipline

  • Git from day one; no result reported from an uncommitted tree.
  • Every finding reproducible from: commit hash + config + data-manifest index + seed + hardware manifest.
  • Python: type hints, ruff, pytest (statistical routines get correctness tests on synthetic data with known answers).

# 1. Mission

Investigate, design, implement, and rigorously validate a systematic methodology — and the tooling behind it — to discover statistical anomalies in open high-frequency market data (mean-reversion, lead-lag relationships, calendar/seasonal effects, and related regularities), and to distinguish genuine, out-of-sample, cost-aware regularities from data artifacts and statistical illusions, using only hfmarketdata.io data, on a consumer Mac, and publish the results as a reproducible atlas.

The target is not:

  • a live trading system, a trading bot, or trading advice;
  • a claim of "arbitrage" or "free money" (see §2.1 — that is almost never real in historical 1-minute data);
  • a backtest that reports in-sample returns as if they were discoveries;
  • p-hacking a large symbol universe until something looks significant;
  • reproducing a known factor/anomaly without adding rigor or a new test.

The target is:

Given only open HF market data, determine which statistical regularities are real (survive out-of-sample testing, multiple-comparison correction, transaction costs, and robustness checks) versus which are artifacts (bid-ask bounce, stale/non-synchronized timestamps, survivorship, look-ahead, microstructure noise) — and publish each with its honest confidence level and the exact script that regenerates it.

Do not assume any anomaly is real. Assume it is an artifact until the evidence forces otherwise.


# 2. Core research questions & the honesty doctrine

# 2.1 The honesty doctrine (non-negotiable, stated up front and on the site)

In financial data, the equivalent of a mathematical proof is a pre-specified, out-of-sample, transaction-cost-adjusted, multiple-testing-corrected test that a naive researcher cannot fake after the fact. There is no riskless "proof of profit" in historical bar data. Therefore:

  • Every candidate anomaly is guilty (an artifact) until proven innocent.
  • Any regularity that looks like arbitrage in 1-minute historical bars is treated as a data-artifact hypothesis first (stale quotes, timestamp misalignment, bid-ask bounce, missing corporate actions), and only after those are excluded may it be considered a real risk-bearing regularity — never riskless arbitrage.
  • In-sample results are never findings. Only out-of-sample, robustness-checked results are.
  • Past statistical regularity does not imply future returns; the site says so on every page.

# 2.2 Research questions

Q1 — Mean-reversion. At which horizons, on which instruments, does price/return mean-revert beyond what bid-ask bounce alone explains? Does any reversion survive realistic costs?

Q2 — Lead-lag. Do movements in one instrument systematically precede correlated instruments (e.g., large-cap → related ETF, index → constituents, one FX cross → another) at the available granularity? Is the lead-lag stable out-of-sample, or an artifact of non-synchronous last-trade timestamps?

Q3 — Calendar / seasonal effects. Are there robust intraday (open/close), day-of-week, turn-of-month, or holiday-adjacent regularities in returns or volatility that survive multiple-testing correction across the many calendar hypotheses one can pose?

Q4 — Artifact taxonomy. What is the complete catalogue of artifacts in this specific dataset that masquerade as anomalies, and how is each detected and neutralized? (This is a primary deliverable, not a side note.)

Q5 — Cost frontier. At what transaction-cost / holding-period assumptions does each surviving regularity stop being economically meaningful?

The distinction to keep sharp everywhere:

text
statistically detectable in-sample

reproducible out-of-sample

robust to artifacts and specification choices

economically meaningful after costs

A finding only advances one confidence level at a time (§10).


# 3. Project structure (real research-project layout)

Create this skeleton (with header-compliant placeholders) before Phase 1 concludes:

text
anomaly-atlas/
├── CLAUDE.md
├── README.md
├── CITATION.cff
├── LICENSE
├── pyproject.toml
├── Makefile                        # setup, lint, test, fetch, scan, validate, site, headers

├── research/
│   ├── LOG.md                      # dated, append-only research log
│   ├── data_source_profile.md      # Phase 0.5: what hfmarketdata.io actually provides
│   ├── state_of_the_art.md        # Phase 2
│   ├── artifact_taxonomy.md        # the catalogue of dataset artifacts (living doc)
│   ├── research_gaps.md           # Phase 3
│   ├── candidate_ranking.md       # Phase 4
│   ├── methodology.md             # the formal validation protocol (Phase 9)
│   ├── novelty_check.md           # Phase 11
│   └── bibliography.md

├── src/
│   └── anomaly_atlas/
│       ├── data/
│       │   ├── hf_client.py         # THE single API client (rate-limit, cache, manifest)
│       │   ├── cache.py             # DuckDB/parquet local store
│       │   ├── universe.py          # symbol universe definitions (versioned)
│       │   ├── calendars.py         # trading calendars, sessions, holidays
│       │   └── cleaning.py          # corporate actions, dedup, alignment, quality flags
│       ├── stats/
│       │   ├── reversion.py         # variance ratios, Hurst, AR tests, half-life
│       │   ├── leadlag.py           # cross-correlation, Granger, lagged regressions
│       │   ├── calendar.py          # seasonal/intraday effect tests
│       │   ├── multiple_testing.py  # FDR/Bonferroni, White reality check, SPA, DSR
│       │   └── bootstrap.py         # block/stationary bootstrap, CIs
│       ├── validation/
│       │   ├── oos.py               # train/validate/holdout splits, walk-forward
│       │   ├── costs.py             # spread/impact/commission models
│       │   ├── artifacts.py         # artifact detectors (bounce, staleness, look-ahead)
│       │   └── robustness.py        # specification/parameter perturbation
│       ├── atlas/                   # finding schema, provenance, confidence serialization
│       ├── viz/
│       └── instrumentation/

├── experiments/
│   ├── micro/
│   │   ├── expA_data_reality/          # verify what the API really gives
│   │   ├── expB_artifact_baselines/    # measure bounce/staleness null distributions
│   │   ├── expC_reversion_scan/
│   │   ├── expD_leadlag_scan/
│   │   ├── expE_calendar_scan/
│   │   ├── expF_multiple_testing/      # how much survives correction
│   │   ├── expG_cost_frontier/
│   │   └── expH_oos_stability/
│   ├── candidate_01/
│   ├── candidate_02/
│   └── candidate_03/
│       └── (each: README.md, hypothesis.md, implementation/, benchmark.py,
│                results/, analysis.md)

├── atlas/                          # the findings, as versioned data products
│   └── <anomaly_id>/<version>/
│       ├── finding.json
│       ├── provenance.json         # commit, config, data-manifest hash, hardware, dates
│       └── confidence.md           # Level 0–3 with evidence

├── site/                           # static site (builds locally on Mac)
│   ├── content/                    # one page per finding + the artifact taxonomy + methodology
│   └── data/

├── benchmarks/
│   ├── hardware_manifest.py
│   └── synthetic/                  # synthetic series with KNOWN properties (test the tests)

├── results/
├── tools/
│   ├── check_headers.py
│   ├── new_experiment.py
│   └── new_finding.py              # scaffolds a compliant atlas entry (provenance+confidence)
└── docs/

tools/new_finding.py refuses to create an atlas entry without a completed provenance.json and confidence.md.


# 4. Phase 1 — Ultra-deep literature research

Before proposing methodology, deeply search the literature and implementations. Prefer primary sources; log everything in research/bibliography.md with URLs and access dates; follow citations both directions.

# Areas that must be investigated

# 4.1 The anomalies themselves

Short-horizon mean-reversion (Lo–MacKinlay, Lehmann); variance-ratio tests; lead-lag effects and the Epps effect; non-synchronous trading bias (Scholes–Williams); calendar effects (Monday/weekend, turn-of-month, turn-of-year, holiday, intraday U-shape); the historical rise-and-fade of these effects after publication.

# 4.2 The statistics of not fooling yourself (most important)

Multiple-testing in finance (Harvey–Liu–Zhu "…and the Cross-Section of Expected Returns"); White's Reality Check; Hansen's SPA test; the Deflated Sharpe Ratio and backtest overfitting (Bailey–López de Prado); data-snooping (Sullivan–Timmermann–White); false-discovery-rate control; the replication crisis in cross-sectional anomalies (McLean–Pontiff decay; Hou–Xue–Zhang replication).

# 4.3 Market microstructure & artifacts

Bid-ask bounce (Roll model); microstructure noise in high-frequency data; realized-volatility bias; stale prices; the effect of using bar/last-trade timestamps vs synchronized clocks; survivorship and look-ahead bias; corporate-action handling.

# 4.4 Time-series methodology

Variance ratios; Hurst/long-memory estimation pitfalls; Granger causality caveats; block/stationary bootstrap (Politis–Romano); walk-forward and combinatorial purged cross-validation; regime changes and structural breaks.

# 4.5 Transaction-cost realism

Effective spread estimation from bar data; the gap between paper returns and net returns; how costs kill most short-horizon anomalies; capacity constraints.

# 4.6 This dataset specifically

Whatever hfmarketdata.io documents about coverage, granularity, timestamp semantics, and corporate-action adjustment — treat gaps in documentation as things to establish empirically (Phase 0.5 / Experiment A).


# 5. Phase 2 — State-of-the-art map

Produce research/state_of_the_art.md. Organize each anomaly/technique by:

text
Anomaly / method
What it claims
Data granularity it requires (does 1-minute-to-daily suffice?)
Known artifact confounds
Post-publication decay evidence
Correct statistical test (and its assumptions)
Multiple-testing exposure
Transaction-cost sensitivity
Open-source implementation (builds on macOS arm64?)
Main limitation
Opportunity for a genuinely new, honest test on THIS data

Do not summarize papers uncritically. Flag each anomaly's epistemic status: robust / decayed-after-publication / disputed / likely-artifact.


# 6. Phase 3 — Identify genuine gaps

Produce research/research_gaps.md. Aim for ≥20 substantially different, testable hypotheses on this dataset. For each:

  1. what existing work shows;
  2. what is untested on open HF data at this granularity;
  3. why it might matter;
  4. the plausible mechanism;
  5. the biggest artifact/statistical reason it would be fake;
  6. the smallest Mac-runnable experiment that would falsify it.

Prefer hypotheses where the null (artifact) model is explicitly constructable, so "real vs artifact" is a clean test — not a vibe.


# 7. Phase 4 — Rank candidate approaches

Create research/candidate_ranking.md. Score every hypothesis 1–10 with reasoning on:

text
Testability on hfmarketdata.io (data actually available)
Cleanliness of the artifact null (can we build the fake-signal baseline?)
Out-of-sample feasibility (enough history to hold out)
Multiple-testing discipline (small, pre-specified hypothesis set?)
Survival odds after costs
Novelty of the honest test
Reproducibility
Public-atlas value
Implementation complexity
Risk of self-deception

Select 3–5 strongest for prototyping. Prefer small, pre-registered hypothesis sets over giant blind scans (which invite p-hacking).


# 8. Phase 5 — Experimental framework

Build the framework before any scan. Correctness first.

# 8.1 The tests must be tested (mandatory)

Before running any anomaly detector on real data, run it on synthetic series with known properties (benchmarks/synthetic/): pure random walks (must yield NO anomaly), series with a planted mean-reversion / lead-lag / seasonal effect (must be recovered), and series with only bid-ask bounce (must be flagged as artifact, not anomaly). A detector that finds anomalies in a random walk is broken and must be fixed before touching real data.

# 8.2 Universe & splits

Pre-specify the symbol universe and the train / validation / holdout time split before looking at results. The holdout is touched once, at the end, for each finding. Walk-forward / purged CV where appropriate.

# 8.3 Instrumentation

Measure API credits consumed, cache hit rate, wall-clock, RAM/peak, DuckDB query cost. And for every statistical result: number of hypotheses tested, correction applied, out-of-sample status, bootstrap CIs, effect size, cost-adjusted effect size.

# 8.4 Most important metrics

text
OUT-OF-SAMPLE SURVIVAL  (does the effect hold on untouched data?)
ARTIFACT-ADJUSTED EFFECT (effect minus what the artifact null explains)
COST-ADJUSTED EFFECT     (net of realistic transaction costs)
CORRECTED SIGNIFICANCE   (after multiple-testing correction)

A large in-sample effect with zero out-of-sample survival is a negative result and is published as one.

# 8.5 Controls

Every detector runs against: a random-walk null, a bid-ask-bounce null, a phase-randomized/bootstrap null, and (for calendar effects) a permuted-calendar null. No claim without its null shown.


# 9. Phase 6 — Micro-experiments

Each in experiments/micro/ with the standard scaffold.

Experiment A — Data reality check. Empirically establish what hfmarketdata.io returns: true granularity, history depth per asset class, timestamp semantics (exchange vs vendor), corporate-action adjustment, missing-data patterns, gaps, and rate/credit limits. Output → research/data_source_profile.md. Everything downstream depends on this.

Experiment B — Artifact baselines. Build and measure the null distributions of the key artifacts on this data: Roll-model bid-ask bounce, stale-price autocorrelation, non-synchronous lead-lag. These nulls become the baseline every real detector must beat.

Experiment C — Mean-reversion scan. Variance ratios / half-life / AR tests across horizons and instruments, always reported net of the bounce null, with OOS split and bootstrap CIs.

Experiment D — Lead-lag scan. Lagged cross-correlations / Granger across correlated instrument pairs, explicitly separating true lead-lag from non-synchronous-timestamp artifacts (test on synchronized vs raw timestamps).

Experiment E — Calendar scan. Intraday, day-of-week, turn-of-month, holiday effects — with a pre-counted hypothesis budget and mandatory multiple-testing correction (the calendar space is where p-hacking is easiest).

Experiment F — Multiple-testing survival. Take everything C–E surfaced and apply White Reality Check / SPA / FDR / Deflated Sharpe. Report the survival rate: what fraction of "significant" effects survive correction. This number calibrates the whole atlas.

Experiment G — Cost frontier. For each surviving effect, sweep transaction-cost assumptions and report where net effect crosses zero.

Experiment H — Out-of-sample stability. For survivors of F and G, evaluate on the untouched holdout and across sub-periods. Only Level-3 findings survive this.


# 10. Scientific discipline & confidence taxonomy

For every experiment write, in hypothesis.md / analysis.md:

text
Hypothesis
Falsification criterion
Artifact null(s)
Method
Result (effect size, CIs, corrected p-values, credits used)
Interpretation (with confidence level)
Next experiment

Never report an in-sample number as a finding. Never claim "arbitrage." Never imply future profit. Report: number of hypotheses, correction method, OOS status, costs, hardware, data-manifest hash, software versions.

# Confidence taxonomy (used everywhere, including on the site)

text
Level 0 — in-sample only        (never published as a finding; scan output only)
Level 1 — corrected & OOS       (survives multiple-testing correction AND a clean
                                 out-of-sample split, with the artifact null subtracted)
Level 2 — robust                (Level 1 + robust to specification/parameter choices
                                 and across sub-periods / instruments)
Level 3 — cost-real & held-out  (Level 2 + economically nonzero after realistic costs
                                 AND confirmed on the once-touched holdout)

Every atlas entry carries its level in confidence.md and visibly on the site. Negative results (effects that vanish out-of-sample or after costs) are first-class published findings — they are the most trustworthy output of an honest anomaly project.


# 11. Phase 7 — Prototype candidate detectors

Each strongest hypothesis → experiments/candidate_0N/ with full scaffold. Each analysis.md ends with an explicit verdict: promote to atlas, iterate, or reject (with reasons and the confidence level reached).


# 12. Phase 8 — Automatic research loop

Operate as a research agent: analyze; pick the most informative next experiment; re-search literature when results surprise (especially if something looks too good — that is a red flag for an artifact, not a discovery); update hypotheses; rerun; compare; continue until evidence settles each hypothesis. Never expand the hypothesis space silently to chase significance — that is p-hacking; any expansion is logged and re-budgeted for multiple testing.

Maintain research/LOG.md (append-only): date/time+timezone, question, experiment, result, interpretation, decision.


# 13. Phase 9 — Formalize the validation methodology

Write research/methodology.md: the reusable, pre-registered protocol that turns (a pre-specified hypothesis + universe + split) into (a confidence-labeled atlas entry). It must specify: the artifact nulls, the correction procedure, the OOS/holdout protocol, the cost model, the robustness perturbations, the finding schema, and the publication checklist. The methodology is itself a primary deliverable and the main defense against self-deception.


# 14. Phase 10 — Build the atlas pipeline and site

Build the pipeline supported by evidence. Possible CLI shape (finalize after research):

bash
anomaly-atlas fetch  --universe universes/core.yaml       # cache from hfmarketdata.io
anomaly-atlas scan   --family reversion,leadlag,calendar  # in-sample candidates (Level 0)
anomaly-atlas validate <anomaly_id>                        # corrections, OOS, costs, robustness
anomaly-atlas publish <anomaly_id> --site site/            # only Level ≥1 gets published

# The site

  • Static, builds locally on the Mac; every page generated from atlas data (never hand-typed numbers).
  • Front-and-center honesty banner (the §2.1 doctrine) and a "not trading advice" statement on every page.
  • One page per finding, showing: confidence level, the artifact null it beat, OOS and cost-adjusted numbers, provenance (commit, data-manifest hash, dates, hardware), and the exact command to reproduce.
  • A prominent artifact taxonomy page (often the most useful output).
  • A prominent negative-results section.
  • Attribution footer: Simon-Pierre Boucher — contact@spboucher.ai.

# Hard constraints

  • hfmarketdata.io is the only data source.
  • No claim above its evidence level; no "arbitrage"/profit claims; not a trading system.
  • Nothing published that cannot be regenerated from committed code + the data-manifest index.

# 15. Target hardware

text
Apple Silicon Mac (laptop or desktop), 16–64 GB unified memory, internal NVMe SSD.

DuckDB + parquet over the internal SSD handles multi-year, multi-symbol data out-of-core; numpy/Accelerate handle the math. Document (without necessarily implementing) how the pipeline maps to other hardware.

# Stretch target

text
A small set (single digits) of Level-3 findings — statistical regularities that survive
correction, out-of-sample testing, robustness checks, AND realistic costs on the
once-touched holdout — OR a rigorously documented conclusion that, on this open data,
the honest answer is "the detectable effects do not survive," which is itself a
publishable, valuable result.

Do not assume Level-3 findings exist. The research must establish the truth either way.


# 16. What counts as a real result

A. A statistical regularity that reaches Level 3 (corrected + OOS + robust + cost-real + held-out) on hfmarketdata.io data. B. A rigorous, reusable artifact taxonomy for this dataset that lets others avoid mistaking artifacts for signals. C. A clean demonstration that a widely-cited anomaly does NOT survive honest testing on this open data (negative result). D. A methodological contribution: a better artifact null or a better honest test for short-horizon effects on bar data. E. A measured "survival curve": what fraction of naively-significant effects remain after each layer of rigor.

# 17. Failure criteria (and why they are fine)

Be willing to conclude:

text
nothing survives multiple-testing correction
everything vanishes out-of-sample
all apparent effects are explained by bid-ask bounce / stale prices / timestamp misalignment
surviving effects die after realistic costs
the data granularity is too coarse to test the interesting hypotheses

If so, document it in research/LOG.md and the relevant analysis.md, publish the negative result, and pivot. A well-documented "it's all artifacts" is a success, not a failure — it is exactly what an honest anomaly project should be willing to conclude.


# 18. Deliverables

text
README.md, CITATION.cff
research/ (data_source_profile, state_of_the_art, artifact_taxonomy, research_gaps,
           candidate_ranking, methodology, novelty_check, LOG, bibliography)
src/       (the anomaly_atlas library, incl. the single hf_client)
experiments/, benchmarks/ (incl. synthetic test series), results/
atlas/     (confidence-labeled findings with provenance)
site/      (the public atlas)
tools/, docs/

The final README explains: the problem; the honesty doctrine; what the data actually provides; the artifact taxonomy; the validation methodology; the findings with their confidence levels (including negatives); limitations; exact Mac reproduction instructions; future research.


# 19. Phase 11 — Novelty verification

Before claiming any finding or method is new, search using terminology derived from what was actually found, plus every synonym and the specific anomaly's literature (it may be a known, already-decayed effect). Assume not novel until evidence says otherwise. Record in research/novelty_check.md. Note explicitly whether each "finding" is a genuinely new regularity or a re-confirmation/decay-measurement of a known one — both are legitimate, but must not be conflated.


# 20. Most important instruction

Do not become attached to finding a profitable anomaly. The purpose of this project is not to discover free money — that almost certainly is not there in open historical bar data, and pretending otherwise is how researchers fool themselves.

The actual assignment is:

Search deeply, test honestly, build the artifact nulls, correct for multiple testing, hold out data you touch only once, subtract realistic costs — and report, at its true confidence level, what actually survives, including (and especially) when the honest answer is "nothing survives."

Start with literature. Verify what the data really is. Build detectors that a random walk cannot fool. Then let the out-of-sample, cost-adjusted, correction-survived evidence — and nothing else — decide what enters the atlas.

The ideal outcome is not a backtest that looks good. The ideal outcome is a trustworthy, reproducible atlas of what is real, what is artifact, and what is merely wishful in open high-frequency market data — where every claim carries its evidence level and the exact command that regenerates it from hfmarketdata.io.


Author: Simon-Pierre Boucher — contact@spboucher.ai — Data source: hfmarketdata.io — All research artifacts in this repository carry this attribution. This project is research on statistical properties of market data; it is not investment advice and is not a trading system.