# project: anomaly-atlas document: Data source profile author: Simon-Pierre Boucher contact: contact@spboucher.ai data_source: hfmarketdata.io created: 2026-08-12 modified: 2026-08-12 status: reviewed

# Data source profile — hfmarketdata.io

Phase 0.5 output (Experiment A). Everything below was established empirically against the live API on 2026-08-12, through the single client (src/anomaly_atlas/data/hf_client.py); every probe is cached and recorded in data_manifest/index.jsonl. Raw evidence: results/expA_data_reality/20260812T054515Z/results.json.

# 1. API shape

OpenAPI-documented FastAPI service, open (no auth), JSON/CSV:

Bar row schema: {ticker, datetime, open, high, low, close, volume} — index bars have no volume field. Errors: 404 unknown ticker/asset, 400 invalid timeframe. end is an exclusive instant (a bare date means midnight — start=D&end=D returns nothing; use end = D+1 day).

# 2. Coverage and history depth (verified per class)

Timeframes: 1min, 5min, 30min, 1hour, 1day — 1-minute is genuinely served (verified), it is the finest granularity. Note: UNADJUSTED intraday exists only at 1min and 1day for stock/etf; 5min/30min/1hour exist only adjusted.

Data lag: this is a periodically refreshed archive, not a live feed. On Wednesday 2026-08-12, equities/fx/index ended 2026-08-07 (previous Friday), futures 2026-08-06, crypto 2026-08-09. Design experiments accordingly — no same-day data.

# 3. Timestamp semantics (critical for lead-lag work)

• Timestamps are US/Eastern wall-clock strings with no timezone marker (YYYY-MM-DD HH:MM:SS). Evidence: FX week runs Sunday 17:00 → Friday 16:59 (the classic ET convention); equity sessions run 04:00–19:59.
• Bars are bar-start labeled (first RTH bar 09:30, last extended bar 19:59).
• Session windows on the probe day (Thu 2026-08-06): stock/etf 04:00–19:59 (pre + RTH + post), futures ≈ 24 h, fx 24 h (ET week), crypto 24/7, index SPX 09:30–16:20 (settlement prints after the close).
• These are vendor-consolidated last-trade bars at minute resolution. Assume vendor time ≈ exchange time, but cross-asset closes are non-synchronous by construction (16:00 auction vs 16:20 index prints vs 24 h sessions) — a built-in source of spurious lead-lag (see artifact taxonomy).

# 4. Missing data: bars exist only where trades occurred

• No zero-volume placeholder bars anywhere (0 found on probe day).
• Liquid names are near-complete in RTH (AAPL & SPY: 390/390 RTH minutes) but sparse pre/post (AAPL: 904 of 960 extended minutes).
• Illiquid names are sparse even in RTH: AIZN printed 38 bars in the whole day. A last-observation-carried-forward join makes such series look autocorrelated and cross-predictable — this is the stale-price artifact, to be neutralized explicitly (Experiment B).

# 5. Daily-bar semantics (do NOT mix carelessly with 1min)

For AAPL on 2026-08-06 (adj_splitdiv):

• Daily bar = RTH-only OHLC with the official auction close: daily close 312.41 vs 312.49 for the last 1min RTH bar — the closing auction is not in the 1min series.
• Daily volume includes consolidated/auction volume absent from 1min bars: daily 46.14 M vs 34.67 M (sum of all extended-hours 1min) vs 25.69 M (RTH 1min only). Any volume-based signal must pick one convention and stick to it.

# 6. Corporate actions / adjustments (verified on AAPL 4:1, 2020-08-31)

• Split arithmetic is exact.
• adj_splitdiv re-bases the entire history to the dataset build date — even the 2020-08-31 close differs from its traded price. Adjusted series are therefore not point-in-time stable: they change whenever a new dividend occurs. For any experiment sensitive to this, use UNADJUSTED + explicit adjustment, or freeze the cache (which our client does by design).

# 7. Options

Daily granularity: one row per contract per trade_date, 17 columns — bid/ask, last_price, bid_iv/ask_iv, delta/gamma/vega/theta/rho, open_interest, volume, strike, expiry, call_put. 67 quarters (2010_q1 → 2026_q3_partial). SPY had 35 listed expirations on 2026-06-15. Old quarters may carry null Greeks (documented server-side NaN handling). This is the only bid/ask information anywhere in the source — options spreads may inform equity cost models (Experiment G), with care.

# 8. Limits, performance, credits

• Hard cap: 50 000 rows per JSON response (requested 1 000 000, got 50 000). The client paginates on the last datetime; verified on a full year of SPY 1min: 218 006 rows, 0 duplicates, strictly ascending.
• No rate-limit headers, no 429 observed at gentle sequential rates (~5–7 req/s). Latency: 0.08–0.22 s small requests, ~1.7 s per 50 000-row page. There is no credit system — but the client throttles anyway (min 0.15 s between requests) and caches everything; the cache is the reproducibility anchor.
• format=csv exists (not exercised in Experiment A; JSON + local parquet cache is our path).

# 9. Consequences for the research design

1. Universe: ample for Q1–Q3 — thousands of stock/etf tickers × 26 years × 1min. Holdout years are affordable.
2. Artifact taxonomy seeds (→ artifact_taxonomy.md): stale/missing minutes; non-synchronous session ends across assets; auction close absent from 1min bars; daily-vs-intraday volume mismatch; rolling adjustment anchor; continuous-futures splicing method (3 variants exposed — good: the choice is testable).
3. No bid/ask on bars: spreads must be estimated (Roll model et al.) — the bounce null (Experiment B) is mandatory, not optional.
4. Minute data is the floor: intraday lead-lag finer than 1 minute is untestable here; hypotheses must respect that (Epps effect at 1min scale).
5. Data ends ~the previous Friday: "recent regime" claims are bounded.