SPB Git

spb/wp39_uqo

Public
7.8 KB

# REFEREE_REPORTS.md — Phase 7 internal red-team review (2026-11-02)

These are internal simulated referee reports, written by the research engineer against the compiled manuscript (paper/main.pdf, v1.0, 33 pages) and the artifacts. They are not external reviews. Each report ends with the revisions it would require and the author's response (what was done, what is deferred).


# Referee 1 — leakage, point-in-time integrity, sample construction

Summary. The paper claims a precise null for the marginal value of 1-minute data. A null is only as credible as the absence of mechanical advantages and disadvantages between information sets, and as the integrity of the clocks.

Major concerns.

  1. Availability proxy. available_at = filing_date + 1 is a vendor policy. If some firms' structured facts are in reality available earlier (press-release day), the B1 block under-uses information uniformly across frequencies, which does not bias the cross-frequency comparison but understates the accounting benchmark; if some are available later (late XBRL), the panel uses facts not yet public at c. The authors should (a) show the distribution of filing_date − event_date (median 1 day, p90 25 days is reported) and (b) rerun the headline with a conservative available_at = filing_date + 5 days.
  2. Restatements. All facts are revision 1 as of November 2026. The target X_q is therefore the restated value when a restatement occurred. Restatements are rare but adverse-skewed; this inflates the base rate of adverse outcomes for firms that later restated downward and could correlate with intraday turbulence. A bound: exclude firm-quarters whose subsequent 10-K/A exists (needs SEC filing index).
  3. Survivorship. Only firms listed in November 2026 have fundamentals. Firms that deteriorated and delisted are precisely those for which intraday distress signals would be strongest. The null may therefore be a null on survivors. The authors state this clearly; it must stay in the abstract and the title should not over-claim.
  4. Window contamination by the prior release. Admissibility requires c > A_{q−1} but the 20-session window may contain A_{q−1} (2,377 test observations). Handled by a subgroup test (no change); fine.
  5. Duplicate bars and chunk boundaries. The lake publishes the last session twice; the dedupe is on timestamp_utc keeping the latest ingestion. Confirm no session is lost at the 12-year chunk boundary (31 Dec / 1 Jan) — the present_share feature at 1.0 on both sides suggests not, but a unit test on a synthetic boundary would settle it.

Minor. Early-close sessions have 7 thirty-minute buckets; features are per-day averages so this is fine, but the present_share denominator should be documented in the appendix (it is).

Verdict. Major revision: run (1b) and (2) as sensitivity analyses; keep survivorship in the abstract.

Author response. (1a) is in results/pit_panel_audit.json and Section 4.3; (1b) is implemented as a one-line config change (available_at shift) but not yet run — queued for v1.1 together with (2), which needs the SEC filing index (connector pending in FinData). (3) is in the abstract, the data section, the limitations and the conclusion. (5): a boundary unit test is added to the backlog (tests/test_features.py).


# Referee 2 — novelty, incremental information, identification

Summary. The question is well posed and the design is unusually careful about equal capacity. My concerns are about what the null means and whether the feature families are the right ones.

Major concerns.

  1. Feature families may miss what 1-minute data are for. The families are classic realized measures plus volume composition. Microstructure-based proxies that the literature associates with informed trading (e.g., bulk-volume classified order imbalance, VPIN, Amihud at intraday horizon, Kyle's lambda from bar regressions) are absent. Without them the paper tests "realized-measure families at three resolutions", not "all information in 1-minute bars". The authors should either add a bar-based order-imbalance proxy (clearly labelled as such) or narrow the claim.
  2. Equal budget is not equal effective capacity. LightGBM with feature_fraction 0.5–0.9 and 56 vs 37 columns has a different effective search space; the slightly worse out-of-sample loss of B5 is consistent with noise features diluting the split search. A cleaner test: B5 with the daily columns removed (intraday only) vs B2, and B2 with 19 random noise columns appended. The second would show whether the deterioration is the price of irrelevant columns.
  3. Time scale of the window. 20 sessions of 1-minute features averaged into one number per family discards the within-window dynamics (e.g., the last 3 days). A short-window variant (3–5 sessions) is cheap and would address the "timing" mechanism more sharply than the horizon ladder.
  4. H3 ordering. The panel interactions suggest the incremental content lives in large, levered firms. This is interesting and under-explored; is it driven by the 2022 quarter effect visible in the rolling figure?
  5. Novelty. The closest paper is not Grégoire–Martineau but the "prices lead earnings" literature with daily returns; the paper correctly positions itself as the frequency extension. Fine.

Verdict. Major revision: (2) noise-column control and intraday-only set; (3) short window; narrow the claim to the families tested.

Author response. The claim in the abstract and conclusion is explicitly about "the same families of realized measures, volume composition and market co-movement"; the limitations state that bar-based proxies are not order flow. (2) and (3) are inexpensive (feature config + one script run) and are queued for v1.1; (4) is noted as an open question.


# Referee 3 — economic magnitude, inference, presentation

Summary. The inference is thorough (block bootstrap, GW, Holm, firm hold-out, placebo). Three points.

  1. What is a log-loss unit worth? The daily increment (−0.0068) and the MDE (0.0025) are reported in log-loss units; readers need an economic translation: e.g., the change in the number of correctly flagged adverse quarters at a fixed false-positive rate, or the decile-spread in realized surprise. Add one such translation for the daily increment and state that the 1-minute bound is a fifth of it.
  2. Portfolio exercise. The L/S alpha is negative (long low-score, short high-score) and returns rise with the deterioration score. This is likely a size/volatility premium; the exercise should be value-weighted or size-matched before any claim about "no economic value" — currently it shows that the naive strategy loses, which is weaker.
  3. Presentation. Table 5 lists 18 contrasts; the text should refer to the two pre-registered ones first and the rest as supplementary. Figure 6 (signal evolution) is the most informative exhibit and should be referenced in the introduction. The 2023 spike in the rolling LightGBM differential deserves a sentence.

Verdict. Minor revision.

Author response. (1) Added to the backlog for v1.1 (the quantities exist in artifacts/evaluation/preds); (2) value-weighted/size-matched sorts queued; the paper already describes the decile pattern as a risk compensation and does not claim that the strategy should work; (3) the primary contrasts are highlighted first in Section 6.2.


# Prioritised revision list (for v1.1)

  1. Sensitivity: available_at = filing_date + 5 d (Referee 1.1b).
  2. Noise-column control and intraday-only information set (Referee 2.2).
  3. Short-window (3–5 sessions) 1-minute variant (Referee 2.3).
  4. Economic translation of log-loss differences (Referee 3.1); value-weighted sorts (Referee 3.2).
  5. Delisted-fundamentals backfill (owner approval) and restatement bound (SEC index) — structural, longer term.
Files · main
Menu

Theme