SPB Git

spb/wp39_uqo

Public
7.8 KB · 107 lines markdown
1# REFEREE_REPORTS.md — Phase 7 internal red-team review (2026-11-02)23These are **internal simulated referee reports**, written by the research engineer against the compiled manuscript4(`paper/main.pdf`, v1.0, 33 pages) and the artifacts. They are not external reviews. Each report ends with the5revisions it would require and the author's response (what was done, what is deferred).67---89## Referee 1 — leakage, point-in-time integrity, sample construction1011**Summary.** The paper claims a precise null for the marginal value of 1-minute data. A null is only as credible as the12absence of mechanical advantages *and* disadvantages between information sets, and as the integrity of the clocks.1314**Major concerns.**151. *Availability proxy.* `available_at = filing_date + 1` is a vendor policy. If some firms' structured facts are in16   reality available earlier (press-release day), the B1 block under-uses information uniformly across frequencies, which17   does not bias the cross-frequency comparison but understates the accounting benchmark; if some are available later18   (late XBRL), the panel uses facts not yet public at `c`. The authors should (a) show the distribution of19   `filing_date − event_date` (median 1 day, p90 25 days is reported) and (b) rerun the headline with a conservative20   `available_at = filing_date + 5 days`.212. *Restatements.* All facts are revision 1 as of November 2026. The *target* `X_q` is therefore the restated value when a22   restatement occurred. Restatements are rare but adverse-skewed; this inflates the base rate of adverse outcomes for23   firms that later restated downward and could correlate with intraday turbulence. A bound: exclude firm-quarters whose24   subsequent 10-K/A exists (needs SEC filing index).253. *Survivorship.* Only firms listed in November 2026 have fundamentals. Firms that deteriorated and delisted are26   precisely those for which intraday distress signals would be strongest. The null may therefore be a null on27   survivors. The authors state this clearly; it must stay in the abstract and the title should not over-claim.284. *Window contamination by the prior release.* Admissibility requires `c > A_{q−1}` but the 20-session window may contain29   `A_{q−1}` (2,377 test observations). Handled by a subgroup test (no change); fine.305. *Duplicate bars and chunk boundaries.* The lake publishes the last session twice; the dedupe is on `timestamp_utc`31   keeping the latest ingestion. Confirm no session is lost at the 12-year chunk boundary (31 Dec / 1 Jan) — the32   `present_share` feature at 1.0 on both sides suggests not, but a unit test on a synthetic boundary would settle it.3334**Minor.** Early-close sessions have 7 thirty-minute buckets; features are per-day averages so this is fine, but the35`present_share` denominator should be documented in the appendix (it is).3637**Verdict.** Major revision: run (1b) and (2) as sensitivity analyses; keep survivorship in the abstract.3839**Author response.** (1a) is in `results/pit_panel_audit.json` and Section 4.3; (1b) is implemented as a one-line40config change (`available_at` shift) but **not yet run** — queued for v1.1 together with (2), which needs the SEC41filing index (connector pending in FinData). (3) is in the abstract, the data section, the limitations and the42conclusion. (5): a boundary unit test is added to the backlog (`tests/test_features.py`).4344---4546## Referee 2 — novelty, incremental information, identification4748**Summary.** The question is well posed and the design is unusually careful about equal capacity. My concerns are49about what the null means and whether the feature families are the right ones.5051**Major concerns.**521. *Feature families may miss what 1-minute data are for.* The families are classic realized measures plus volume53   composition. Microstructure-based proxies that the literature associates with informed trading (e.g., bulk-volume54   classified order imbalance, VPIN, Amihud at intraday horizon, Kyle's lambda from bar regressions) are absent. Without55   them the paper tests "realized-measure families at three resolutions", not "all information in 1-minute bars". The56   authors should either add a bar-based order-imbalance proxy (clearly labelled as such) or narrow the claim.572. *Equal budget is not equal effective capacity.* LightGBM with `feature_fraction` 0.5–0.9 and 56 vs 37 columns has a58   different effective search space; the slightly *worse* out-of-sample loss of B5 is consistent with noise features59   diluting the split search. A cleaner test: B5 with the daily columns removed (intraday only) vs B2, and B2 with 1960   random noise columns appended. The second would show whether the deterioration is the price of irrelevant columns.613. *Time scale of the window.* 20 sessions of 1-minute features averaged into one number per family discards the62   within-window dynamics (e.g., the last 3 days). A short-window variant (3–5 sessions) is cheap and would address the63   "timing" mechanism more sharply than the horizon ladder.644. *H3 ordering.* The panel interactions suggest the incremental content lives in large, levered firms. This is65   interesting and under-explored; is it driven by the 2022 quarter effect visible in the rolling figure?665. *Novelty.* The closest paper is not Grégoire–Martineau but the "prices lead earnings" literature with daily returns;67   the paper correctly positions itself as the frequency extension. Fine.6869**Verdict.** Major revision: (2) noise-column control and intraday-only set; (3) short window; narrow the claim to the70families tested.7172**Author response.** The claim in the abstract and conclusion is explicitly about "the same families of realized73measures, volume composition and market co-movement"; the limitations state that bar-based proxies are not order flow.74(2) and (3) are inexpensive (feature config + one script run) and are queued for v1.1; (4) is noted as an open question.7576---7778## Referee 3 — economic magnitude, inference, presentation7980**Summary.** The inference is thorough (block bootstrap, GW, Holm, firm hold-out, placebo). Three points.81821. *What is a log-loss unit worth?* The daily increment (−0.0068) and the MDE (0.0025) are reported in log-loss units;83   readers need an economic translation: e.g., the change in the number of correctly flagged adverse quarters at a fixed84   false-positive rate, or the decile-spread in realized surprise. Add one such translation for the daily increment and85   state that the 1-minute bound is a fifth of it.862. *Portfolio exercise.* The L/S alpha is negative (long low-score, short high-score) and returns rise with the87   deterioration score. This is likely a size/volatility premium; the exercise should be value-weighted or size-matched88   before any claim about "no economic value" — currently it shows that the *naive* strategy loses, which is weaker.893. *Presentation.* Table 5 lists 18 contrasts; the text should refer to the two pre-registered ones first and the rest as90   supplementary. Figure 6 (signal evolution) is the most informative exhibit and should be referenced in the91   introduction. The 2023 spike in the rolling LightGBM differential deserves a sentence.9293**Verdict.** Minor revision.9495**Author response.** (1) Added to the backlog for v1.1 (the quantities exist in `artifacts/evaluation/preds`); (2)96value-weighted/size-matched sorts queued; the paper already describes the decile pattern as a risk compensation and97does not claim that the strategy *should* work; (3) the primary contrasts are highlighted first in Section 6.2.9899---100101## Prioritised revision list (for v1.1)1021. Sensitivity: `available_at = filing_date + 5 d` (Referee 1.1b).1032. Noise-column control and intraday-only information set (Referee 2.2).1043. Short-window (3–5 sessions) 1-minute variant (Referee 2.3).1054. Economic translation of log-loss differences (Referee 3.1); value-weighted sorts (Referee 3.2).1065. Delisted-fundamentals backfill (owner approval) and restatement bound (SEC index) — structural, longer term.107
Files · main
Menu

Theme