mode: computational discipline: computer-science title: ""
Research brief — (computational)
1. Research objective and scope
- Question. <e.g., Does method M improve metric K over baseline B on task T under budget C?>
- Hypotheses. <H1…; include the null.>
- Compute scope. CPU-only Apple Silicon node (16 cores, ≤ 32 GB RAM), ≤ hours; no GPUs. Size experiments accordingly.
2. Domain, mode and methodology
- Discipline: <AI/ML/systems/algorithms/optimization>. Primary mode: computational.
- Methodology: <benchmark design, baselines, metrics, statistical tests across seeds>.
3. Contribution and novelty
- <Expected finding; the closest prior methods/benchmarks to compare with.>
4. Literature and source rules
- Indexes: arXiv + Semantic Scholar + Crossref. Target ≥ 12 verified sources; must-read: .
5. Benchmark and data
- Datasets/tasks: <public, licensed; exact versions/URLs; splits>. Leakage controls: <…>.
- Baselines: <at least two, including a strong simple one>. Implementation sources (cite).
6. Procedure
- Environment lock in
code/(uv), deterministic seeds (≥ 5 seeds), fixed splits. - Implement method and baselines; unit-test core functions.
- Run the pre-registered grid; log every run with commands, seeds, wall time, resources in
results/experiment-log.jsonl. - Aggregate deterministically into
results/metrics.json; produce tables/figures with CIs. code/reproduce.shregenerates everything.
7. Baselines, robustness and falsification
- Ablations: . Sensitivity: <hyper-parameters, data size>. Report when the method loses.
8. Reproducibility and artifacts
- Pinned environment, seeds, logs, metrics, figure scripts, resource usage table.
9. Manuscript
- AutoScholar class; algorithm pseudo-code; results tables with mean ± CI over seeds; 8–16 pages.
10. Review and revision
- Reviewer focus: unfair baselines, seed variance, leakage, cherry-picked metrics, missing ablations.
11. Limitations, ethics, legal
- <Compute limits, dataset licenses, generalization.>
12. Acceptance criteria
- All runs logged and reproducible; ≥ 5 seeds; ≥ 2 baselines; ≥ 2 ablations; reported numbers traceable; ≥ 12 verified citations.