SPB Git

spb/autoscholar

Public

# mode: computational discipline: computer-science title: ""

# Research brief — (computational)

# 1. Research objective and scope

  • Question. <e.g., Does method M improve metric K over baseline B on task T under budget C?>
  • Hypotheses. <H1…; include the null.>
  • Compute scope. CPU-only Apple Silicon node (16 cores, ≤ 32 GB RAM), ≤ hours; no GPUs. Size experiments accordingly.

# 2. Domain, mode and methodology

  • Discipline: <AI/ML/systems/algorithms/optimization>. Primary mode: computational.
  • Methodology: <benchmark design, baselines, metrics, statistical tests across seeds>.

# 3. Contribution and novelty

  • <Expected finding; the closest prior methods/benchmarks to compare with.>

# 4. Literature and source rules

  • Indexes: arXiv + Semantic Scholar + Crossref. Target ≥ 12 verified sources; must-read: .

# 5. Benchmark and data

  • Datasets/tasks: <public, licensed; exact versions/URLs; splits>. Leakage controls: <…>.
  • Baselines: <at least two, including a strong simple one>. Implementation sources (cite).

# 6. Procedure

  1. Environment lock in code/ (uv), deterministic seeds (≥ 5 seeds), fixed splits.
  2. Implement method and baselines; unit-test core functions.
  3. Run the pre-registered grid; log every run with commands, seeds, wall time, resources in results/experiment-log.jsonl.
  4. Aggregate deterministically into results/metrics.json; produce tables/figures with CIs.
  5. code/reproduce.sh regenerates everything.

# 7. Baselines, robustness and falsification

  • Ablations: . Sensitivity: <hyper-parameters, data size>. Report when the method loses.

# 8. Reproducibility and artifacts

  • Pinned environment, seeds, logs, metrics, figure scripts, resource usage table.

# 9. Manuscript

  • AutoScholar class; algorithm pseudo-code; results tables with mean ± CI over seeds; 8–16 pages.

# 10. Review and revision

  • Reviewer focus: unfair baselines, seed variance, leakage, cherry-picked metrics, missing ablations.
  • <Compute limits, dataset licenses, generalization.>

# 12. Acceptance criteria

  • All runs logged and reproducible; ≥ 5 seeds; ≥ 2 baselines; ≥ 2 ablations; reported numbers traceable; ≥ 12 verified citations.
Files · 0d8280a29f83e41fbeda1fded3fa94709eba2e7d
Menu

Theme