1problem_id,statement,categories,why_unresolved,testable_direction,supporting_keys2OP-F01,"Whether paged (non-contiguous) or virtual-memory-backed contiguous KV caches are the better abstraction for LLM serving, and at what cost in programmability, is unsettled.",operating_systems;distributed_computing,"Each design is evaluated by its own authors on specific GPU generations and workloads; no neutral, versioned benchmark across hardware exists.","A shared serving benchmark (fixed traces, several GPU generations, open harness) reporting throughput-per-watt and tail latency for vLLM-style paging, vAttention-style virtual memory and phase-split deployments.",kwon2023vllm;prabhu2025vattention;patel2024splitwise3OP-F02,Whether the memory wall (bandwidth growing slower than FLOPS) can be circumvented by architectural redesign rather than by model-side sparsity/quantization.,computer_architecture;ai_accelerators,The argument rests on hardware trend extrapolation and on serving workloads of 2023-2024; it has not been tested as the installed base changes.,"Track bandwidth-normalised decode throughput across accelerator generations and model families; test whether gains come from bandwidth or from algorithmic sparsity (AWQ, PowerInfer-style locality).",gholami2024memorywall;lin2025awq;song2024powerinfer4OP-F03,"Device-level efficiency figures of photonic and neuromorphic accelerators (e.g., 160 TOPS/W) have not been shown to translate into end-to-end system gains once conversion, memory and precision costs are included.",ai_accelerators;sustainable_computing,Demonstrations are on small tasks or shallow models and report energy at the photonic core; no agreed end-to-end benchmark; abstracts are the only evidence reviewed here.,"An MLPerf-style end-to-end benchmark for analog/photonic accelerators including ADC/DAC and memory energy, with precision-matched digital baselines.",xu2024taichi;ahmed2025photonic;lam2026neuromorphicphotonic;fu2024onn5OP-F04,Whether pre-fault-tolerant quantum processors deliver computational utility beyond the best classical simulation remains contested.,quantum_computing,Utility claims are stated relative to brute-force simulation; improved classical methods move the baseline after publication; the landmark set contains the claim but not its rebuttals.,Preregistered utility experiments with independently committed classical baselines (tensor-network and Clifford-perturbation methods) evaluated by third parties before publication.,kim2023utility;bharti2022nisq6OP-F05,Whether variational quantum algorithms can avoid barren plateaus without losing classical-simulability or expressivity.,quantum_computing,Theoretical results suggest a trade-off between trainability and classical simulability; empirical demonstrations are small-scale.,"Scaling experiments on error-corrected hardware with trainability diagnostics, compared against classical surrogates such as quantum-selected configuration interaction.",larocca2025barren;cerezo2021vqa;kanno2026qsci7OP-F06,"Scaling modular and distributed quantum computing (photonic interconnects, gate teleportation) to rates and fidelities compatible with error correction.",quantum_computing;computer_networks,Demonstrations are two-module or sub-performant scale models; fidelity and entanglement rate at scale are unknown.,"Multi-module logical-qubit experiments with reported entanglement rates, fidelities and decoder latency; comparison of trapped-ion, neutral-atom and photonic modules under a common protocol.",main2025distributedqc;aghaee2025aurora;bluvstein2024logical8OP-F07,There is no agreed protocol for evaluating retrieval-augmented generation that separates the contribution of retrieval quality from parametric memory and from generation.,information_retrieval,"Surveys propose taxonomies, not shared evaluation standards; IR metrics (recall, nDCG) and generation metrics are reported inconsistently.","Benchmarks with held-out, time-stamped corpora and ablations over retriever, index freshness and generator, reporting retrieval metrics and answer faithfulness jointly.",fan2024ragsurvey;huang2026ragsurvey;zhao2026ragaigc;lewis2020rag9OP-F08,Whether vector search belongs in specialised vector databases or as an extension of relational systems is contested.,database_systems,Library-level evidence (Faiss) and surveys exist; workload-level comparisons under realistic transactional and hybrid queries are rare.,"Open benchmarks mixing structured predicates and approximate nearest-neighbour search over billion-scale embeddings, comparing dedicated and extended systems on cost and recall.",douze2026faiss;pan2024vectordb;stonebraker2024goesaround;malkov2020hnsw10OP-F09,"Where LLM inference should sit on the device-edge-cloud continuum under energy, latency and privacy constraints.",edge_computing;computer_networks,The literature is dominated by surveys and visions; measured trade-offs on reproducible testbeds are scarce.,"Testbed studies of split inference (on-device, mobile-edge, cloud) with measured energy per token, latency and accuracy for identical models and traffic.",qu2025meillm;liu2026eccc;zhang2026edgegeneral;an2026aiflow11OP-F10,"The energy, carbon and water footprints of AI inference at fleet scale cannot be measured from public data; rebound effects are untested.",sustainable_computing;cloud_computing,Operators do not disclose AI-specific metrics; estimates rely on models and extrapolation; industry figures quoted in abstracts are not independently verifiable.,Mandatory disclosure of AI workload energy and water by region; benchmarks attributing energy to prefill/decode; natural experiments on efficiency gains and total consumption.,chien2023carbongenai;li2025thirsty;devriesgao2026datacenters;luccioni2025jevons;niu2026tokenpowerbench;chen2026aielectricity12OP-F11,"LLM-based operations tooling (incident root-cause analysis, scheduling) is evaluated on proprietary traces; external validity is unknown.",cloud_computing,Evaluations use one provider's incidents or trace-driven simulations; no public incident corpus with ground-truth causes.,"A public, anonymised incident benchmark with labelled root causes and an evaluation protocol for LLM-assisted diagnosis.",chen2024rcacopilot;liao2026caps;li2026deepservecb13OP-F12,"The classifier assigns scientific software, social-science and health records to systems categories, and the landmark score elevates re-published classics and anomalous citation counts; category-level results inherit this noise.",computer_architecture;operating_systems;distributed_computing;cloud_computing,Rule-based multi-label classification over OpenAlex topics; no venue-stream validation for systems categories in the landmark lists reviewed here.,Venue-based validation (DBLP streams such as OSDI/SOSP/ISCA/SIGCOMM/SIGMOD) of category precision; exclusion of re-publication records from citation-window scores.,priem2022openalex;ley2002dblp14OP-F13,Whether the 2023-2025 output recovery of the classical systems categories reflects LLM-infrastructure research or mean reversion after a 2018-2022 decline.,computer_architecture;operating_systems;distributed_computing;cloud_computing;database_systems;computer_networks,The ITS slope change is significant for all categories but placebo dates perform similarly; break dates are scattered; the landmark lists show LLM-serving papers but their share of each category's volume is not measured.,"Topic-level decomposition of each systems category (LLM-serving, GPU scheduling, vector search subtopics) and their share of category growth; comparison with control fields outside AI.",kwon2023vllm;gholami2024memorywall15